[2024-10-16 14:57:13.163222] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53B9A-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:13.163246] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53B9A-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:13.172338] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061833171234000, start_id=1729061833171234000, end_id=1729061833171234001) [2024-10-16 14:57:13.172825] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] no need to update min/max version(new_version={val:1729061832870698000, v:0}, min_version={val:1729061832870698000, v:0}, max_version={val:1729061832870698000, v:0}) [2024-10-16 14:57:13.184386] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:13.210047] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5870-0-0] [lt=18] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:13.210148] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5870-0-0] [lt=21][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:13.217043] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22455][T1003_TenantWea][T1003][YB4A0A000407-00061E9B3ABCFB2C-0-0] [lt=14] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.125591659545898438e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=306, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.980318913277876618e+03) [2024-10-16 14:57:13.217205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061833216347000, start_id=1729061833216347000, end_id=1729061833216347001) [2024-10-16 14:57:13.219740] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061833219523000, start_id=1729061833219523000, end_id=1729061833219523001) [2024-10-16 14:57:13.220864] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061833220582000, start_id=1729061833220582000, end_id=1729061833220582001) [2024-10-16 14:57:13.221687] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=246, start_service_tstamp_=1722494263598349, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:13.233920] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=22] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209255}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.233994] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=20] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209255}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.237708] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106825][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7605-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=9, avg_log_batch_cnt=0, total_group_log_size=1695, avg_group_log_size=188) [2024-10-16 14:57:13.239132] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1028-0-0] [lt=17] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:13.240514] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1695, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:13.240534] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] succ to submit log task to replay service(to_submit_lsn={lsn:213056747277}, to_submit_scn={val:1729061833118455001, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061833240486, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213056747343}, next_to_submit_scn_:{val:1729061833118455002, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213056747221}, end_lsn:{lsn:213056747343}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061833118455001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061833118455001, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:2757407906, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061833118455001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560288, miss_cnt:1, cache_read_size:152589187325, hit_ratio:"9.999999928346379185e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:13.245552] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.247113] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=12] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209237}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.247170] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=17] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209237}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.251484] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD01-0-0] [lt=7][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:13.253322] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:13.256929] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.257379] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=8] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209246}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.257438] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=19] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209246}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.258931] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=10] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209249}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.258983] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=18] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209249}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.262547] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=24] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=219, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:13.262565] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=86, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:13.264173] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=12] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209243}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.264221] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=17] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209243}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.265991] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53B9C-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:13.266542] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=45] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=31, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:13.266561] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:13.268562] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=8] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209258}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.268614] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=14] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209258}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.269083] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106825][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C39FC-0-0] [lt=20] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=20, avg_log_submit_to_flush_time=404, avg_log_submit_to_slide_time=689) [2024-10-16 14:57:13.269560] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=7] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209231}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.269606] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=15] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209231}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.273916] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061833273352000, start_id=1729061833273352000, end_id=1729061833273352001) [2024-10-16 14:57:13.276332] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=9] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209252}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.276377] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=14] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209252}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.276768] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB8F-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12239, add_timestamp:1729061833276758}) [2024-10-16 14:57:13.276798] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E03-0-0] [lt=35] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12239, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061833276783, sql_renew_time:1729061833276783, is_mark_fail:false}]) [2024-10-16 14:57:13.276817] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E03-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:12239, add_timestamp:1729061833276758}, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061833276783, sql_renew_time:1729061833276783, is_mark_fail:false}]) [2024-10-16 14:57:13.281223] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB8F-0-0] [lt=14][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:13.281406] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C34-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:13.281427] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C34-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061833118455002, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061833118455002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:13.281812] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=7] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209240}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.281855] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=13] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209240}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.282571] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C34-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061833282243000, start_id=1729061833282243000, end_id=1729061833282243001) [2024-10-16 14:57:13.283627] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=7] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209234}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836143346544000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:13.283674] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=19] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:209234}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.283777] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1156546560, global_cache_size=1591787520, tenant_max_wash_size=15178631, tenant_min_wash_size=15178631, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:13.284073] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash memory, (tenant_id=1, cache_size=859357184, lower_mem_limit=2147483648, upper_mem_limit=2147483648, min_wash_size=5555489, max_wash_size=5555489, mem_usage=2015858688, reserve_mem=137180449, wash_size=5555489) [2024-10-16 14:57:13.284087] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash memory, (tenant_id=1002, cache_size=93634560, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:57:13.284319] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=4] Wash time detail, (compute_wash_size_time=92, refresh_score_time=80, wash_time=214) [2024-10-16 14:57:13.286858] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=11] start to print blacklist info [2024-10-16 14:57:13.287008] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=9] ls blacklist refresh finish(cost_time=10866) [2024-10-16 14:57:13.287236] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Cache wash clean map node details(ret=0, clean_node_count=29, clean_time=0, clean_start_pos=3082786, clean_num=62914) [2024-10-16 14:57:13.288364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC692-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061833287612000, start_id=1729061833287612000, end_id=1729061833287612001) [2024-10-16 14:57:13.289091] INFO [STORAGE.TRANS] handle_2pc_prepare_request_impl_ (ob_two_phase_downstream_committer.cpp:350) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=11] committer is under logging(ret=0, *this={this:0x2b0e0c214b50, ref:4, trans_id:{txid:392284891}, tenant_id:1002, is_exiting:false, trans_expired_time:1729062433233262, cluster_version:17180066304, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1729061833233262}{self_ls_id:{id:1001}, session_id:3221770339, part_trans_action:3, pending_write:0, 2pc_role:"LEAF", ctx_tx_data_:{ctx_mgr_:0x2b0e26cb6030, tx_data_guard_:{tx_data:{tx_id:{txid:392284891}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:1729061833008525003, v:0}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:36834273249420035, replay_completeness_:1, upstream_state:"PREPARE", collected:[], rec_log_ts:{val:1729061833008525003, v:0}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1729061833283431, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008951550040 max_table_version=1729008951550040 trans_version={val:1729061833008525001, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284891} ls_id=1001 row_callback[alloc:335, free:0, unsubmit:0] redo[fill:345,sync_succ:248, sync_fail:0] main_list_len=345 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,345,345,248,345,0,0,0)|]}} -{busy_cbs_cnt:1, oldest_busy_cb:{ObTxBaseLogCb:{base_ts:{val:1729061833008525001, v:0}, log_ts:{val:1729061833008525004, v:0}, lsn:{lsn:204021714984}, submit_ts:1729061833289024}, this:0x2b0e0c217aa8, is_inited_:true, trans_id:{txid:392284891}, ls_id:{id:1001}, ctx:0x2b0e0c214b50, tx_data_guard:{tx_data:NULL}, is_callbacked_:false, is_dynamic_:false, mds_range_:{range_array_.count():0, range_array_:[]}, cb_arg_array_:[{log_type:0x1, arg:null}, {log_type:0x40, arg:null}, {log_type:0x80, arg:null}], first_part_scn_:{val:18446744073709551615, v:3}, extra_cb_:null, need_free_extra_cb_:false, callbacks_.count():1}, -{info_log_submitted:1 state_log_submitting:1 state_log_submitted:1 }-, -{downstream_state:"INIT", upstream:{id:1002}, participants:[], redo_log_no:1, scheduler:"10.0.4.8:2890", prepare_version:{val:1729061833008525004, v:0}, trans_type:2, next_log_entry_no:2, max_applied_log_ts:{val:1729061833008525003, v:0}, max_appling_log_ts:{val:1729061833008525003, v:0}, max_applying_part_log_no:0, max_submitted_seq_no:{branch:0, seq:494224}, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, redo_lsns:[{lsn:204021678801}], max_durable_lsn:{lsn:204021678801}, }- }-}) [2024-10-16 14:57:13.289164] INFO [RPC] process (ob_batch_processor.cpp:106) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=72] batch rpc statistics(clog_batch_nodelay_cnt=0, clog_batch_cnt=0, trx_batch_cnt=2, sql_batch_cnt=0) [2024-10-16 14:57:13.289810] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] rpc statistics(total_trans_msg_count=444, total_batch_msg_count=1) [2024-10-16 14:57:13.290226] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1003}, tablet_meta.tablet_id_={id:209254}) [2024-10-16 14:57:13.290244] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] success to init medium clog recorder(ret=0, ls_id={id:1003}, tablet_id={id:209254}, max_saved_version=0) [2024-10-16 14:57:13.290253] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=8] success to init storage recorder(ret=0, ls_id={id:1003}, tablet_id={id:209254}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:57:13.290269] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778310817000, v:0}, new_clog_checkpoint_scn_:{val:1729061778310817000, v:0}, for_replay_:true, for_inc_direct_load_:false}, ret=0, ls_id={id:1003}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e6b412a50, key:{tablet_id:{id:209254}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778310817000, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061833290267}, ls_id_:{id:1003}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061833290267, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e27a120d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f1f1451b0, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e6b412a50, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1003}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f1f1451b0, ref_cnt:2, is_inited:true, tablet_id:{id:209254}, freezer:0x2b0e27a120d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e6b412a50, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e27a0ced0}, ls_id:{id:1003}, tablet_id:{id:209254}}}) [2024-10-16 14:57:13.290315] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=46] succeeded to create memtable for tablet(ret=0, ls_id={id:1003}, tablet_id={id:209254}, clog_checkpoint_scn={val:1729061778310817000, v:0}, schema_version=0, for_replay=true) [2024-10-16 14:57:13.290325] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] table store update memtable success(ret=0, ls_id={id:1003}, tablet_id={id:209254}, table_store_addr={ptr:0x2b0e664dc6b0, addr:[7663857-6566-0-2305843508505706496-1](offset=2064384,size=465,type=2,seq=1)}, this=0x2b0e664dc0a0) [2024-10-16 14:57:13.290631] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C34-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061833289823000, start_id=1729061833289823000, end_id=1729061833289823001) [2024-10-16 14:57:13.303043] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C084E-0-0] [lt=5] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209285}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833287612000) [2024-10-16 14:57:13.303117] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C084E-0-0] [lt=25] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209285}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.303315] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106825][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E33-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=14, avg_log_batch_cnt=0, total_group_log_size=51046, avg_group_log_size=3646) [2024-10-16 14:57:13.304410] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [22251][T1002_SSTableGC][T1002][YB4A0A000407-00061E997A3CB6A4-0-0] [lt=9] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60379}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015250305282004}, old_min_reserved_snapshot=1729060030183137000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060020174728000) [2024-10-16 14:57:13.304993] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C084F-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1002, table_id:10009, add_timestamp:1729061833304983}) [2024-10-16 14:57:13.305407] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C084F-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301173800, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893127, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.307162] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4890, mem_used=10398720) [2024-10-16 14:57:13.310538] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:13.310806] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:13.310925] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FB1A-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=767, avg_log_submit_to_slide_time=985) [2024-10-16 14:57:13.319653] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E04-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1002, table_id=10009, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061833319640, sql_renew_time:1729061833319640, is_mark_fail:false}]) [2024-10-16 14:57:13.319676] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E04-0-0] [lt=22] success to process renew task(task={tenant_id:1002, table_id:10009, add_timestamp:1729061833304983}, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061833319640, sql_renew_time:1729061833319640, is_mark_fail:false}]) [2024-10-16 14:57:13.320167] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:13.320699] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:13.321197] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:13.321215] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:13.323017] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0850-0-0] [lt=11] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209267}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833287612000) [2024-10-16 14:57:13.323078] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0850-0-0] [lt=15] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209267}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.332333] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0851-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301200727, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893128, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.332552] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [22251][T1002_SSTableGC][T1002][YB4A0A000407-00061E997A3CB6A4-0-0] [lt=23] SSTableGCTask(cost_ts=28316) [2024-10-16 14:57:13.334107] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=9] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:13.334262] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0853-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301202657, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893129, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.337186] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2239-0-0] [lt=17] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209276}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833287612000) [2024-10-16 14:57:13.337261] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2239-0-0] [lt=23] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209276}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.337665] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1A9F-0-0] [lt=16] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:13.337680] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1A9F-0-0] [lt=11] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=16) [2024-10-16 14:57:13.339358] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0856-0-0] [lt=29][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301207752, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893130, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.345619] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.346222] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0857-0-0] [lt=16] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209279}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833287612000) [2024-10-16 14:57:13.346294] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0857-0-0] [lt=20] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209279}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.348661] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0858-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301217052, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893131, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.353432] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C085A-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301221823, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893132, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.354251] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=38, avg_log_submit_to_flush_time=672, avg_log_submit_to_slide_time=912) [2024-10-16 14:57:13.356997] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.358461] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C085B-0-0] [lt=19] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209273}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833287612000) [2024-10-16 14:57:13.358540] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C085B-0-0] [lt=23] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209273}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.360483] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C085C-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301228876, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893133, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.364863] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C085E-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301233257, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893134, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.364943] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8EE6-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:13.364952] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8EE6-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11299, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:13.364956] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8EE6-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=1059, avg_log_submit_to_slide_time=11292) [2024-10-16 14:57:13.365114] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:13.371301] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C085F-0-0] [lt=15] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209288}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833287612000) [2024-10-16 14:57:13.371372] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C085F-0-0] [lt=26] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209288}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.372428] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061833371669000, start_id=1729061833371669000, end_id=1729061833371669001) [2024-10-16 14:57:13.374756] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0860-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301243149, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893135, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.376751] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0862-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301245144, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893136, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.381480] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=6] query dest_info_ succ(dest_info=[], archive_dest_changed=false) [2024-10-16 14:57:13.381495] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=14] finish to resize log external storage handler(current_ts=1729061833381494, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:13.383082] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0864-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301251476, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893137, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.383143] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:13.384461] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:13.385279] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0865-0-0] [lt=13] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209261}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833287612000) [2024-10-16 14:57:13.385346] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0865-0-0] [lt=23] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209261}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.387148] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0866-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301255542, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893138, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.404955] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0868-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301273348, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893139, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.408768] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3824-0-0] [lt=14] MediumCheckTask(cost_ts=8) [2024-10-16 14:57:13.413237] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C086A-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301281630, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893140, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.413639] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C086B-0-0] [lt=18] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209282}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833287612000) [2024-10-16 14:57:13.413694] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C086B-0-0] [lt=16] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209282}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.415565] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C086C-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301283958, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893141, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.417423] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061833416699000, start_id=1729061833416699000, end_id=1729061833416699001) [2024-10-16 14:57:13.419959] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061833418823000, start_id=1729061833418823000, end_id=1729061833418823001) [2024-10-16 14:57:13.420961] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061833420049000, start_id=1729061833420049000, end_id=1729061833420049001) [2024-10-16 14:57:13.426319] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=12] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:13.426340] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:14538453778}, scn={val:1729061833118455002, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061833253641, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538453778}, next_to_submit_scn_:{val:1729061833118455002, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538453656}, end_lsn:{lsn:14538453778}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061833118455001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061833118455001, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:234325918, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061833118455001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62649933, miss_cnt:32, cache_read_size:8050382863, hit_ratio:"9.999994892255725798e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:13.426383] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=39] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:13.429681] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C086E-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301298076, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893142, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.437672] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0870-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301306064, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893143, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.437773] INFO [LIB] stat (utility.h:1165) [106825][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7607-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:13.437784] INFO [LIB] stat (utility.h:1165) [106825][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7607-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22766, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:13.437793] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106825][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7607-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220937250, log_task={header:{begin_lsn:{lsn:213056798146}, end_lsn:{lsn:213056798318}, log_id:220937250, min_scn:{val:1729061833287612002, v:0}, max_scn:{val:1729061833287612002, v:0}, data_len:116, proposal_id:88, prev_lsn:{lsn:213056797867}, prev_proposal_id:88, committed_end_lsn:{lsn:213056798146}, data_checksum:1281358363, accum_checksum:685259284, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061833295011, freeze_ts:1729061833295011, submit_ts:1729061833295024, flushed_ts:1729061833296284, gen_to_freeze cost time:0, gen_to_submit cost time:13, submit_to_flush cost time:1260}, fs_cb_begin_ts=1729061833437766, log_life_time=142755) [2024-10-16 14:57:13.440203] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A63-0-0] [lt=14] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:13.440220] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A63-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:13.443606] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0872-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301311994, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893144, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.445682] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.447688] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=6][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:13.449576] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0874-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301317969, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893145, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.453130] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A1F-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178174, log_task={header:{begin_lsn:{lsn:14538453778}, end_lsn:{lsn:14538453900}, log_id:115178174, min_scn:{val:1729061833118455002, v:0}, max_scn:{val:1729061833118455002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538453656}, prev_proposal_id:23, committed_end_lsn:{lsn:14538453778}, data_checksum:1715074492, accum_checksum:773308204, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061833353057, freeze_ts:1729061833353057, submit_ts:1729061833353060, flushed_ts:1729061833354248, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:1188}, fs_cb_begin_ts=1729061833453124, log_life_time=100067) [2024-10-16 14:57:13.457059] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.458693] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0876-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301327086, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893146, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.459644] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79CE-0-0] [lt=9] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061833450810, keepalive_ts:-1}) [2024-10-16 14:57:13.459660] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79CE-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061833450810, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:13.462286] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3666) [2024-10-16 14:57:13.470134] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C087B-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301338525, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893147, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.472050] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C087D-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301340445, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893148, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.473219] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] no need to update min/max version(new_version={val:1729061833070584000, v:0}, min_version={val:1729061833070584000, v:0}, max_version={val:1729061833070584000, v:0}) [2024-10-16 14:57:13.474864] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:13.474875] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:13.474886] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:13.474881] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A1F-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:13.474892] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:13.474896] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:13.474898] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:13.474895] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A1F-0-0] [lt=12] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:13.474901] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:13.474907] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:13.474910] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:13.474913] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:13.474915] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:13.480328] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C087F-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301348720, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893149, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.480336] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=35] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:13.482009] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0881-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301350405, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893150, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.483691] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0883-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301352086, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893151, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.487540] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1175420928, global_cache_size=1583464448, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:13.487622] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0885-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301356018, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893152, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.487692] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:57:13.487704] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:13.487718] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70391625, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:13.491249] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=4] Garbage Collector is running(seq_=656492, gc_interval=10000000) [2024-10-16 14:57:13.491282] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=15] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:13.491293] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:13.491598] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:13.491874] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=19] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:13.491887] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] gc_check_member_list_ cost time(ret=0, time_us=624) [2024-10-16 14:57:13.491899] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:13.492841] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:13.493565] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=15] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:13.494227] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=13] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:13.494842] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=14] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:13.494858] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=13] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:13.494866] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:13.494871] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:13.494880] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:13.494888] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] execute_gc cost time(ret=0, time_us=31) [2024-10-16 14:57:13.494892] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:13.497342] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0887-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301365737, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893153, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.499994] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:13.500733] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=10] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:13.500765] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=32] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:13.500789] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=19] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:13.500863] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=12] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061845590307000, latest_log_ts:{val:1729061828964666001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:13.500912] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=45] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:13.500933] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=18] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:13.500943] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=0, rec_log_ts={val:1729061828964666001, v:0}) [2024-10-16 14:57:13.500951] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:13.500954] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:13.500964] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:13.500977] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:13.500988] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:13.500994] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:13.502673] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0889-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301371068, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893154, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.504223] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C088B-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301372619, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893155, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.504909] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106635][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47CF-0-0] [lt=32] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061833472691, keepalive_ts:-1}) [2024-10-16 14:57:13.504924] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106635][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47CF-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061833472691, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:13.505976] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C088D-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301374371, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893156, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.509227] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061833508923000, start_id=1729061833508923000, end_id=1729061833508923001) [2024-10-16 14:57:13.509305] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=16] submit log success(service_type_=0, last_id=1729061850098662000, limited_id=1729061850098662000) [2024-10-16 14:57:13.509312] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=6] submit log with lock success(service_type_=0, last_id=1729061850098662000, limited_id=1729061850098662000) [2024-10-16 14:57:13.509459] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=3] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061833509454], gts:1729061833508923000, latest_srr:[mts=1729061833509454]}) [2024-10-16 14:57:13.509797] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D51-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:13.509841] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D51-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061833118455003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061833118455003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:13.509921] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=9] update id meta(ret=0, service_type=0, limited_id=1729061850098662000, latest_scn={val:1729061833508923001, v:0}, *this={tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061830505634001, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850098662000, latest_log_ts:{val:1729061833508923001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:13.509956] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=35] handle submit callback(service_type_=0, success=true, used_ts=652, log_ts={val:1729061833508923001, v:0}, limited_id=1729061850098662000) [2024-10-16 14:57:13.509963] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:13.509967] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:13.519031] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D51-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061833518948000, start_id=1729061833518948000, end_id=1729061833518948001) [2024-10-16 14:57:13.522223] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C088F-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301390617, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893157, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.523550] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D51-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061833523194000, start_id=1729061833523194000, end_id=1729061833523194001) [2024-10-16 14:57:13.524841] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0891-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301393236, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893158, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.528905] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0893-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301397300, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893159, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.532507] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0895-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301400902, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893160, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.541233] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0897-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301409628, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893161, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.545741] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.548809] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0899-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301417202, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893162, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.552991] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C089B-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301421385, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893163, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.554515] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=381, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:13.554533] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=471, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:13.554567] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:13.557127] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.558423] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C089D-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301426816, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893164, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.562355] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:13.562380] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:13.563635] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106635][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45AC0-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2958) [2024-10-16 14:57:13.563952] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:13.564284] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:13.564301] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:13.564368] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=20] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:13.564378] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=765, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:13.569249] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.569423] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.569550] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.569900] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C089F-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301438294, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893165, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.570115] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.572499] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061833571748000, start_id=1729061833571748000, end_id=1729061833571748001) [2024-10-16 14:57:13.574031] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061833572811000, start_id=1729061833572811000, end_id=1729061833572811001) [2024-10-16 14:57:13.574312] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3080) [2024-10-16 14:57:13.575330] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] worker thread created(tenant_->id()=1001, group_id_=1, token=1) [2024-10-16 14:57:13.575307] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [106951][][T1001][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1001, ret=0, thread_count_=172) [2024-10-16 14:57:13.575387] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [106951][][T1001][Y0-0000000000000000-0-0] [lt=78] Init thread local success [2024-10-16 14:57:13.576867] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08A1-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301445261, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893166, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.577145] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1003}, {id:1002}, {id:1}]) [2024-10-16 14:57:13.577179] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=10] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:57:13.580754] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08A3-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301449149, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893167, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.581633] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26148][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB19CC6E9-0-0] [lt=20] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:13.582850] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08A4-0-0] [lt=16] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209270}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833508923000) [2024-10-16 14:57:13.582912] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08A4-0-0] [lt=22] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209270}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.584543] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:13.584661] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08A5-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301452995, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893168, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.594858] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08A7-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301463250, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893169, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.595273] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=26] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4377, mem_used=8318976) [2024-10-16 14:57:13.603401] INFO [LIB] stat (utility.h:1165) [106825][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E39-0-0] [lt=23] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:13.603727] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=223, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:13.603750] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=117, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:13.603795] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:13.603924] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106825][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A0C-0-0] [lt=26] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=51046) [2024-10-16 14:57:13.603991] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=51046, hit_cnt=13, read_cnt=13, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:13.609416] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08A9-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301477808, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893170, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.610383] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:13.610398] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] try gc retain ctx [2024-10-16 14:57:13.610403] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] try to retry start_working [2024-10-16 14:57:13.610428] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] get_max_applied_scn(scn={val:1729061833508923000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493424654}, palf_committed_end_scn_:{val:1729061833508923000, v:0}, last_check_scn_:{val:1729061833508923000, v:0}, max_applied_cb_scn_:{val:1729061833508923000, v:0}}) [2024-10-16 14:57:13.610451] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:13.611333] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=54] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061830506702000, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061840721265000, latest_log_ts:{val:1729061824110629001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:13.611485] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=69] get_max_applied_scn(scn={val:1729061833508923000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645384916}, palf_committed_end_scn_:{val:1729061833508923000, v:0}, last_check_scn_:{val:1729061833508923000, v:0}, max_applied_cb_scn_:{val:1729061833508923000, v:0}}) [2024-10-16 14:57:13.611504] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:13.612282] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=53] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061830506702000, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:13.613178] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=52] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061830506702000, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:13.613247] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=59] get_min_unreplayed_log_info(lsn={lsn:14352340380}, scn={val:1729061833118455006, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061833564433, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352340380}, next_to_submit_scn_:{val:1729061833118455006, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352340258}, end_lsn:{lsn:14352340380}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061833118455005, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061833118455005, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:900601392, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061833118455005, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656671, miss_cnt:17, cache_read_size:8050130601, hit_ratio:"9.999997286802009278e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:13.614061] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=45] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061830506702000, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:13.615596] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08AB-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301483990, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893171, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.617454] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061833616959000, start_id=1729061833616959000, end_id=1729061833616959001) [2024-10-16 14:57:13.619504] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B8B2-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061833619065000, start_id=1729061833619065000, end_id=1729061833619065001) [2024-10-16 14:57:13.619659] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B8B5-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061833619065000, start_id=1729061833619065000, end_id=1729061833619065001) [2024-10-16 14:57:13.619759] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B8B6-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061833619065000, start_id=1729061833619065000, end_id=1729061833619065001) [2024-10-16 14:57:13.619788] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0B3F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061833619065000, start_id=1729061833619065000, end_id=1729061833619065001) [2024-10-16 14:57:13.619985] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B8B3-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061833619065000, start_id=1729061833619065000, end_id=1729061833619065001) [2024-10-16 14:57:13.620096] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061833619065000, start_id=1729061833619065002, end_id=1729061833619065003) [2024-10-16 14:57:13.621224] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061833620272000, start_id=1729061833620272000, end_id=1729061833620272001) [2024-10-16 14:57:13.621696] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [106951][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F92E1-0-0] [lt=4] init point thread id with(&point=0x55e3435c2040, idx_=455, point=[thread id=106951, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=106951) [2024-10-16 14:57:13.623388] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08AD-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301491783, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893172, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.632588] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08AF-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301500982, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893173, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.636969] INFO [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:262) [22249][T1002_ArcSender][T1002][YB4A0A000407-00061E996F9960C1-0-0] [lt=14] ObArchiveSender is running(thread_index=1) [2024-10-16 14:57:13.637554] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08B1-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301505948, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893174, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.641763] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:33216819320}, base_lsn={lsn:33149755392}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:13.641823] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=59] get_replay_process success(id={id:1}, replayed_log_size=67063928, unreplayed_log_size=0) [2024-10-16 14:57:13.641841] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=63, last_replayed_log_size(MB)=63, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:13.645806] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.646491] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:13.646504] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=11] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=101, cache_node_num=101) [2024-10-16 14:57:13.646511] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=101, cache_node_num=101) [2024-10-16 14:57:13.646516] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:13.648407] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=9] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:13.648608] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=101, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}]) [2024-10-16 14:57:13.648735] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=125] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:13.651099] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08B3-0-0] [lt=31][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301519493, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893175, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.652262] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=255, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:13.652277] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=255, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:13.652320] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:13.657185] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.663961] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=242, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:13.663982] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=792, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:13.664021] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:13.665309] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=20] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:13.665400] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08B8-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301533794, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893176, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.666134] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061833664353000, start_id=1729061833664353000, end_id=1729061833664353001) [2024-10-16 14:57:13.670028] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08BA-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301538421, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893177, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.673521] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08BC-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301541916, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893178, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.674919] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08BE-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301543315, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893179, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.676401] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=9] report RowHolderMapper summary info(count=0, bkt_cnt=311) [2024-10-16 14:57:13.683004] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08C0-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301551399, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893180, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.684407] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08C2-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301552803, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893181, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.685723] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08C4-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301554119, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893182, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.687197] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08C6-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301555593, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893183, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.687837] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1171226624, global_cache_size=1585545216, tenant_max_wash_size=7525990, tenant_min_wash_size=7525990, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:13.690841] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08C8-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301559237, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893184, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.694994] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:13.695816] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08CA-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301564211, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893185, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.697578] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08CC-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301565974, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893186, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.698954] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08CE-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301567349, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893187, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.703040] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08D0-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301571436, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893188, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.704832] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08D1-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301573226, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893189, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.706634] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08D3-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301575030, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893190, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.708362] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08D5-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301576758, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893191, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.709701] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08D7-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301578096, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893192, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.710505] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=48] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:13.710524] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:13.711233] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08D9-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301579629, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893193, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.712733] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08DB-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301581129, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893194, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.715991] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15C6-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:13.716071] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15C8-0-0] [lt=15] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:13.718018] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257642, offset=38999161}, end_cursor:ObLogCursor{file_id=197, log_id=49257643, offset=38999444}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a20050, len:2951, log_data_len:283, seq:49257642, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:13.719948] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:13.720192] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB353-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061833719742000, start_id=1729061833719742000, end_id=1729061833719742001) [2024-10-16 14:57:13.721213] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=23] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257643, offset=38999444}, end_cursor:ObLogCursor{file_id=197, log_id=49257644, offset=38999726}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a64050, len:2668, log_data_len:282, seq:49257643, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:13.723304] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257644, offset=38999726}, end_cursor:ObLogCursor{file_id=197, log_id=49257645, offset=39000008}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2924050, len:2386, log_data_len:282, seq:49257644, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:13.723487] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=35] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257645, offset=39000008}, end_cursor:ObLogCursor{file_id=197, log_id=49257646, offset=39000290}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2928050, len:2104, log_data_len:282, seq:49257645, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:13.723533] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62649937, miss_cnt:32, cache_read_size:8050383351, hit_ratio:"9.999994892256052204e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:13.723556] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656672, miss_cnt:17, cache_read_size:8050130723, hit_ratio:"9.999997286802052576e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:13.726479] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601276, offset=9594206}, end_cursor:ObLogCursor{file_id=192, log_id=47601277, offset=9594490}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29dc050, len:2722, log_data_len:284, seq:47601276, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:13.728192] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA488-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061833727280000, start_id=1729061833727280000, end_id=1729061833727280001) [2024-10-16 14:57:13.728251] INFO [LIB] stat (utility.h:1165) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA488-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=21, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:13.728264] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA488-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9328, avg_group_log_size=666) [2024-10-16 14:57:13.728272] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA488-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9328) [2024-10-16 14:57:13.728279] INFO [LIB] stat (utility.h:1165) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA488-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:13.728610] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=248, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:13.728622] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9328) [2024-10-16 14:57:13.728631] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=82, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:13.728645] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9328) [2024-10-16 14:57:13.728655] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:13.728788] INFO [LIB] stat (utility.h:1165) [106951][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F92E2-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:13.728810] INFO [LIB] stat (utility.h:1165) [106951][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F92E2-0-0] [lt=22] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=1007, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:13.728820] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106951][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F92E2-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=418, avg_log_submit_to_slide_time=978) [2024-10-16 14:57:13.729272] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA864-0-0] [lt=10] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:13.729291] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA864-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061833287612005, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061833287612005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:13.729569] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601277, offset=9594490}, end_cursor:ObLogCursor{file_id=192, log_id=47601278, offset=9594764}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a84050, len:2438, log_data_len:274, seq:47601277, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:13.730537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA864-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061833729446000, start_id=1729061833729446000, end_id=1729061833729446001) [2024-10-16 14:57:13.730953] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08DD-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301599347, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893195, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.733211] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA864-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061833732862000, start_id=1729061833732862000, end_id=1729061833732862001) [2024-10-16 14:57:13.734274] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=17] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:13.735041] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:13.736384] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08DF-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301604768, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893196, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.736507] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601278, offset=9594764}, end_cursor:ObLogCursor{file_id=192, log_id=47601279, offset=9595038}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a10050, len:2164, log_data_len:274, seq:47601278, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:13.736981] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601279, offset=9595038}, end_cursor:ObLogCursor{file_id=192, log_id=47601280, offset=9595312}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29b0050, len:1890, log_data_len:274, seq:47601279, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:13.737070] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560295, miss_cnt:1, cache_read_size:152589238666, hit_ratio:"9.999999928346382516e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:13.737096] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137597891, miss_cnt:9, cache_read_size:149776497439, hit_ratio:"9.999999345920250748e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:13.738075] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08E1-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301606471, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893197, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.738164] INFO [LIB] stat (utility.h:1165) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE760D-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:13.739666] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08E3-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301608063, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893198, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.741245] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061829738202, last_first_register_time_us=1722494157071428, register_time_us=1729061829738202, curr_time_us=1729061833741244) [2024-10-16 14:57:13.741284] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061833741264, last_submit_register_req_time_us=1729061833741264, register_parent_reason=3) [2024-10-16 14:57:13.741294] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:13.743215] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=25] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}) [2024-10-16 14:57:13.745866] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=35][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.747116] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08E5-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301615510, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893199, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.748347] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:13.748372] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=12] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:13.748379] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:13.748542] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08E7-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301616938, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893200, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.752003] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:13.752019] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:13.752296] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08E9-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301620691, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893201, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.752472] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=17] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:13.752747] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=51928) [2024-10-16 14:57:13.752777] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=29] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=51928) [2024-10-16 14:57:13.753783] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8EED-0-0] [lt=27] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:13.757248] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.759136] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08EB-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301627531, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893202, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.762022] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08ED-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301630418, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893203, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.767858] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:13.767937] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08EF-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301636332, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893204, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.772542] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061833771367000, start_id=1729061833771367000, end_id=1729061833771367001) [2024-10-16 14:57:13.774579] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061833772862000, start_id=1729061833772862000, end_id=1729061833772862001) [2024-10-16 14:57:13.776214] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08F1-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301644608, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893205, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.776649] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1001, new_min_version={val:1729061833571748000, v:0}, new_max_version={val:1729061833571748000, v:0}, last_min_version={val:1729061833371669000, v:0}, last_max_version={val:1729061833371669000, v:0}, total_time_us=4918, delta_us=204899) [2024-10-16 14:57:13.776666] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1001, current_version={val:1729061833571748000, v:0}, min_version={val:1729061833571748000, v:0}, max_version={val:1729061833571748000, v:0}, in_service=true, leader_epoch=33705) [2024-10-16 14:57:13.777858] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=17] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:13.779615] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08F3-0-0] [lt=35][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301647996, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893206, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.782971] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08F5-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301651367, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893207, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.784592] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:13.788262] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08F7-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301656657, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893208, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.788649] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788653] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788672] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788688] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788703] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788718] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788728] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788754] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788894] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788911] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.788950] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:13.791362] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08F9-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301659758, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893209, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.803621] INFO [LIB] stat (utility.h:1165) [106825][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E3D-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:13.803639] INFO [LIB] stat (utility.h:1165) [106825][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E3D-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=16556, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:13.810802] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=540, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:13.810826] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=41, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:13.812920] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08FB-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301681312, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893210, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.814475] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08FD-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301682870, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893211, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.815010] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1003}, {id:1}, {id:1002}]) [2024-10-16 14:57:13.817547] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061833816711000, start_id=1729061833816711000, end_id=1729061833816711001) [2024-10-16 14:57:13.818905] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C08FF-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301687301, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893212, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.820351] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0901-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301688746, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893213, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.821208] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061833819951000, start_id=1729061833819951000, end_id=1729061833819951001) [2024-10-16 14:57:13.821220] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061833819951000, start_id=1729061833819951000, end_id=1729061833819951001) [2024-10-16 14:57:13.824058] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0905-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061833823455000, start_id=1729061833823455000, end_id=1729061833823455001) [2024-10-16 14:57:13.826840] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:57:13.826856] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:13.826872] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52125383, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:13.837893] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0909-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301706269, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893214, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.838142] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=26, stat_interval=1000000, avg (count/cost)=106, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:13.838162] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=26, stat_interval=1000000, avg (count/cost)=135, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:13.838195] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=256, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:13.838205] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=346, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:13.838271] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:13.844913] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C090B-0-0] [lt=47][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301713307, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893215, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.845938] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.848867] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2675, mem_used=6239232) [2024-10-16 14:57:13.852072] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=51928, avg_group_log_size=3709) [2024-10-16 14:57:13.852086] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=51928) [2024-10-16 14:57:13.853451] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A27-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:13.856437] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=42] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:13.857106] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:13.857158] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061827226807000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:13.857199] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=39] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061827226807000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1003, freeze_info:{latest_snapshot_gc_scn:{val:1729061827226807000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}}) [2024-10-16 14:57:13.857359] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.860610] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:13.861987] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C090D-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301730380, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893216, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.865737] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C090F-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301734132, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893217, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.867477] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0911-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301735873, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893218, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.868495] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061833868494, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007616, add_bytes: 0 [2024-10-16 14:57:13.869636] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0912-0-0] [lt=20] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:209264}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1728836144450006000, delete_commit_version:-1}, snapshot_version=1729061833823455000) [2024-10-16 14:57:13.869688] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0912-0-0] [lt=17] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:209264}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:13.871307] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0913-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301739703, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893219, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.872854] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0915-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301741251, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893220, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.876167] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 615.50; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 18165.63, iops: 468.17, rt: 189.80; ) [2024-10-16 14:57:13.876190] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:13.876234] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 177.25; ) [2024-10-16 14:57:13.876243] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:13.876255] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:13.878809] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0917-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301747205, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893221, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.880106] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0919-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301748502, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893222, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.882591] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=14] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:13.883459] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:13.888116] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1167032320, global_cache_size=1589706752, tenant_max_wash_size=11720294, tenant_min_wash_size=11720294, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:13.888413] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash memory, (tenant_id=1002, cache_size=95715328, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=11720294, max_wash_size=11720294, mem_usage=978087936, reserve_mem=107374182, wash_size=11720294) [2024-10-16 14:57:13.888538] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash time detail, (compute_wash_size_time=101, refresh_score_time=72, wash_time=215) [2024-10-16 14:57:13.888578] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=8] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=0, clean_start_pos=3145700, clean_num=62914) [2024-10-16 14:57:13.899298] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C091B-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301767690, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893223, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.901073] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C091D-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301769469, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893224, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.902466] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C091F-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301770862, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893225, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.911418] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0921-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301779813, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893226, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.915895] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0923-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301784290, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893227, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.917349] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0925-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301785745, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893228, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.919005] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:13.928830] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0927-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301797224, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893229, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.935748] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:185) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=13] sql audit evict mem start(request_manager_->get_tenant_id()=1002, evict_low_mem_level=0, evict_high_mem_level=16106127, size_used=7423, mem_used=16637952) [2024-10-16 14:57:13.936545] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:191) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] release old cannot free more memory [2024-10-16 14:57:13.936553] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=7] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=3, elapse_time=805, size_used=0, mem_used=2079744) [2024-10-16 14:57:13.944845] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061833944843, bytes: 4446786078505, bw: 0.537132 MB/s, add_ts: 1000027, add_bytes: 563239 [2024-10-16 14:57:13.945985] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:13.946004] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:13.946011] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.952082] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=29, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:13.952102] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:13.952397] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23239561, this={dio_aligned_buf:{buf_write_offset:3075, buf_padding_size:1021, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:93866168, truncate_used_ts:39944582}, log_block_size:67108864, dir_fd:173, io_fd:196}, aligned_buf_len=4096, aligned_block_offset=23236608, buf_len=122, total_write_size_=62758, total_write_size_after_dio_=483328, ob_pwrite_used_ts=29686, count=103, each_pwrite_cost=288) [2024-10-16 14:57:13.952483] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=298, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:13.952503] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=255, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:13.952517] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1226) [2024-10-16 14:57:13.952764] INFO [LIB] stat (utility.h:1165) [106825][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A16-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:13.952776] INFO [LIB] stat (utility.h:1165) [106825][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A16-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=901, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:13.953397] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A29-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:13.954275] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0929-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301822669, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893230, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.955744] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C092B-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301824139, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893231, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.957379] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:13.960816] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=9] worker thread began to shrink(tenant_->id()=1002, group_id_=1, token=0) [2024-10-16 14:57:13.961105] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C092D-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301829500, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893232, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.962700] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C092F-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301831095, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893233, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.962889] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106825][T1002_L0_G1][T1002][Y0-0000000000000000-0-0] [lt=4] worker thread exit(tenant_->id()=1002, workers_.get_size()=3) [2024-10-16 14:57:13.962903] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106825][T1002_L0_G1][T0][Y0-0000000000000000-0-0] [lt=10] tenant thread end_run(id_=1002, ret=0, thread_count_=191) [2024-10-16 14:57:13.971819] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0931-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301840213, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893234, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.973427] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061833971891000, start_id=1729061833971891000, end_id=1729061833971891001) [2024-10-16 14:57:13.975111] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061833974627000, start_id=1729061833974627000, end_id=1729061833974627001) [2024-10-16 14:57:13.979309] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0933-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301847703, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893235, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.981366] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0935-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301849761, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893236, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.984676] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:13.986770] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0937-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301855165, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893237, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.988339] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0939-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301856734, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893238, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.989690] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C093B-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301858087, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893239, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:13.992319] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:13.992342] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=206) [2024-10-16 14:57:13.996255] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [22325][T1003_COMMONLSS][T1003][YB4A0A000407-00061E9B3A19CAF8-0-0] [lt=17][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:13.998557] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=14] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:14.003076] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:14.003816] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.003845] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=29] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.003866] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:14.004002] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823001, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850098662000, latest_log_ts:{val:1729061833508923001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:14.004049] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823001, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850098662000, latest_log_ts:{val:1729061833508923001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:14.004083] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=29] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:14.004090] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061833508923001, v:0}) [2024-10-16 14:57:14.004094] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:14.004101] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:14.004111] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:14.004127] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:14.004829] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.004846] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.004858] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:14.004896] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823001, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:14.004926] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=29] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823001, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:14.004951] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=23] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:14.004957] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:14.004963] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:14.004969] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:14.005546] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0940-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301873940, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893240, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.005669] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.005684] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.005695] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:14.005730] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823001, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:14.005759] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823001, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:14.005782] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:14.005788] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:14.005794] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:14.005800] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:14.006497] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.006509] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.006518] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:14.006564] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823001, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:14.006593] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823001, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:14.006625] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:14.006631] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:14.006638] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:14.006643] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:14.006647] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:14.006915] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=51046) [2024-10-16 14:57:14.009685] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061834008796000, start_id=1729061834008796000, end_id=1729061834008796001) [2024-10-16 14:57:14.009710] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061834009678], gts:1729061834008796000, latest_srr:[mts=1729061834009678]}) [2024-10-16 14:57:14.009723] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:14.010746] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=28, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:14.010764] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:14.011215] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1636) [2024-10-16 14:57:14.011251] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:14.011255] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:14.015070] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0942-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301883462, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893241, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.016819] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0944-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301885212, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893242, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.017697] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061834017394000, start_id=1729061834017394000, end_id=1729061834017394001) [2024-10-16 14:57:14.018111] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9575) [2024-10-16 14:57:14.019449] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0946-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301887844, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893243, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.020893] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0948-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301889288, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893244, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.021386] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061834019523000, start_id=1729061834019523000, end_id=1729061834019523001) [2024-10-16 14:57:14.021882] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061834019523000, start_id=1729061834019523000, end_id=1729061834019523001) [2024-10-16 14:57:14.031172] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C094A-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301899565, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893245, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.035362] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] Garbage Collector is running(seq_=656489, gc_interval=10000000) [2024-10-16 14:57:14.035400] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=17] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:14.035410] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:14.036751] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:14.037040] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=17] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:14.037053] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] gc_check_member_list_ cost time(ret=0, time_us=1673) [2024-10-16 14:57:14.037061] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=1) [2024-10-16 14:57:14.037766] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=4] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:14.038108] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C094C-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301906495, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893246, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.038437] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=10] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:14.039048] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:14.039618] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:14.039629] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:14.039637] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:14.039643] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:14.039648] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=4] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:14.039653] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=24) [2024-10-16 14:57:14.039657] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=2] ObSafeDestroyHandler start process [2024-10-16 14:57:14.039660] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:133) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-10-16 14:57:14.039684] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:140) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-10-16 14:57:14.039884] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7613-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=52073) [2024-10-16 14:57:14.039905] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7613-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220937256, log_task={header:{begin_lsn:{lsn:213056798928}, end_lsn:{lsn:213056799050}, log_id:220937256, min_scn:{val:1729061833619065003, v:0}, max_scn:{val:1729061833619065003, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213056798806}, prev_proposal_id:88, committed_end_lsn:{lsn:213056798928}, data_checksum:1715074492, accum_checksum:1156047707, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061833937803, freeze_ts:1729061833937803, submit_ts:1729061833937809, flushed_ts:1729061833939639, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:1830}, fs_cb_begin_ts=1729061834039899, log_life_time=102096) [2024-10-16 14:57:14.040097] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=52073) [2024-10-16 14:57:14.041637] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C094E-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301910031, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893247, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.044409] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=40] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:14.045029] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:14.045080] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=14] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061830183137000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:14.046047] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.046954] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0950-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301915349, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893248, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.048507] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0952-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301916903, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893249, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.052184] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0954-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301920579, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893250, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.052619] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1348) [2024-10-16 14:57:14.052637] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:14.053859] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8EF3-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:14.053870] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8EF3-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=10008, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:14.053875] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8EF3-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=494, avg_log_submit_to_slide_time=10003) [2024-10-16 14:57:14.053872] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:14.057440] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.068897] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0956-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301937290, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893251, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.072049] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0958-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301940443, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893252, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.073572] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C095A-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301941968, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893253, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.076797] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C095C-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301945192, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893254, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.087544] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C095E-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301955938, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893255, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.088824] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:14.090299] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9A8-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:14.090326] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9A8-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061833619065004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061833619065004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:14.090859] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0960-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301959253, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893256, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.091397] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9A8-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061834090564000, start_id=1729061834090564000, end_id=1729061834090564001) [2024-10-16 14:57:14.094004] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9A8-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061834093871000, start_id=1729061834093871000, end_id=1729061834093871001) [2024-10-16 14:57:14.094071] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9A8-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=9575, avg_group_log_size=598) [2024-10-16 14:57:14.094081] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9A8-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9575) [2024-10-16 14:57:14.094921] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0962-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301963315, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893257, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.095204] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=337, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:14.095218] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9575) [2024-10-16 14:57:14.095229] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=132, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:14.095261] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:14.095267] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=925, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:14.095271] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=564, avg_log_submit_to_slide_time=891) [2024-10-16 14:57:14.095278] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:14.095309] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:14.099200] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C80-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301967594, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410499, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.100167] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0964-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301968563, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893258, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.103735] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106947][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E43-0-0] [lt=18][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218496418, log_task={header:{begin_lsn:{lsn:207588133364}, end_lsn:{lsn:207588133486}, log_id:218496418, min_scn:{val:1729061833621332003, v:0}, max_scn:{val:1729061833621332003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207588133242}, prev_proposal_id:81, committed_end_lsn:{lsn:207588133364}, data_checksum:940009244, accum_checksum:2662904695, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061834003656, freeze_ts:1729061834003656, submit_ts:1729061834003662, flushed_ts:1729061834006945, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:3283}, fs_cb_begin_ts=1729061834103728, log_life_time=100072) [2024-10-16 14:57:14.106921] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0966-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301975315, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893259, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.109698] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:14.109714] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:14.110801] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1636, avg_group_log_size=163) [2024-10-16 14:57:14.110814] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1636) [2024-10-16 14:57:14.110847] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:14.110853] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:14.111124] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=229, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:14.111139] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=354, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:14.111214] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:14.111370] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FB32-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:14.111378] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FB32-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=934, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:14.111434] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCED5-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:14.111439] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCED5-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=750, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:14.111455] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCED5-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=21, avg_log_submit_to_flush_time=443, avg_log_submit_to_slide_time=729) [2024-10-16 14:57:14.114705] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21659][T1_L0_G0][T1][YB4A0A000409-00061E998FB3C8CB-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301983100, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410500, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.119761] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B42-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061834119131000, start_id=1729061834119131000, end_id=1729061834119131001) [2024-10-16 14:57:14.119791] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0B44-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061834119131000, start_id=1729061834119131000, end_id=1729061834119131001) [2024-10-16 14:57:14.119992] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B8B8-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061834119131000, start_id=1729061834119131000, end_id=1729061834119131001) [2024-10-16 14:57:14.120254] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B8BB-0-0] [lt=15] submit log success(service_type_=0, last_id=1729061850721265000, limited_id=1729061850721265000) [2024-10-16 14:57:14.120261] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B8BB-0-0] [lt=7] submit log with lock success(service_type_=0, last_id=1729061850721265000, limited_id=1729061850721265000) [2024-10-16 14:57:14.120265] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B8BB-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061834119131000, start_id=1729061834119131000, end_id=1729061834119131001) [2024-10-16 14:57:14.120485] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:109) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B46-0-0] [lt=4] is logging [2024-10-16 14:57:14.120495] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:57) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B46-0-0] [lt=9] submit log with lock(ret=-4023, service_type_=0, last_id=1729061850721265000, limited_id=1729061850721265000) [2024-10-16 14:57:14.122738] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:14.123697] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=17179, mem_used=31196160) [2024-10-16 14:57:14.124049] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061834123939000, start_id=1729061834123939000, end_id=1729061834123939001) [2024-10-16 14:57:14.124199] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=10] current monitor number(seq_=-1) [2024-10-16 14:57:14.125117] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061850721265000, latest_scn={val:1729061834119131001, v:0}, *this={tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823000, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850721265000, latest_log_ts:{val:1729061834119131001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:14.125156] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=39] handle submit callback(service_type_=0, success=true, used_ts=4903, log_ts={val:1729061834119131001, v:0}, limited_id=1729061850721265000) [2024-10-16 14:57:14.125163] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:14.125168] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:14.127681] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0968-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301996075, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893260, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.131579] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C096A-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510301999973, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893261, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.136318] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C096C-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302004713, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893262, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.139830] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C096E-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302008225, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893263, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.140316] INFO [SHARE.LOCATION] renew_all_ls_locations_by_rpc (ob_ls_location_service.cpp:801) [21248][AutoLSLocRpc][T0][YB4A0A000407-00061E9B390A223C-0-0] [lt=46] [LS_LOCATION] Get ls leaders by RPC(ret=0, ret="OB_SUCCESS", dests=["10.0.4.7:2890", "10.0.4.8:2890", "10.0.4.9:2890"], leaders=[{key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}}, {key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}}, {key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}}, {key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}}, {key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}}, {key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}}, {key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}}, {key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}}, {key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}]) [2024-10-16 14:57:14.146111] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.146431] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0970-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302014825, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893264, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.152250] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1226, avg_group_log_size=122) [2024-10-16 14:57:14.152268] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1226) [2024-10-16 14:57:14.153313] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0972-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302021708, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893265, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.153784] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:14.153796] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=913, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:14.157466] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.157702] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFF7-0-0] [lt=25] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061834148923, keepalive_ts:-1}) [2024-10-16 14:57:14.157717] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFF7-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061834148923, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:14.157894] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFF6-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061834148879, keepalive_ts:-1}) [2024-10-16 14:57:14.157904] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFF6-0-0] [lt=10] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061834148879, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:14.159021] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0974-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302027416, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893266, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.159211] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5010-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:14.159230] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5010-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:14.161231] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D2D-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061834152129, keepalive_ts:-1}) [2024-10-16 14:57:14.161269] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D2D-0-0] [lt=36] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061834152129, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:14.161293] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D2C-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061834151938, keepalive_ts:-1}) [2024-10-16 14:57:14.161298] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D2C-0-0] [lt=4] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061834151938, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:14.162675] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3667, avg_group_log_size=333) [2024-10-16 14:57:14.162922] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=11, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:14.163093] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106635][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0EFD-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=542, avg_log_submit_to_slide_time=811) [2024-10-16 14:57:14.164468] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BAE-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:14.164485] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BAE-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:14.166813] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0976-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302035207, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893267, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.168474] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0978-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302036870, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893268, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.169728] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C097A-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302038125, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893269, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.173616] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061834172030000, start_id=1729061834172030000, end_id=1729061834172030001) [2024-10-16 14:57:14.176386] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15C0-0-0] [lt=11] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:14.177411] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061834176023000, start_id=1729061834176023000, end_id=1729061834176023001) [2024-10-16 14:57:14.184740] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:14.190519] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15C4-0-0] [lt=20] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:14.191194] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C097C-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302059587, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893270, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.194378] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C097E-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302062772, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893271, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.197707] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0980-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302066101, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893272, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.207515] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0982-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302075909, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893273, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.209278] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:14.209994] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.210020] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.210039] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:14.210162] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823000, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850721265000, latest_log_ts:{val:1729061834119131001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:14.210217] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=46] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:14.210240] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=19] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:14.210248] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=0, rec_log_ts={val:1729061834119131001, v:0}) [2024-10-16 14:57:14.210255] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.210264] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:14.210278] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:14.210288] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:14.210988] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.211002] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.211014] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:14.211080] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823000, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:14.211116] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=36] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823000, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:14.211142] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=24] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:14.211148] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:14.211155] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:14.211160] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:14.211857] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.211871] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.211880] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:14.211944] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823000, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:14.211974] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823000, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:14.211998] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=23] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:14.212003] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:14.212010] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:14.212014] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:14.212717] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.212731] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:14.212740] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:14.212805] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823000, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:14.212833] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061831506823000, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:14.212856] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:14.212861] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:14.212867] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:14.212872] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:14.212876] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:14.213454] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0984-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302081848, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893274, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.216425] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0986-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302084820, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893275, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.217715] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22455][T1003_TenantWea][T1003][YB4A0A000407-00061E9B3ABCFB3B-0-0] [lt=16] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.125743007659912109e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=306, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.980124782037972636e+03) [2024-10-16 14:57:14.217840] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0988-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302086236, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893276, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.217856] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061834216477000, start_id=1729061834216477000, end_id=1729061834216477001) [2024-10-16 14:57:14.219525] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C098A-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302087921, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893277, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.223447] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061834223162000, start_id=1729061834223162000, end_id=1729061834223162001) [2024-10-16 14:57:14.228635] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183801038, max_lsn:114264443768, max_scn:{val:1729061834223162000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45336899169}, reuse_lsn:{lsn:114264443768}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114222500728}}, last_submit_lsn:{lsn:114264443175}, last_submit_end_lsn:{lsn:114264443768}, last_submit_log_id:183801038, last_submit_log_pid:33576, max_flushed_lsn:{lsn:114264443175}, max_flushed_end_lsn:{lsn:114264443768}, max_flushed_log_pid:33576, committed_end_lsn:{lsn:114264443768}, last_slide_log_id:183801038, last_slide_scn:{val:1729061834223162000, v:0}, last_slide_lsn:{lsn:114264443175}, last_slide_end_lsn:{lsn:114264443768}, last_slide_log_pid:33576, last_slide_log_accum_checksum:3310865129, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e2efe2530}, [StateMgr]={this:0x2b0e2efe6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, leader:"10.0.4.7:2890", leader_epoch:2, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183801039, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261840845, palf_role_change_cb:0x2b0e2efed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, reconfig_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, persistent_config_version:{proposal_id:33576, config_seq:9}, ms_ack_list:0, resend_config_version:{proposal_id:33576, config_seq:9}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494259837171, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494259837171, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e2efe3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262041271, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729031774558563002, v:0}, min_block_id:1697, min_block_min_scn:{val:1729025370959302001, v:0}, base_lsn_for_block_gc:{lsn:114212315136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, log_config_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, log_snapshot_meta:{version:1, base_lsn:{lsn:114212315136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:232079360}, readable_log_tail:{lsn:232079360}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114279419904}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/meta", dir_fd:187, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:36339712, block_header_serialize_buf_:0x2b0e2efe89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114264443768}, readable_log_tail:{lsn:114264443768}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114212315136}, min_scn:{val:1729057087482360001, v:0}, curr_block_id:1702, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/log", dir_fd:190, min_block_id:1697, max_block_id:1703, curr_writable_block_id:1702}, logical_block_size_:67104768, curr_block_writable_size_:14976136, block_header_serialize_buf_:0x2b0e2efea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261839896, this:0x2b0e2efe6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33576, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33575, majority_max_lsn:{lsn:45338084276}, saved_end_lsn:{lsn:45338084276}, last_submit_prepare_req_time_us:1722494261840968, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e2efe69b0}) [2024-10-16 14:57:14.228781] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=143] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061834224105, last_flushed_end_lsn:{lsn:114264443768}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061834224309, last_flushed_end_lsn:{lsn:114264443768}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061834226557, last_flushed_end_lsn:{lsn:114264443768}}]) [2024-10-16 14:57:14.228797] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183801039, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061834224105, last_flushed_end_lsn:{lsn:114264443768}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061834224309, last_flushed_end_lsn:{lsn:114264443768}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061834226557, last_flushed_end_lsn:{lsn:114264443768}}]) [2024-10-16 14:57:14.230060] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E06-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061834229570000, start_id=1729061834229570000, end_id=1729061834229570001) [2024-10-16 14:57:14.233516] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C098C-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302101910, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893278, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.238268] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1DFB-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061834236966000, start_id=1729061834236966000, end_id=1729061834236966001) [2024-10-16 14:57:14.238608] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7617-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=13, avg_log_batch_cnt=0, total_group_log_size=52073, avg_group_log_size=4005) [2024-10-16 14:57:14.241537] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C098E-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302109931, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893279, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.243113] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0990-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302111494, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893280, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.246176] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.251860] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0992-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302120254, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893281, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.257534] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.258759] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0994-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302127153, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893282, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.276808] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0996-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302145195, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893283, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.280737] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0997-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302149130, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893284, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.282306] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0999-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302150701, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893285, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.283810] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C099B-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302152205, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893286, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.283952] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=21] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:14.284113] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=10] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:14.284980] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985C3-0-0] [lt=61] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=189) [2024-10-16 14:57:14.285108] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061834284371000, start_id=1729061834284371000, end_id=1729061834284371001) [2024-10-16 14:57:14.285282] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4886-0-0] [lt=9] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061834252635, keepalive_ts:-1}) [2024-10-16 14:57:14.285298] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4886-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061834252635, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:14.287732] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:14.289113] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1583464448, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:14.289690] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C099D-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302158085, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893287, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.290110] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B1D-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061834288857000, start_id=1729061834288857000, end_id=1729061834288857001) [2024-10-16 14:57:14.291547] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C099F-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302159941, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893288, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.294418] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B5A0C6793-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061834293340000, start_id=1729061834293340000, end_id=1729061834293340001) [2024-10-16 14:57:14.295396] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C35-0-0] [lt=21] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:14.295418] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C35-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061834119131001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061834119131001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:14.296541] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C35-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061834295504000, start_id=1729061834295504000, end_id=1729061834295504001) [2024-10-16 14:57:14.299284] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=20] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):113, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:14.299513] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C35-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061834298714000, start_id=1729061834298714000, end_id=1729061834298714001) [2024-10-16 14:57:14.301858] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09A1-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302170252, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893289, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.307086] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106947][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E47-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:14.307272] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4948, mem_used=10398720) [2024-10-16 14:57:14.310517] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:14.310539] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:14.310563] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] cannot_recycle_log_size statistics(cannot_recycle_log_size=373469171, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:14.311168] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09A3-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302179561, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893290, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.311485] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:14.311484] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1764) [2024-10-16 14:57:14.311891] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FB39-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=38, avg_log_submit_to_flush_time=1151, avg_log_submit_to_slide_time=1394) [2024-10-16 14:57:14.317538] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B5A0C6793-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061834316786000, start_id=1729061834316786000, end_id=1729061834316786001) [2024-10-16 14:57:14.320237] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=10] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:14.324226] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09A5-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302192619, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893291, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.324450] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061834322070000, start_id=1729061834322070000, end_id=1729061834322070001) [2024-10-16 14:57:14.325438] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=246, start_service_tstamp_=1722494263598349, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:14.326670] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09A7-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302195064, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893292, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.327631] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B5A0C6793-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061834327606000, start_id=1729061834327606000, end_id=1729061834327606001) [2024-10-16 14:57:14.337841] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B5A0C6793-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061834337174000, start_id=1729061834337174000, end_id=1729061834337174001) [2024-10-16 14:57:14.343819] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09AC-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302212212, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893293, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.345519] INFO [SHARE] adaptive_update_limit (ob_share_throttle_define.cpp:112) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789D6-0-0] [lt=15] adaptive update(Tenant ID=1004, Config Specify Resource Limit(MB)=511, TxShare Current Memory Limit(MB)=159, Holding Memory(MB)=46, Tenant Remain Memory(MB)=187, Usable Remain Memory(MB)=112, Last Update Limit Timestamp=1729061834344611, Is Updated=true) [2024-10-16 14:57:14.345618] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789D6-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061834344611000, start_id=1729061834344611000, end_id=1729061834344611001) [2024-10-16 14:57:14.346180] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=46190590, this={dio_aligned_buf:{buf_write_offset:545, buf_padding_size:3551, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33201797, truncate_used_ts:11502068}, log_block_size:67108864, dir_fd:195, io_fd:131}, aligned_buf_len=8192, aligned_block_offset=46186496, buf_len=547, total_write_size_=15422, total_write_size_after_dio_=425984, ob_pwrite_used_ts=31186, count=100, each_pwrite_cost=311) [2024-10-16 14:57:14.346234] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.350117] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09AE-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302218510, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893294, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.353803] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:14.354070] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=62, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:14.354084] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:14.356230] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09B0-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302224623, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893295, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.357608] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.358040] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09B2-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302226436, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893296, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.359622] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09B4-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302228018, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893297, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.363007] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=267, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:14.363031] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=54, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:14.363398] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C7A-0-0] [lt=8] [APPLY STAT CB APPEND COST TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=47, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:14.363428] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C7A-0-0] [lt=30] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=768, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:14.363433] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C7A-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:14.363436] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C7A-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=-478, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:14.363439] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C7A-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=337, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:14.364618] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BB2-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:14.364639] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BB2-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=227, avg_log_submit_to_slide_time=11510) [2024-10-16 14:57:14.364667] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:14.364923] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09B6-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302233318, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893298, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.365378] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCEDE-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:14.365389] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCEDE-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10440, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:14.366869] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09B8-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302235264, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893299, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.368794] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09BA-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302237190, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893300, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.371085] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09BC-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302239481, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893301, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.371429] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:14.371448] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=4194304, ctx.total_memstore_used_=37748736, ctx.total_memstore_hold_=37748736, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:14.371464] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:14.372864] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09BE-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302241259, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893302, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.373824] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061834373243000, start_id=1729061834373243000, end_id=1729061834373243001) [2024-10-16 14:57:14.374877] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF223D-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302243270, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893303, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.376773] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09C0-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302245168, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893304, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.377895] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061834376049000, start_id=1729061834376049000, end_id=1729061834376049001) [2024-10-16 14:57:14.381109] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09C2-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302249504, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893305, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.383215] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:14.384819] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:14.388599] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09C4-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302256992, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893306, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.390170] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09C6-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302258565, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893307, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.391654] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=7] Garbage Collector is running(seq_=656746, gc_interval=10000000) [2024-10-16 14:57:14.391683] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=11] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:14.391696] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=30) [2024-10-16 14:57:14.391702] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:14.391707] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:14.391713] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:14.391718] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=5) [2024-10-16 14:57:14.391722] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=2] ObSafeDestroyHandler start process [2024-10-16 14:57:14.394649] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:14.394663] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53593280, tenant_id=1003) [2024-10-16 14:57:14.394675] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:14.396819] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09C8-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302265214, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893308, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.398566] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09CA-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302266960, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893309, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.400621] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09CC-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302269016, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893310, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.410264] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09CE-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302278657, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893311, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.412113] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09D0-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302280507, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893312, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.417891] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061834416449000, start_id=1729061834416449000, end_id=1729061834416449001) [2024-10-16 14:57:14.417962] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:14.417975] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:14.418155] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09D2-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302286548, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893313, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.423733] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061834422664000, start_id=1729061834422664000, end_id=1729061834422664001) [2024-10-16 14:57:14.429455] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [21736][T1001_COMMONLSS][T1001][YB4A0A000407-00061E9B3A29CADF-0-0] [lt=26][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:14.438267] INFO [LIB] stat (utility.h:1165) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE761B-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:14.438281] INFO [LIB] stat (utility.h:1165) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE761B-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40369, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:14.438286] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE761B-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220937260, log_task={header:{begin_lsn:{lsn:213056799416}, end_lsn:{lsn:213056799538}, log_id:220937260, min_scn:{val:1729061834325279000, v:0}, max_scn:{val:1729061834325279000, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213056799294}, prev_proposal_id:88, committed_end_lsn:{lsn:213056799416}, data_checksum:1715074492, accum_checksum:232960526, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061834338078, freeze_ts:1729061834338078, submit_ts:1729061834338084, flushed_ts:1729061834339003, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:919}, fs_cb_begin_ts=1729061834438261, log_life_time=100183) [2024-10-16 14:57:14.438334] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=52195, hit_cnt=14, read_cnt=14, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:14.440984] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A64-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:14.441001] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A64-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:14.443793] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09D4-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302312187, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893314, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.446445] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09D6-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302314840, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893315, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.446571] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.447989] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09D8-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302316385, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893316, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.449371] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09DA-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302317768, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893317, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.450617] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09DC-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302319013, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893318, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.453638] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A24-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=38, avg_log_submit_to_flush_time=462, avg_log_submit_to_slide_time=748) [2024-10-16 14:57:14.453698] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3828-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=22, avg_log_submit_to_flush_time=652, avg_log_submit_to_slide_time=919) [2024-10-16 14:57:14.457583] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09DE-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302325977, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893319, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.457716] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.467189] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09E3-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302335582, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893320, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.479136] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09E5-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302347530, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893321, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.481010] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C80-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061834480947000, start_id=1729061834480947000, end_id=1729061834480947001) [2024-10-16 14:57:14.481798] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=7] [APPLY STAT CB APPEND COST TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=49, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:57:14.481810] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=997, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:57:14.481815] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:57:14.481818] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=-576, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:57:14.481821] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=2] [APPLY STAT CB TOTAL TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=471, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:57:14.488298] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09E7-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302356690, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893322, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.489409] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1585545216, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:14.489939] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09E9-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302358333, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893323, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.495333] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09EB-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302363728, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893324, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.503930] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106947][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E4B-0-0] [lt=19][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218496422, log_task={header:{begin_lsn:{lsn:207588133843}, end_lsn:{lsn:207588133965}, log_id:218496422, min_scn:{val:1729061834123939002, v:0}, max_scn:{val:1729061834123939002, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207588133730}, prev_proposal_id:81, committed_end_lsn:{lsn:207588133843}, data_checksum:940009244, accum_checksum:3043640251, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061834403848, freeze_ts:1729061834403848, submit_ts:1729061834403855, flushed_ts:1729061834404679, gen_to_freeze cost time:0, gen_to_submit cost time:7, submit_to_flush cost time:824}, fs_cb_begin_ts=1729061834503923, log_life_time=100075) [2024-10-16 14:57:14.509278] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09ED-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302377671, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893325, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.510140] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061834509512000, start_id=1729061834509512000, end_id=1729061834509512001) [2024-10-16 14:57:14.510158] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] ts response task statistics(alloc_count=261363346, free_count=130681672) [2024-10-16 14:57:14.510173] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=495, get_gts_with_stc_cnt=74, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=74, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:14.510403] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=509, get_gts_with_stc_cnt=9, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:14.514733] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09EF-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302383127, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893326, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.516616] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09F1-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302385011, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893327, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.517955] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09F3-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302386350, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893328, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.524276] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09F5-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302392670, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893329, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.525143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061834517997000, start_id=1729061834517997000, end_id=1729061834517997001) [2024-10-16 14:57:14.525422] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:14.528534] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D52-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:14.528558] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D52-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061834347829000, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061834347829000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:14.529564] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D52-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061834528927000, start_id=1729061834528927000, end_id=1729061834528927001) [2024-10-16 14:57:14.530737] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:14.530754] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4381) [2024-10-16 14:57:14.530769] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1440265373, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:14.532337] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D52-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061834532085000, start_id=1729061834532085000, end_id=1729061834532085001) [2024-10-16 14:57:14.539460] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09F7-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302407853, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893330, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.546624] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.554639] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09F9-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302423033, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893331, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.554723] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A35-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178185, log_task={header:{begin_lsn:{lsn:14538455120}, end_lsn:{lsn:14538455242}, log_id:115178185, min_scn:{val:1729061834347829001, v:0}, max_scn:{val:1729061834347829001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538454998}, prev_proposal_id:23, committed_end_lsn:{lsn:14538455120}, data_checksum:1715074492, accum_checksum:1479183525, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061834453614, freeze_ts:1729061834453614, submit_ts:1729061834453618, flushed_ts:1729061834454797, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1179}, fs_cb_begin_ts=1729061834554716, log_life_time=101102) [2024-10-16 14:57:14.554918] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=264, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:14.554935] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=29, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:14.554964] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:14.557774] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.564012] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:14.564028] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:14.564150] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:14.564164] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=5542) [2024-10-16 14:57:14.564171] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:14.564202] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09FB-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302432596, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893332, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.564698] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=5420) [2024-10-16 14:57:14.565036] INFO [LIB] stat (utility.h:1165) [106635][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45ACD-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:14.565049] INFO [LIB] stat (utility.h:1165) [106635][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45ACD-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=688, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:14.565076] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:14.568881] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.569125] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.569469] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.569488] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.573928] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061834572902000, start_id=1729061834572902000, end_id=1729061834572902001) [2024-10-16 14:57:14.576392] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] worker thread began to shrink(tenant_->id()=1001, group_id_=1, token=0) [2024-10-16 14:57:14.577967] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061834577157000, start_id=1729061834577157000, end_id=1729061834577157001) [2024-10-16 14:57:14.578182] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1003}, {id:1002}]) [2024-10-16 14:57:14.578297] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23403220, this={dio_aligned_buf:{buf_write_offset:3359, buf_padding_size:737, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:47003840, truncate_used_ts:19010200}, log_block_size:67108864, dir_fd:134, io_fd:183}, aligned_buf_len=4096, aligned_block_offset=23400448, buf_len=587, total_write_size_=57079, total_write_size_after_dio_=577536, ob_pwrite_used_ts=41996, count=127, each_pwrite_cost=330) [2024-10-16 14:57:14.578318] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=20] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=5420) [2024-10-16 14:57:14.579769] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B27-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061834578227000, start_id=1729061834578227000, end_id=1729061834578227001) [2024-10-16 14:57:14.579784] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1DFA-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061834578227000, start_id=1729061834578227000, end_id=1729061834578227001) [2024-10-16 14:57:14.580118] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=52135901, this={dio_aligned_buf:{buf_write_offset:2562, buf_padding_size:1534, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80310636, truncate_used_ts:29009380}, log_block_size:67108864, dir_fd:190, io_fd:179}, aligned_buf_len=4096, aligned_block_offset=52133888, buf_len=549, total_write_size_=103882, total_write_size_after_dio_=790528, ob_pwrite_used_ts=67904, count=168, each_pwrite_cost=404) [2024-10-16 14:57:14.584667] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106951][T1001_L0_G1][T1001][Y0-0000000000000000-0-0] [lt=6] worker thread exit(tenant_->id()=1001, workers_.get_size()=2) [2024-10-16 14:57:14.584688] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106951][T1001_L0_G1][T0][Y0-0000000000000000-0-0] [lt=16] tenant thread end_run(id_=1001, ret=0, thread_count_=171) [2024-10-16 14:57:14.584897] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:14.585437] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C09FD-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302453830, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893333, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.589342] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF223E-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302457735, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893334, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.590848] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A00-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302459242, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893335, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.592144] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A02-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302460539, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893336, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.593440] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A04-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302461836, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893337, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.594701] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A06-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302463097, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893338, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.595384] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4402, mem_used=8318976) [2024-10-16 14:57:14.597864] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A08-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302466261, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893339, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.598724] INFO [ARCHIVE] do_thread_task_ (ob_archive_fetcher.cpp:310) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=9] ObArchiveFetcher is running(thread_index=0) [2024-10-16 14:57:14.599110] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A0A-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302467507, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893340, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.603933] INFO [LIB] stat (utility.h:1165) [106947][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E4D-0-0] [lt=31] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:14.603947] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A0C-0-0] [lt=9][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302472342, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893341, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.604249] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=252, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:14.604268] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=372, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:14.604306] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=27, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:14.604491] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A29-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:14.604545] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:14.608887] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A0E-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302477283, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893342, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.611756] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A10-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302480151, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893343, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.618093] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061834617100000, start_id=1729061834617100000, end_id=1729061834617100001) [2024-10-16 14:57:14.619644] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A12-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302488038, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893344, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.620549] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B4B-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061834619448000, start_id=1729061834619448000, end_id=1729061834619448001) [2024-10-16 14:57:14.620578] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B47-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061834620560000, start_id=1729061834620560000, end_id=1729061834620560001) [2024-10-16 14:57:14.620662] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B48-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1729061834620560000, start_id=1729061834620560000, end_id=1729061834620560001) [2024-10-16 14:57:14.620687] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0B49-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061834620560000, start_id=1729061834620560000, end_id=1729061834620560001) [2024-10-16 14:57:14.624561] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A14-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302492955, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893345, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.626011] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A16-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302494406, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893346, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.627329] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A18-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302495725, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893347, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.629985] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C82-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302498380, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410501, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.630874] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=8] LogLoopThread round_cost_time(us)(round_cost_time=5) [2024-10-16 14:57:14.630956] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A1A-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302499353, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893348, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.631168] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C84-0-0] [lt=23][errcode=-4201] hashtable not init [2024-10-16 14:57:14.631178] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C84-0-0] [lt=9][errcode=-4016] get cursor info failed(cursor_id=1107, get_sessid()=3221565766) [2024-10-16 14:57:14.631260] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C85-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061834631016000, start_id=1729061834631016000, end_id=1729061834631016001) [2024-10-16 14:57:14.636975] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A1C-0-0] [lt=9][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302505371, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893349, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.645409] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A1E-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302513804, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893350, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.646690] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.648498] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:14.650615] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A21-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302518993, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893351, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.652064] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A23-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302520460, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893352, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.654045] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A25-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302522441, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893353, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.654084] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=324, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:14.654103] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=241, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:14.654138] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:14.657837] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.659304] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A27-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302527699, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893354, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.665108] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=209, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:14.665130] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=50, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:14.665167] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:14.667834] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A2C-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302536220, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893355, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.671325] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A2E-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302539719, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893356, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.672741] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A30-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302541137, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893357, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.674083] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A32-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302542479, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893358, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.675367] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A34-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302543763, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893359, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.677148] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061834395003000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=282138, in_cluster_service=true, cluster_version={val:1729061834229570000, v:0}, min_cluster_version={val:1729061834229570000, v:0}, max_cluster_version={val:1729061834229570000, v:0}, get_cluster_version_err=0, cluster_version_delta=447571, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65248631, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=331949856, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:14.685193] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B1A-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061834684909000, start_id=1729061834684909000, end_id=1729061834684909001) [2024-10-16 14:57:14.685544] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=42690477, this={dio_aligned_buf:{buf_write_offset:2513, buf_padding_size:1583, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35133110, truncate_used_ts:12254288}, log_block_size:67108864, dir_fd:169, io_fd:144}, aligned_buf_len=4096, aligned_block_offset=42688512, buf_len=548, total_write_size_=15427, total_write_size_after_dio_=425984, ob_pwrite_used_ts=38923, count=100, each_pwrite_cost=389) [2024-10-16 14:57:14.685799] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:14.686663] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061834685959000, start_id=1729061834685959000, end_id=1729061834685959001) [2024-10-16 14:57:14.689437] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A36-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302557831, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893360, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.689709] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1181712384, global_cache_size=1589706752, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:14.691620] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:14.694879] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22398][T1003_TntShared][T1003][Y0-0000000000000000-0-0] [lt=10] clear dtl interm result cost(us)(clear_cost=16, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:14.695018] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:14.707141] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31896][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB1FCC6D8-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:14.707301] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15CA-0-0] [lt=4] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:14.711221] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=61] success to update min reserved snapshot(reserved_snapshot=1729060031822855000, duration=1800, snapshot_gc_ts=1729061831822855000) [2024-10-16 14:57:14.711387] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A38-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302579778, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893361, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.711504] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:14.711519] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:14.711746] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23122441, this={dio_aligned_buf:{buf_write_offset:643, buf_padding_size:3453, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33384804, truncate_used_ts:14052801}, log_block_size:67108864, dir_fd:200, io_fd:174}, aligned_buf_len=4096, aligned_block_offset=23121920, buf_len=122, total_write_size_=12060, total_write_size_after_dio_=417792, ob_pwrite_used_ts=31480, count=99, each_pwrite_cost=317) [2024-10-16 14:57:14.713062] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=15] update reserved snapshot finished(cost_ts=1843, reserved_snapshot=1729060031822855000) [2024-10-16 14:57:14.715331] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=8] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:14.715373] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=207) [2024-10-16 14:57:14.717398] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=12][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:14.720041] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:14.721090] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A3A-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302589484, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893362, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.725391] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061834724817000, start_id=1729061834724817000, end_id=1729061834724817001) [2024-10-16 14:57:14.726542] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061834724817000, start_id=1729061834724817000, end_id=1729061834724817001) [2024-10-16 14:57:14.727676] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257646, offset=39000290}, end_cursor:ObLogCursor{file_id=197, log_id=49257647, offset=39000573}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2850050, len:1822, log_data_len:283, seq:49257646, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:14.729431] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22628][T1004_TeRec][T1004][YB4A0A000407-00061E9B221FB354-0-0] [lt=17] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=1, MAX_GAP=900000000000, new_sync_scn={val:1729061833619065004, v:0}, new_replayable_scn={val:1729061833619065004, v:0}, new_readable_scn={val:1729061833619065003, v:0}, old_tenant_info={tenant_id:1004, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061832508384002, v:0}, replayable_scn:{val:1729061832508384002, v:0}, standby_scn:{val:1729061832508384002, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"ARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:57:14.729630] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB354-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061834728811000, start_id=1729061834728811000, end_id=1729061834728811001) [2024-10-16 14:57:14.730430] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257647, offset=39000573}, end_cursor:ObLogCursor{file_id=197, log_id=49257648, offset=39000855}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2950050, len:1539, log_data_len:282, seq:49257647, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:14.733664] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257648, offset=39000855}, end_cursor:ObLogCursor{file_id=197, log_id=49257649, offset=39001137}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2938050, len:1257, log_data_len:282, seq:49257648, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:14.733846] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257649, offset=39001137}, end_cursor:ObLogCursor{file_id=197, log_id=49257650, offset=39001419}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a90050, len:975, log_data_len:282, seq:49257649, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:14.733910] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62649946, miss_cnt:32, cache_read_size:8050384449, hit_ratio:"9.999994892256786061e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:14.733933] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656682, miss_cnt:17, cache_read_size:8050131943, hit_ratio:"9.999997286802485563e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:14.733950] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A3C-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302602344, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893363, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.738310] INFO [LIB] stat (utility.h:1165) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7621-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:14.738783] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560306, miss_cnt:1, cache_read_size:152589240008, hit_ratio:"9.999999928346389177e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:14.739352] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA865-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:14.739375] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA865-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061834629634000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061834629634000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:14.740548] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A3E-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302608942, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893364, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.740654] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA865-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061834739937000, start_id=1729061834739937000, end_id=1729061834739937001) [2024-10-16 14:57:14.741069] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601280, offset=9595312}, end_cursor:ObLogCursor{file_id=192, log_id=47601281, offset=9595596}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd282c050, len:1616, log_data_len:284, seq:47601280, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:14.743268] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA865-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061834743026000, start_id=1729061834743026000, end_id=1729061834743026001) [2024-10-16 14:57:14.743324] INFO [LIB] stat (utility.h:1165) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA865-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:14.743336] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA865-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10323, avg_group_log_size=573) [2024-10-16 14:57:14.743345] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA865-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10710) [2024-10-16 14:57:14.743351] INFO [LIB] stat (utility.h:1165) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA865-0-0] [lt=5] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:14.743551] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=428, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:14.743561] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10323) [2024-10-16 14:57:14.743571] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=282, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:14.743590] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10710) [2024-10-16 14:57:14.743597] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=10, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:14.743700] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA489-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061834743026000, start_id=1729061834743026001, end_id=1729061834743026002) [2024-10-16 14:57:14.744135] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F92F4-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:14.744146] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F92F4-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=1289, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:14.744151] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F92F4-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=67, avg_log_gen_to_submit_time=96, avg_log_submit_to_flush_time=802, avg_log_submit_to_slide_time=1193) [2024-10-16 14:57:14.744173] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:14.745194] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601281, offset=9595596}, end_cursor:ObLogCursor{file_id=192, log_id=47601282, offset=9595870}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29d0050, len:1332, log_data_len:274, seq:47601281, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:14.746718] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A40-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302615112, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893365, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.746961] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.748417] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:14.748441] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:14.748451] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:14.748972] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601282, offset=9595870}, end_cursor:ObLogCursor{file_id=192, log_id=47601283, offset=9596144}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2970050, len:1058, log_data_len:274, seq:47601282, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:14.749235] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=34] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601283, offset=9596144}, end_cursor:ObLogCursor{file_id=192, log_id=47601284, offset=9596418}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b14050, len:784, log_data_len:274, seq:47601283, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:14.749269] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106947][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBCA-0-0] [lt=16] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061834717007, keepalive_ts:-1}) [2024-10-16 14:57:14.749280] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106947][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBCA-0-0] [lt=10] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061834717007, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:14.749363] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137597902, miss_cnt:9, cache_read_size:149776498772, hit_ratio:"9.999999345920302929e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:14.753068] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:14.753098] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:14.753322] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:14.753353] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:14.753915] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A39-0-0] [lt=27][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178187, log_task={header:{begin_lsn:{lsn:14538455364}, end_lsn:{lsn:14538455486}, log_id:115178187, min_scn:{val:1729061834619448000, v:0}, max_scn:{val:1729061834619448000, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538455242}, prev_proposal_id:23, committed_end_lsn:{lsn:14538455364}, data_checksum:1715074492, accum_checksum:3284945668, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061834653816, freeze_ts:1729061834653816, submit_ts:1729061834653821, flushed_ts:1729061834654330, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:509}, fs_cb_begin_ts=1729061834753899, log_life_time=100083) [2024-10-16 14:57:14.754258] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FB49-0-0] [lt=23] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:14.754455] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=11] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:14.755880] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A42-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302624275, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893366, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.757894] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.760986] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A44-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302629379, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893367, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.765801] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A46-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302634195, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893368, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.767916] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:14.770643] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=59030234, this={dio_aligned_buf:{buf_write_offset:2900, buf_padding_size:1196, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:38334652, truncate_used_ts:14110921}, log_block_size:67108864, dir_fd:207, io_fd:202}, aligned_buf_len=4096, aligned_block_offset=59027456, buf_len=122, total_write_size_=12182, total_write_size_after_dio_=421888, ob_pwrite_used_ts=24816, count=100, each_pwrite_cost=248) [2024-10-16 14:57:14.774034] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061834773972000, start_id=1729061834773972000, end_id=1729061834773972001) [2024-10-16 14:57:14.777991] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061834776135000, start_id=1729061834776135000, end_id=1729061834776135001) [2024-10-16 14:57:14.778933] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=13] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:14.779309] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061834395003000, v:0}, new_max_version={val:1729061834395003000, v:0}, last_min_version={val:1729061834229570000, v:0}, last_max_version={val:1729061834229570000, v:0}, total_time_us=2121, delta_us=384305) [2024-10-16 14:57:14.779340] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=29] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061834395003000, v:0}, min_version={val:1729061834395003000, v:0}, max_version={val:1729061834395003000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:14.784971] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:14.785531] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A48-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302653925, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893369, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.788567] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.788576] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.788595] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.788617] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.788626] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.788635] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.788653] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.788887] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.788949] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=35] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.788984] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.789621] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:14.796152] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A4A-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302664545, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893370, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.798944] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A4C-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302667334, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893371, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.802296] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=75] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:14.802886] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=20] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:14.802934] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061834390737000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:14.804705] INFO [LIB] stat (utility.h:1165) [106947][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4BA0-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:14.804717] INFO [LIB] stat (utility.h:1165) [106947][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4BA0-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=36511, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:14.806595] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A4E-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302674990, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893372, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.808190] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A50-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302676586, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893373, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.809753] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A52-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302678149, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893374, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.811823] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=232, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:14.811840] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=113, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:14.815098] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1002}, {id:1001}]) [2024-10-16 14:57:14.817912] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A54-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302686307, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893375, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.818465] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061834817475000, start_id=1729061834817475000, end_id=1729061834817475001) [2024-10-16 14:57:14.820333] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A56-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302688728, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893376, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.824279] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=100] success to update min reserved snapshot(reserved_snapshot=1729060027226807000, duration=1800, snapshot_gc_ts=1729061827226807000) [2024-10-16 14:57:14.825016] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=16] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060027226807000) [2024-10-16 14:57:14.825029] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=11] update reserved snapshot finished(cost_ts=753, reserved_snapshot=1729060027226807000) [2024-10-16 14:57:14.829616] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A58-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302697998, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893377, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.829736] INFO [SQL.SESSION] deep_copy_sys_variable (ob_basic_session_info.cpp:1901) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A59-0-0] [lt=14] Too much memory used for system variable values. do defragment(before=7951, after=146, ids=[14, 15, 23, 37, 55, 10039, 10060, 10102, 10150, 10154, 10156, 10020, 10026, 10005, 10008, 13, 28, 10037, 92, 10151, 36, 10108, 2], sessid_=3221499809, proxy_sessid_=13882479996272637379) [2024-10-16 14:57:14.831944] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A5C-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061834831295000, start_id=1729061834831295000, end_id=1729061834831295001) [2024-10-16 14:57:14.833860] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2240-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302702253, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893378, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.838254] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A5E-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302706649, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893379, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.838840] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=355, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:14.838856] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=171, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:14.838905] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:14.845864] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A62-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302714258, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893380, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.847255] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A64-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302715651, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893381, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.847352] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.848555] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A66-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302716951, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893382, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.848979] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=2737, mem_used=6239232) [2024-10-16 14:57:14.849849] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A68-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302718245, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893383, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.852371] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=22] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:14.852982] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:14.853028] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=14] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061831822855000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:14.853164] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:14.853177] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:14.854013] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A3B-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:14.857916] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.860386] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A6A-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302728781, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893384, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.861816] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A6C-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302730211, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893385, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.863088] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A6E-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302731485, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893386, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.870149] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A70-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302738544, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893387, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.870433] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BBC-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172881, log_task={header:{begin_lsn:{lsn:14352341722}, end_lsn:{lsn:14352341844}, log_id:115172881, min_scn:{val:1729061834620560001, v:0}, max_scn:{val:1729061834620560001, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352341600}, prev_proposal_id:17, committed_end_lsn:{lsn:14352341722}, data_checksum:940009244, accum_checksum:4211017099, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061834770409, freeze_ts:1729061834770409, submit_ts:1729061834770415, flushed_ts:1729061834770678, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:263}, fs_cb_begin_ts=1729061834870426, log_life_time=100017) [2024-10-16 14:57:14.874501] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A72-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302742896, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893388, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.876149] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061834876147, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007653, add_bytes: 0 [2024-10-16 14:57:14.876694] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A74-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302745090, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893389, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.879118] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 295.25; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 15765.74, iops: 52.84, rt: 123.64; ) [2024-10-16 14:57:14.879141] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:14.879180] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 307.25; ) [2024-10-16 14:57:14.879188] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:14.879199] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:14.882678] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=20] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:14.890002] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1181712384, global_cache_size=1589706752, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:14.898169] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A76-0-0] [lt=9][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302766563, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893390, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.901575] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A78-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302769969, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893391, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.905335] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A7A-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302773724, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893392, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.906012] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2347-0-0] [lt=30] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:57:14.906666] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2347-0-0] [lt=15] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1004) [2024-10-16 14:57:14.906684] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2347-0-0] [lt=16] no transfer partition job [2024-10-16 14:57:14.906690] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2347-0-0] [lt=4] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:57:14.907955] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2347-0-0] [lt=7] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1004, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:57:14.907980] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2347-0-0] [lt=19] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:57:14.909162] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2347-0-0] [lt=14] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1002, status:0}], ls_array_=[{tenant_id:1004, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729059563578310, last_statistic_bg_stat_time=1725269669481368, last_statistic_schema_version=1725266205820024, last_statistic_max_transfer_task_id={id:5}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:57:14.909251] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=41] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:14.909265] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:14.914621] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A7C-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302783015, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893393, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.917910] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A7E-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302786306, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893394, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.919099] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:14.921421] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A80-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302789817, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893395, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.922330] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=13][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:14.922390] WDIAG read_multi_row (ob_table_access_helper.h:238) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=17][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:14.923200] INFO delete_row (ob_table_access_helper.h:312) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=8] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1001, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061834922399) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061834922399) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:14.925445] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061834924207000, start_id=1729061834924207000, end_id=1729061834924207001) [2024-10-16 14:57:14.926349] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:14.926395] WDIAG read_multi_row (ob_table_access_helper.h:238) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:14.926779] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061834924207000, start_id=1729061834924207000, end_id=1729061834924207001) [2024-10-16 14:57:14.927203] INFO delete_row (ob_table_access_helper.h:312) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=5] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1004, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061834926401) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061834926401) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:14.927287] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=11] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:14.927333] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:14.927376] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:14.929619] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A82-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302797998, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893396, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.931350] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:14.931365] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=28047760, tenant_id=1004) [2024-10-16 14:57:14.931374] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:14.934535] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A84-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302802930, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893397, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.936653] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=7] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=380, mem_used=2079744) [2024-10-16 14:57:14.938672] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106947][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7625-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220937265, log_task={header:{begin_lsn:{lsn:213056800026}, end_lsn:{lsn:213056800148}, log_id:220937265, min_scn:{val:1729061834620560002, v:0}, max_scn:{val:1729061834620560002, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213056799904}, prev_proposal_id:88, committed_end_lsn:{lsn:213056800026}, data_checksum:1715074492, accum_checksum:3439862463, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061834838462, freeze_ts:1729061834838462, submit_ts:1729061834838468, flushed_ts:1729061834838888, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:420}, fs_cb_begin_ts=1729061834938665, log_life_time=100203) [2024-10-16 14:57:14.939063] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A86-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302807458, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893398, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.940424] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A88-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302808820, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893399, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.946908] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=9] ====== tenant freeze timer task ====== [2024-10-16 14:57:14.947101] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A8A-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302815496, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893400, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.947370] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.948868] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061834948867, bytes: 4446786534362, bw: 0.432997 MB/s, add_ts: 1004024, add_bytes: 455857 [2024-10-16 14:57:14.952287] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] worker thread began to shrink(tenant_->id()=1, group_id_=19, token=0) [2024-10-16 14:57:14.953199] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:14.953217] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:14.953869] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106947][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3837-0-0] [lt=26] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2498) [2024-10-16 14:57:14.954181] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A3D-0-0] [lt=25] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:14.954203] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A3D-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178189, log_task={header:{begin_lsn:{lsn:14538455608}, end_lsn:{lsn:14538455730}, log_id:115178189, min_scn:{val:1729061834619448002, v:0}, max_scn:{val:1729061834619448002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538455486}, prev_proposal_id:23, committed_end_lsn:{lsn:14538455608}, data_checksum:1715074492, accum_checksum:1708611085, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061834854005, freeze_ts:1729061834854005, submit_ts:1729061834854008, flushed_ts:1729061834854262, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:254}, fs_cb_begin_ts=1729061834954197, log_life_time=100192) [2024-10-16 14:57:14.954926] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=367, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:14.954939] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=304, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:14.954960] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:14.954965] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=935, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:14.956866] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A8C-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302825261, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893401, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.957225] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106703][T1_L0_G19][T1][Y0-0000000000000000-0-0] [lt=15] worker thread exit(tenant_->id()=1, workers_.get_size()=1) [2024-10-16 14:57:14.957242] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106703][T1_L0_G19][T0][Y0-0000000000000000-0-0] [lt=14] tenant thread end_run(id_=1, ret=0, thread_count_=239) [2024-10-16 14:57:14.957970] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:14.962459] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=11] worker thread began to shrink(tenant_->id()=1002, group_id_=1, token=0) [2024-10-16 14:57:14.964128] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106947][T1002_L0_G1][T1002][Y0-0000000000000000-0-0] [lt=20] worker thread exit(tenant_->id()=1002, workers_.get_size()=2) [2024-10-16 14:57:14.964142] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106947][T1002_L0_G1][T0][Y0-0000000000000000-0-0] [lt=9] tenant thread end_run(id_=1002, ret=0, thread_count_=190) [2024-10-16 14:57:14.964785] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A8E-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302833178, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893402, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.967734] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106743][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE200F-0-0] [lt=24] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:14.967752] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106743][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE200F-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:14.970519] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BBE-0-0] [lt=19][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172882, log_task={header:{begin_lsn:{lsn:14352341844}, end_lsn:{lsn:14352341966}, log_id:115172882, min_scn:{val:1729061834620560002, v:0}, max_scn:{val:1729061834620560002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352341722}, prev_proposal_id:17, committed_end_lsn:{lsn:14352341844}, data_checksum:940009244, accum_checksum:3286758840, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061834870412, freeze_ts:1729061834870413, submit_ts:1729061834870418, flushed_ts:1729061834871398, gen_to_freeze cost time:1, gen_to_submit cost time:6, submit_to_flush cost time:980}, fs_cb_begin_ts=1729061834970512, log_life_time=100100) [2024-10-16 14:57:14.972194] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [106955][][T1][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1, ret=0, thread_count_=240) [2024-10-16 14:57:14.972226] INFO [SERVER.OMT] recv_group_request (ob_tenant.cpp:1419) [21189][pnio1][T0][YB4A0A000408-00061E9B44AE2010-0-0] [lt=12] worker thread created(id()=1, group->group_id_=19) [2024-10-16 14:57:14.972318] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [106955][][T1][Y0-0000000000000000-0-0] [lt=122] Init thread local success [2024-10-16 14:57:14.974083] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061834973282000, start_id=1729061834973282000, end_id=1729061834973282001) [2024-10-16 14:57:14.978040] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061834977952000, start_id=1729061834977952000, end_id=1729061834977952001) [2024-10-16 14:57:14.983592] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A90-0-0] [lt=40][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302851987, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893403, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.985089] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A92-0-0] [lt=30][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302853484, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893404, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.986424] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] Cache replace map node details(ret=0, replace_node_count=1, replace_time=0, replace_start_pos=440398, replace_num=31457) [2024-10-16 14:57:14.986443] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=1, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:14.993618] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A94-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302862013, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893405, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.996530] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A96-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302864925, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893406, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:14.997915] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A99-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302866311, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893407, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.004139] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A9D-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302872535, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893408, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.005665] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0A9F-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302874061, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893409, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.010565] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061835010555000, start_id=1729061835010555000, end_id=1729061835010555001) [2024-10-16 14:57:15.010608] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=21] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:15.010630] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=545, get_gts_with_stc_cnt=81, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=78, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:15.010640] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061835010626], gts:1729061835010555000, latest_srr:[mts=1729061835010626]}) [2024-10-16 14:57:15.011722] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=28, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:15.011740] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:15.012098] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:15.012117] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:15.012391] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2310) [2024-10-16 14:57:15.014114] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AA1-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302882508, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893410, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.015046] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:15.015166] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DA6-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061834973282000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061845590307000, latest_log_ts:{val:1729061828964666001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:15.015228] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DA6-0-0] [lt=58] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:15.018973] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061835018049000, start_id=1729061835018049000, end_id=1729061835018049001) [2024-10-16 14:57:15.019311] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=12019) [2024-10-16 14:57:15.020852] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2241-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302889247, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893411, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.027641] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AA4-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302896035, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893412, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.036176] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:15.036197] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=16] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:15.036211] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=291821727, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:15.038181] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AA6-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302906574, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893413, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.042007] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AA8-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302910400, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893414, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.047102] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AAA-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302915496, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893415, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.047439] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.050044] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AAC-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302918439, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893416, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.050282] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4807-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061835018029, keepalive_ts:-1}) [2024-10-16 14:57:15.050301] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4807-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061835018029, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:15.053534] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2498) [2024-10-16 14:57:15.053568] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:15.054023] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A3F-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:15.054035] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A3F-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=55815, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:15.054041] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A3F-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=395, avg_log_submit_to_slide_time=55810) [2024-10-16 14:57:15.054106] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:15.055564] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AAE-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302923959, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893417, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.058053] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.060549] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AB0-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302928943, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893418, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.065324] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AB2-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302933719, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893419, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.066649] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AB4-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302935045, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893420, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.070334] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AB6-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302938730, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893421, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.072010] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:15.072024] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=149267640, tenant_id=1) [2024-10-16 14:57:15.072038] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:15.073264] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get_max_applied_scn(scn={val:1729061834973282000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618247046}, palf_committed_end_scn_:{val:1729061834973282000, v:0}, last_check_scn_:{val:1729061834973282000, v:0}, max_applied_cb_scn_:{val:1729061834973282000, v:0}}) [2024-10-16 14:57:15.073288] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:15.075566] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AB8-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302943962, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893422, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.076899] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0ABA-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302945295, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893423, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.078267] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0ABC-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302946664, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893424, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.079632] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0ABE-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302948028, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893425, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.081481] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:15.082225] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=10] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:15.082250] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:15.082269] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:15.082379] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061847008855000, latest_log_ts:{val:1729061830358352001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:15.082425] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=43] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:15.082435] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:57:15.082442] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061830358352001, v:0}) [2024-10-16 14:57:15.082445] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:15.082454] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:15.082461] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:15.082471] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:57:15.082479] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:15.082483] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:15.086209] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AC0-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302954605, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893426, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.087597] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AC2-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302955993, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893427, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.090290] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1179615232, global_cache_size=1591787520, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:15.093926] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:15.093999] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AC4-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302962394, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893428, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.097338] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9A9-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:15.097361] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9A9-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061834620560004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061834620560004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:15.098683] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9A9-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061835098302000, start_id=1729061835098302000, end_id=1729061835098302001) [2024-10-16 14:57:15.103785] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C86-0-0] [lt=8][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302972178, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410502, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.104031] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9A9-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061835103891000, start_id=1729061835103891000, end_id=1729061835103891001) [2024-10-16 14:57:15.104112] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9A9-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=11426, avg_group_log_size=571) [2024-10-16 14:57:15.104136] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9A9-0-0] [lt=21] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11426) [2024-10-16 14:57:15.104453] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=243, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:15.104474] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=21] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11426) [2024-10-16 14:57:15.104486] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=123, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:15.104540] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:15.104583] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1333) [2024-10-16 14:57:15.104886] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797A79-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:15.104893] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797A79-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=753, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:15.104902] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797A79-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=457, avg_log_submit_to_slide_time=720) [2024-10-16 14:57:15.105025] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:15.105036] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=46, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:57:15.105040] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=1219, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:57:15.105043] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:57:15.105046] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=-456, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:57:15.105049] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=809, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:57:15.109632] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:15.109645] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:15.111262] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AC6-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302979655, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893429, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.111778] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=2310, avg_group_log_size=192) [2024-10-16 14:57:15.111796] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2310) [2024-10-16 14:57:15.111824] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:15.111831] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:57:15.112121] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=464, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:15.112136] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=473, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:15.112139] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:15.112337] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCEF5-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:15.112348] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCEF5-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1236, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:15.112381] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCEF6-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:15.112386] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCEF6-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=689, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:15.112390] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCEF6-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=20, avg_log_submit_to_flush_time=353, avg_log_submit_to_slide_time=668) [2024-10-16 14:57:15.115086] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AC8-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302983481, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893430, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.115185] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=48] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:57:15.115195] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:57:15.115199] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:15.115814] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061845590307000, latest_log_ts:{val:1729061828964666001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:15.118119] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061834817475001, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=300638, in_cluster_service=true, cluster_version={val:1729061834817475000, v:0}, min_cluster_version={val:1729061834817475000, v:0}, max_cluster_version={val:1729061834817475000, v:0}, get_cluster_version_err=0, cluster_version_delta=300638, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65243947, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=444187916, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:15.119426] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0ACA-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302987822, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893431, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.120875] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B4C-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061835119854000, start_id=1729061835119854000, end_id=1729061835119854001) [2024-10-16 14:57:15.120912] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B50-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061835119854000, start_id=1729061835119854000, end_id=1729061835119854001) [2024-10-16 14:57:15.120964] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B8C3-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061835120915000, start_id=1729061835120915000, end_id=1729061835120915001) [2024-10-16 14:57:15.121205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B4D-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061835120915000, start_id=1729061835120915000, end_id=1729061835120915001) [2024-10-16 14:57:15.122764] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:15.124284] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=7] current monitor number(seq_=-1) [2024-10-16 14:57:15.124473] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0ACC-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510302992867, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893432, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.124530] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=17259, mem_used=31196160) [2024-10-16 14:57:15.125568] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061835125098000, start_id=1729061835125098000, end_id=1729061835125098001) [2024-10-16 14:57:15.127086] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061835126164000, start_id=1729061835126164000, end_id=1729061835126164001) [2024-10-16 14:57:15.132377] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0ACE-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303000771, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893433, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.135892] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AD0-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303004286, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893434, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.137296] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AD2-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303005692, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893435, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.140502] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7629-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:15.140761] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1333) [2024-10-16 14:57:15.144397] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AD4-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303012791, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893436, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.147505] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.153289] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2498, avg_group_log_size=249) [2024-10-16 14:57:15.153307] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2498) [2024-10-16 14:57:15.154018] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A39-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:15.154029] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A39-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=751, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:15.158118] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.160012] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5011-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:15.160032] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5011-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:15.163497] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AD6-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303031890, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893437, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.164486] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=6946, avg_group_log_size=463) [2024-10-16 14:57:15.164726] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:15.165124] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AD8-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303033519, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893438, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.165287] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106635][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F0C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=388, avg_log_submit_to_slide_time=722) [2024-10-16 14:57:15.166593] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AD9-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303034989, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893439, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.167820] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0ADB-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303036216, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893440, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.171436] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BC2-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:15.171455] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BC2-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:15.174241] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=42] get number(ret=0, service_type_=0, range=1, base_id=1729061835173053000, start_id=1729061835173053000, end_id=1729061835173053001) [2024-10-16 14:57:15.178349] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061835176500000, start_id=1729061835176500000, end_id=1729061835176500001) [2024-10-16 14:57:15.182230] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106635][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A07-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061835173450, keepalive_ts:-1}) [2024-10-16 14:57:15.182250] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106635][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A07-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061835173450, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:15.186315] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0ADD-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303054709, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893441, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.187895] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=471855, replace_num=31457) [2024-10-16 14:57:15.187915] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:15.188421] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31906][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB20CC6D8-0-0] [lt=26] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:15.195223] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0ADF-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303063615, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893442, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.196700] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AE1-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303065094, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893443, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.210895] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AE3-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303079288, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893444, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.212517] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AE5-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303080912, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893445, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.214136] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AE7-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303082532, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893446, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.218951] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22455][T1003_TenantWea][T1003][YB4A0A000407-00061E9B3ABCFB4A-0-0] [lt=28] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.125743007659912109e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=306, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.980124782037972636e+03) [2024-10-16 14:57:15.219112] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061835218066000, start_id=1729061835218066000, end_id=1729061835218066001) [2024-10-16 14:57:15.225082] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AE9-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303093477, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893447, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.227425] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AEB-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303095819, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893448, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.238109] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AED-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303106503, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893449, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.238692] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE762B-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:15.242474] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1029-0-0] [lt=9] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:15.244203] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AEF-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303112597, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893450, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.248405] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.255861] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AF1-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303124254, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893451, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.257469] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AF3-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303125863, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893452, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.258535] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.273409] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061834973282000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=300119, in_cluster_service=true, cluster_version={val:1729061834924207000, v:0}, min_cluster_version={val:1729061834924207000, v:0}, max_cluster_version={val:1729061834924207000, v:0}, get_cluster_version_err=0, cluster_version_delta=349194, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244196, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=209669572, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:15.280330] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2244-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303148723, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893453, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.280677] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2055) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=12] start schedule all tablet merge(merge_version=1729015204839740000, medium_ls_tablet_iter_={ls_idx:-1, ls_ids:[{id:1}, {id:1001}, {id:1002}, {id:1003}], tablet_idx:0, tablet_ids_.count():0, schedule_tablet_cnt:0, max_batch_tablet_cnt:50000}) [2024-10-16 14:57:15.280930] INFO [STORAGE] get_adaptive_merge_reason (ob_partition_merge_policy.cpp:1751) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=16] Check tablet adaptive merge reason(ret=0, ls_id={id:1}, tablet_id={id:60379}, reason=0, tablet_analyzer={tablet_stat:{ls_id:1, tablet_id:60379, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, total_tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_small_tenant:false, boost_factor:"1.000000000000000000e+00"}, crazy_medium_flag=false) [2024-10-16 14:57:15.290582] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1169129472, global_cache_size=1598029824, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:15.290888] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash memory, (tenant_id=1002, cache_size=104038400, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:57:15.291019] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Wash time detail, (compute_wash_size_time=107, refresh_score_time=75, wash_time=215) [2024-10-16 14:57:15.293803] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=5] Cache wash clean map node details(ret=0, clean_node_count=34, clean_time=0, clean_start_pos=0, clean_num=62914) [2024-10-16 14:57:15.304075] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=44] batch clear tablet stat in ls(ret=0, ls_id={id:1}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:57:15.304075] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C36-0-0] [lt=5] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:15.304103] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C36-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061835120915003, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061835120915003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:15.304173] INFO [SHARE] batch_update_unequal_report_scn_tablet (ob_tablet_meta_table_compaction_operator.cpp:190) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=17] start to update unequal tablet id array(ret=0, ret="OB_SUCCESS", ls_id={id:1}, major_frozen_scn=1729015204839740000, input_tablet_id_array_cnt=681) [2024-10-16 14:57:15.305347] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C36-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061835305113000, start_id=1729061835305113000, end_id=1729061835305113001) [2024-10-16 14:57:15.307385] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5028, mem_used=10398720) [2024-10-16 14:57:15.309437] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AF4-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303177831, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893454, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.309868] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3B878A60-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061835309549000, start_id=1729061835309549000, end_id=1729061835309549001) [2024-10-16 14:57:15.310378] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=12, avg_log_batch_cnt=0, total_group_log_size=52187, avg_group_log_size=4348) [2024-10-16 14:57:15.311120] INFO [STORAGE.TRANS] handle_2pc_prepare_request_impl_ (ob_two_phase_downstream_committer.cpp:350) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=16] committer is under logging(ret=0, *this={this:0x2b0e0c214b50, ref:4, trans_id:{txid:392202127}, tenant_id:1002, is_exiting:false, trans_expired_time:1729062435262631, cluster_version:17180066304, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1729061835262631}{self_ls_id:{id:1001}, session_id:3222199901, part_trans_action:3, pending_write:0, 2pc_role:"LEAF", ctx_tx_data_:{ctx_mgr_:0x2b0e26cb6030, tx_data_guard_:{tx_data:{tx_id:{txid:392202127}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:1729061835010555003, v:0}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:36834275278980458, replay_completeness_:1, upstream_state:"PREPARE", collected:[], rec_log_ts:{val:1729061835010555003, v:0}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1729061835307641, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008951550040 max_table_version=1729008951550040 trans_version={val:1729061835010555001, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202127} ls_id=1001 row_callback[alloc:313, free:0, unsubmit:0] redo[fill:323,sync_succ:236, sync_fail:0] main_list_len=323 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,323,323,236,323,0,0,0)|]}} -{busy_cbs_cnt:1, oldest_busy_cb:{ObTxBaseLogCb:{base_ts:{val:1729061835010555001, v:0}, log_ts:{val:1729061835010555004, v:0}, lsn:{lsn:204021766338}, submit_ts:1729061835310970}, this:0x2b0e0c217aa8, is_inited_:true, trans_id:{txid:392202127}, ls_id:{id:1001}, ctx:0x2b0e0c214b50, tx_data_guard:{tx_data:NULL}, is_callbacked_:false, is_dynamic_:false, mds_range_:{range_array_.count():0, range_array_:[]}, cb_arg_array_:[{log_type:0x1, arg:null}, {log_type:0x40, arg:null}, {log_type:0x80, arg:null}], first_part_scn_:{val:18446744073709551615, v:3}, extra_cb_:null, need_free_extra_cb_:false, callbacks_.count():1}, -{info_log_submitted:1 state_log_submitting:1 state_log_submitted:1 }-, -{downstream_state:"INIT", upstream:{id:1003}, participants:[], redo_log_no:1, scheduler:"10.0.4.9:2890", prepare_version:{val:1729061835010555004, v:0}, trans_type:2, next_log_entry_no:2, max_applied_log_ts:{val:1729061835010555003, v:0}, max_appling_log_ts:{val:1729061835010555003, v:0}, max_applying_part_log_no:0, max_submitted_seq_no:{branch:0, seq:51114}, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, redo_lsns:[{lsn:204021731827}], max_durable_lsn:{lsn:204021731827}, }- }-}) [2024-10-16 14:57:15.312234] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] rpc statistics(total_trans_msg_count=3, total_batch_msg_count=3) [2024-10-16 14:57:15.312279] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:15.312313] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2182) [2024-10-16 14:57:15.315853] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCEFB-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=34, avg_log_gen_to_submit_time=70, avg_log_submit_to_flush_time=565, avg_log_submit_to_slide_time=1018) [2024-10-16 14:57:15.320314] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:15.322858] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C36-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061835321982000, start_id=1729061835321982000, end_id=1729061835321982001) [2024-10-16 14:57:15.326062] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061835324091000, start_id=1729061835324091000, end_id=1729061835324091001) [2024-10-16 14:57:15.326785] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2091) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=8] try to update report scn as ls leader(tmp_ret=0, ls_id={id:1}) [2024-10-16 14:57:15.327488] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061835326179000, start_id=1729061835326179000, end_id=1729061835326179001) [2024-10-16 14:57:15.328207] INFO [STORAGE] find_adaptive_merge_tables (ob_partition_merge_policy.cpp:1633) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=14] no enough table or no enough rows for meta merge(ret=-4677, scanty_tx_determ_table=false, scanty_inc_row_cnt=true, inc_row_cnt=442, base_inc_row_cnt=90523, result={version_range:{multi_version_start:1729061761547003000, base_version:0, snapshot_version:1729061761547003000}, scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729061778253675001, v:0}}, merge_version:0, handle:{tablet_id:{id:1152921504606846983}, table_count:3, [{i:0, table_key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref:0}{i:1, table_key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref:0}{i:2, table_key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref:0}]}, update_tablet_directly:false, schedule_major:false, is_backfill:false, backfill_scn:{val:18446744073709551615, v:3}}, PRINT_TS_WRAPPER(table_store_wrapper)={ObTabletTableStore_Pretty:this:0x2b0e2a14e1f8, major_tables:{cnt:1, sstable_array:0x2b0df1ca6810, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0df1ca7298, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0df1ca8516, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22248] [ YB4A0A000407-00061E9973CF3855-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22248] [ YB4A0A000407-00061E9973CF3855-0-0 ] MAJOR MAJOR 0 1729015201157706000 0 1729015204839740000 0 false [22248] [ YB4A0A000407-00061E9973CF3855-0-0 ] MINOR MINOR 1729061401350833000 1729061401350833000 1729015204702466001 1729061414084957000 0 false [22248] [ YB4A0A000407-00061E9973CF3855-0-0 ] MINI 1729061761547003000 1729061761547003000 1729061414084957000 1729061778253675001 0 false [22248] [ YB4A0A000407-00061E9973CF3855-0-0 ] MDS MDS_MINOR 1717560328401377000 1717560328401377000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:15.328276] WDIAG [STORAGE.COMPACTION] decide_medium_snapshot (ob_medium_compaction_func.cpp:594) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=67][errcode=-4677] failed to choose medium scn for medium(ret=-4677, this={ls_id:{id:1001}, tablet_id:{id:1152921504606846983}}) [2024-10-16 14:57:15.328421] WDIAG [STORAGE.COMPACTION] decide_medium_snapshot (ob_medium_compaction_func.cpp:594) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=9][errcode=-4677] failed to choose medium scn for medium(ret=-4677, this={ls_id:{id:1001}, tablet_id:{id:1152921504606846990}}) [2024-10-16 14:57:15.329369] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [106956][][T1003][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1003, ret=0, thread_count_=173) [2024-10-16 14:57:15.329412] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] worker thread created(tenant_->id()=1003, group_id_=1, token=1) [2024-10-16 14:57:15.329442] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [106956][][T1003][Y0-0000000000000000-0-0] [lt=71] Init thread local success [2024-10-16 14:57:15.329465] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [106956][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCF68-0-0] [lt=5] init point thread id with(&point=0x55e3435c2180, idx_=460, point=[thread id=106956, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=106956) [2024-10-16 14:57:15.337869] WDIAG [STORAGE.COMPACTION] decide_medium_snapshot (ob_medium_compaction_func.cpp:594) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=8][errcode=-4677] failed to choose medium scn for medium(ret=-4677, this={ls_id:{id:1001}, tablet_id:{id:1152921504606846984}}) [2024-10-16 14:57:15.338287] WDIAG [STORAGE.COMPACTION] decide_medium_snapshot (ob_medium_compaction_func.cpp:594) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=21][errcode=-4677] failed to choose medium scn for medium(ret=-4677, this={ls_id:{id:1001}, tablet_id:{id:1152921504606846989}}) [2024-10-16 14:57:15.338419] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AF6-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303206812, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893455, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.338949] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA0-0-0] [lt=22] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:15.338966] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA0-0-0] [lt=13] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=18) [2024-10-16 14:57:15.339472] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=9] batch clear tablet stat in ls(ret=0, ls_id={id:1001}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:57:15.339567] INFO [SHARE] batch_update_unequal_report_scn_tablet (ob_tablet_meta_table_compaction_operator.cpp:190) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=11] start to update unequal tablet id array(ret=0, ret="OB_SUCCESS", ls_id={id:1001}, major_frozen_scn=1729015204839740000, input_tablet_id_array_cnt=491) [2024-10-16 14:57:15.348472] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.353280] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AF9-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303221671, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893456, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.353890] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2091) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=7] try to update report scn as ls leader(tmp_ret=0, ls_id={id:1001}) [2024-10-16 14:57:15.354423] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=22] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=113, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:15.354438] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:15.355869] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:15.358612] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.360259] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=14] batch clear tablet stat in ls(ret=0, ls_id={id:1002}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:57:15.360281] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2091) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=15] try to update report scn as ls leader(tmp_ret=-4722, ls_id={id:1002}) [2024-10-16 14:57:15.362343] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AFD-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303230737, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893457, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.364379] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0AFF-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303232774, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893458, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.365497] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=6] batch clear tablet stat in ls(ret=0, ls_id={id:1003}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:57:15.365513] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2091) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=12] try to update report scn as ls leader(tmp_ret=-4722, ls_id={id:1003}) [2024-10-16 14:57:15.365521] INFO [STORAGE.COMPACTION] after_schedule_tenant_medium (ob_tenant_tablet_scheduler.cpp:2161) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=6] finish schedule all tablet merge(merge_version=1729015204839740000, schedule_stats_={schedule_dag_cnt:0, submit_clog_cnt:0, finish_cnt:2138, wait_rs_validate_cnt:0}, tenant_merge_finish=true, merged_version_=1729015204839740000, is_scan_all_tablet_finish=true, time_guard=) [2024-10-16 14:57:15.365530] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:103) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3855-0-0] [lt=9] MediumLoopTask(cost_ts=84893) [2024-10-16 14:57:15.365542] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3856-0-0] [lt=3] MediumCheckTask(cost_ts=5) [2024-10-16 14:57:15.366090] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=306, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:15.366109] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=112, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:15.366528] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B01-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303234922, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893459, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.371574] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BC6-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:15.372458] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B04-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303240853, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893460, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.373314] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B05-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303241710, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893461, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.374341] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=39] get number(ret=0, service_type_=0, range=1, base_id=1729061835374022000, start_id=1729061835374022000, end_id=1729061835374022001) [2024-10-16 14:57:15.378094] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B07-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303246489, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893462, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.378978] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061835378138000, start_id=1729061835378138000, end_id=1729061835378138001) [2024-10-16 14:57:15.379882] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:15.379929] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=35] get_max_applied_scn(scn={val:1729061835176500000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390019624}, palf_committed_end_scn_:{val:1729061835378138000, v:0}, last_check_scn_:{val:1729061835378138000, v:0}, max_applied_cb_scn_:{val:1729061835176500000, v:0}}) [2024-10-16 14:57:15.379946] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:15.380384] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:15.380405] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=209) [2024-10-16 14:57:15.383278] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:15.384659] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B09-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303253052, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893463, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.386506] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B0B-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303254900, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893464, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.387965] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:15.394814] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2246-0-0] [lt=82][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303263207, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893465, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.394814] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B0D-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303263208, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893466, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.395439] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=11] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:15.396960] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B0F-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303265354, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893467, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.402786] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B11-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303271181, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893468, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.406331] INFO [SHARE] get_server_tenant_configs (ob_unit_getter.cpp:231) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=5] get_server_tenant_configs(unit_infos=[{unit:unit_id:1, resource_pool_id:1, unit_group_id:1, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, pool:{resource_pool_id:1, name:"sys_pool", unit_count:1, unit_config_id:1, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1, replica_type:0}}, {unit:unit_id:1001, resource_pool_id:1001, unit_group_id:1001, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1001, name:"ocp_pool", unit_count:1, unit_config_id:1001, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1002, replica_type:0}}, {unit:unit_id:1005, resource_pool_id:1003, unit_group_id:1002, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1003, name:"pool_bddb_myoceanbase1_oav", unit_count:1, unit_config_id:1003, zone_list:["myoceanbase1"], tenant_id:1004, replica_type:0}}]) [2024-10-16 14:57:15.406373] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=38] jx_debug: get tenant compatibility mode(tenant_id=1002, compat_mode=0) [2024-10-16 14:57:15.406379] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=5] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:15.406391] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=11] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:15.406407] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=15] jx_debug: get tenant compatibility mode(tenant_id=1004, compat_mode=0) [2024-10-16 14:57:15.406412] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=5] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:15.406422] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:15.407828] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:191) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=26] refresh resource plan success(tenant_id=1, plan_name=, directives=[]) [2024-10-16 14:57:15.408542] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=24] refresh resource user mapping rule(tenant_id=1, plan=, user_rules=[]) [2024-10-16 14:57:15.409261] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] refresh_resource_function_mapping_rule(tenant_id=1, plan=, rules=[]) [2024-10-16 14:57:15.409272] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=10] refresh resource mapping rule success(tenant_id=1, plan=) [2024-10-16 14:57:15.410006] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=5] trace tenant dfc parameters(max_parallel_cnt_=30, max_blocked_buffer_size_=9830400, max_buffer_size_=19660800) [2024-10-16 14:57:15.410060] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=12] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:15.410068] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=6] tenant dfc status(ret=0, get_tenant_id()=1, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=30, get_max_blocked_buffer_size()=9830400, get_max_buffer_size()=19660800, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:15.412101] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=7] refresh resource user mapping rule(tenant_id=1001, plan=, user_rules=[]) [2024-10-16 14:57:15.412775] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] refresh_resource_function_mapping_rule(tenant_id=1001, plan=, rules=[]) [2024-10-16 14:57:15.412785] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] refresh resource mapping rule success(tenant_id=1001, plan=) [2024-10-16 14:57:15.413432] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:15.413494] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:15.413501] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=6] tenant dfc status(ret=0, get_tenant_id()=1001, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:15.416019] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B13-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303284414, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893469, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.416099] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=7] refresh resource user mapping rule(tenant_id=1002, plan=, user_rules=[]) [2024-10-16 14:57:15.416776] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=10] refresh_resource_function_mapping_rule(tenant_id=1002, plan=, rules=[]) [2024-10-16 14:57:15.416785] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] refresh resource mapping rule success(tenant_id=1002, plan=) [2024-10-16 14:57:15.417435] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=5] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:15.417505] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:15.417512] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=6] tenant dfc status(ret=0, get_tenant_id()=1002, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=4, get_max_size_per_channel()=655360) [2024-10-16 14:57:15.419466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061835418230000, start_id=1729061835418230000, end_id=1729061835418230001) [2024-10-16 14:57:15.419497] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=7] refresh resource user mapping rule(tenant_id=1003, plan=, user_rules=[]) [2024-10-16 14:57:15.419529] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:15.419544] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:15.420191] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=11] refresh_resource_function_mapping_rule(tenant_id=1003, plan=, rules=[]) [2024-10-16 14:57:15.420202] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=11] refresh resource mapping rule success(tenant_id=1003, plan=) [2024-10-16 14:57:15.420851] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:15.420912] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=10] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:15.420926] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=13] tenant dfc status(ret=0, get_tenant_id()=1003, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:15.422942] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=10] refresh resource user mapping rule(tenant_id=1004, plan=, user_rules=[]) [2024-10-16 14:57:15.423648] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=15] refresh_resource_function_mapping_rule(tenant_id=1004, plan=, rules=[]) [2024-10-16 14:57:15.423660] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=12] refresh resource mapping rule success(tenant_id=1004, plan=) [2024-10-16 14:57:15.424742] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:15.424807] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=14] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:15.424818] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=10] tenant dfc status(ret=0, get_tenant_id()=1004, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:15.424834] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:108) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=10] refresh tenant units(sys_unit_cnt=3, units=[{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, {tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, {tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}], ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:15.424869] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=31] finish set nway(tenant_id_=1, nway=3) [2024-10-16 14:57:15.424874] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=4] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1, nway=3, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}) [2024-10-16 14:57:15.424887] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=12] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=1.638399963378906250e+02, this=0x2b0ddc2dc170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=819, Throttle Trigger(MB)=491, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.948457122979588263e+01) [2024-10-16 14:57:15.424902] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=14] succ to set_memstore_threshold of memstore allocator(tenant_id=1, ret=0) [2024-10-16 14:57:15.424907] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=3] finish set nway(tenant_id_=1002, nway=1) [2024-10-16 14:57:15.424913] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=5] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1002, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:15.424923] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e0c298170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:15.424932] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] succ to set_memstore_threshold of memstore allocator(tenant_id=1002, ret=0) [2024-10-16 14:57:15.424937] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=3] finish set nway(tenant_id_=1001, nway=1) [2024-10-16 14:57:15.424942] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=5] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1001, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:15.424952] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0df7098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:15.424958] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=6] succ to set_memstore_threshold of memstore allocator(tenant_id=1001, ret=0) [2024-10-16 14:57:15.424962] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=2] finish set nway(tenant_id_=1004, nway=1) [2024-10-16 14:57:15.424964] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1004, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:15.424973] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=8] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e40098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:15.424978] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=5] succ to set_memstore_threshold of memstore allocator(tenant_id=1004, ret=0) [2024-10-16 14:57:15.424986] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=6] finish set nway(tenant_id_=1003, nway=1) [2024-10-16 14:57:15.424988] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1003, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:15.424998] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e2c698170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:15.425006] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=8] succ to set_memstore_threshold of memstore allocator(tenant_id=1003, ret=0) [2024-10-16 14:57:15.425626] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=2] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:15.425639] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=12] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1001, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:15.425644] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=4] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:15.425648] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1003, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:15.425651] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:15.425671] INFO [SERVER] cal_all_part_disk_default_percentage (ob_server_utils.cpp:301) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=3] cal_all_part_disk_default_percentage succ(data_dir="/mnt/myoceanbase/oceanbase/data//sstable", clog_dir="/mnt/myoceanbase/oceanbase/data//clog", shared_mode=true, data_disk_total_size=214429704192, data_disk_default_percentage=60, clog_disk_total_size=214429704192, clog_disk_default_percentage=30) [2024-10-16 14:57:15.425677] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:337) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=6] decide disk size finished(suggested_disk_size=32212254720, suggested_disk_percentage=0, default_disk_percentage=30, total_space=214429704192, disk_size=32212254720) [2024-10-16 14:57:15.425682] INFO [SERVER] get_log_disk_info_in_config (ob_server_utils.cpp:88) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=4] get_log_disk_info_in_config(suggested_data_disk_size=32212254720, suggested_clog_disk_size=32212254720, suggested_data_disk_percentage=0, suggested_clog_disk_percentage=0, log_disk_size=32212254720, log_disk_percentage=0, total_log_disk_size=214429704192) [2024-10-16 14:57:15.425689] INFO [CLOG] try_resize (ob_server_log_block_mgr.cpp:800) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=5] try_resize success(ret=0, log_disk_size=32212254720, total_log_disk_size=214429704192, this={dir::"/mnt/myoceanbase/oceanbase/data//clog/log_pool", dir_fd:15, meta_fd:16, log_pool_meta:{curr_total_size:32212254720, next_total_size:32212254720, status:0}, min_block_id:15036, max_block_id:15335, min_log_disk_size_for_all_tenants_:15032385536, is_inited:true}) [2024-10-16 14:57:15.425697] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:132) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C09-0-0] [lt=8] refresh tenant config(tenants=[1, 1001, 1002, 1003, 1004], ret=0) [2024-10-16 14:57:15.427801] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B15-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303296194, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893470, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.427879] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061835010555003, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=417318, in_cluster_service=true, cluster_version={val:1729061834619448004, v:0}, min_cluster_version={val:1729061834619448004, v:0}, max_cluster_version={val:1729061834619448004, v:0}, get_cluster_version_err=0, cluster_version_delta=808425, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65227976, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=380819394, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:15.429827] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B17-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303298222, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893471, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.431555] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B19-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303299949, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893472, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.438322] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=37] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:15.438954] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:15.439000] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061828711668000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:15.439567] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F384D-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:15.439578] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F384D-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=16850, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:15.439609] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=53953, hit_cnt=15, read_cnt=15, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:15.441744] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A65-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:15.441762] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A65-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:15.444196] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B1B-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303312590, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893473, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.445746] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B1D-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303314141, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893474, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.448522] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.454396] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A47-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178194, log_task={header:{begin_lsn:{lsn:14538456209}, end_lsn:{lsn:14538456331}, log_id:115178194, min_scn:{val:1729061835119854002, v:0}, max_scn:{val:1729061835119854002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538456087}, prev_proposal_id:23, committed_end_lsn:{lsn:14538456209}, data_checksum:1715074492, accum_checksum:1674580961, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061835354360, freeze_ts:1729061835354360, submit_ts:1729061835354364, flushed_ts:1729061835355894, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1530}, fs_cb_begin_ts=1729061835454384, log_life_time=100024) [2024-10-16 14:57:15.454535] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A49-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=25, avg_log_submit_to_flush_time=442, avg_log_submit_to_slide_time=770) [2024-10-16 14:57:15.454683] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:7668488755}, base_lsn={lsn:7582838784}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:15.454756] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=72] get_replay_process success(id={id:1}, replayed_log_size=85649971, unreplayed_log_size=0) [2024-10-16 14:57:15.454764] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:6198988121}, base_lsn={lsn:6106533888}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:15.454785] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=22] get_replay_process success(id={id:1001}, replayed_log_size=92454233, unreplayed_log_size=0) [2024-10-16 14:57:15.454790] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=8163503371, unreplayed_log_size=0) [2024-10-16 14:57:15.454795] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1003}, replayed_log_size=8111599030, unreplayed_log_size=0) [2024-10-16 14:57:15.454810] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1004, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=15691, last_replayed_log_size(MB)=15690, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:15.455167] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F384E-0-0] [lt=26] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=37, avg_log_submit_to_flush_time=594, avg_log_submit_to_slide_time=869) [2024-10-16 14:57:15.455831] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B1F-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303324225, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893475, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.458672] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.463052] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B21-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303331446, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893476, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.463129] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061835462262000, start_id=1729061835462262000, end_id=1729061835462262001) [2024-10-16 14:57:15.470036] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B26-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303338429, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893477, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.470906] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218496435, log_task={header:{begin_lsn:{lsn:207588186309}, end_lsn:{lsn:207588186481}, log_id:218496435, min_scn:{val:1729061835309549002, v:0}, max_scn:{val:1729061835309549002, v:0}, data_len:116, proposal_id:81, prev_lsn:{lsn:207588186030}, prev_proposal_id:81, committed_end_lsn:{lsn:207588186309}, data_checksum:732153500, accum_checksum:1713249829, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061835328275, freeze_ts:1729061835328275, submit_ts:1729061835328279, flushed_ts:1729061835328595, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:316}, fs_cb_begin_ts=1729061835470899, log_life_time=142624) [2024-10-16 14:57:15.472468] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BC8-0-0] [lt=38] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:15.472481] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BC8-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30624, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:15.472485] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BC8-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=8, avg_log_submit_to_flush_time=635, avg_log_submit_to_slide_time=27899) [2024-10-16 14:57:15.472487] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:15.472492] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BC8-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172887, log_task={header:{begin_lsn:{lsn:14352342454}, end_lsn:{lsn:14352342576}, log_id:115172887, min_scn:{val:1729061835120915002, v:0}, max_scn:{val:1729061835120915002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352342332}, prev_proposal_id:17, committed_end_lsn:{lsn:14352342454}, data_checksum:940009244, accum_checksum:3091194841, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061835371564, freeze_ts:1729061835371564, submit_ts:1729061835371569, flushed_ts:1729061835374749, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:3180}, fs_cb_begin_ts=1729061835472460, log_life_time=100896) [2024-10-16 14:57:15.474108] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B28-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303342502, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893478, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.474549] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.474567] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.474574] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044559560704},{"BIGINT":1729058400},{"VARCHAR":"?±ë…¸Qì?®¸Që…¸?®¸Që…¸?®¸Që…¸?©™™™™™š?®¸Që…¸?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?· =p£× ?±ë…¸Qì?´záG®{?®¸Që…¸?©™™™™™š?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?´záG®{?´záG®{?´záG®{?Ç =p£× ?±ë…¸Qì?±ë…¸Qì?®¸Që…¸?®¸Që…¸?©™™™™™š?Íp£× =q?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?· =p£× ?´záG®{?¼(õÂ\)?´záG®{?©™™™™™š?®¸Që…¸?´záG®{?· =p£× ?´záG®{?· =p£× ?´záG®{?Áë…¸Qì?´záG®{?±ë…¸Qì?±ë…¸Qì?´záG®{?´záG®{?À£× =p¤?· =p£× ?· =p£× ?Õ¸Që…?®¸Që…¸¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.474645] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=39][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.474888] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284901} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.475185] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:15.475195] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:15.475205] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:15.475209] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:15.475202] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A20-0-0] [lt=7] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:15.475213] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:15.475216] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:15.475218] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:15.475216] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A20-0-0] [lt=12] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:15.475220] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:15.475223] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:15.475225] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:15.475228] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:15.478936] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061835478278000, start_id=1729061835478278000, end_id=1729061835478278001) [2024-10-16 14:57:15.484945] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.484960] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.484967] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044697972736},{"BIGINT":1729058400},{"VARCHAR":"?±ë…¸Qì?®¸Që…¸?®¸Që…¸?®¸Që…¸?©™™™™™š?®¸Që…¸?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?· =p£× ?±ë…¸Qì?´záG®{?®¸Që…¸?©™™™™™š?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?´záG®{?´záG®{?´záG®{?Ç =p£× ?±ë…¸Qì?±ë…¸Qì?®¸Që…¸?®¸Që…¸?©™™™™™š?Íp£× =q?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?· =p£× ?´záG®{?¼(õÂ\)?´záG®{?©™™™™™š?®¸Që…¸?´záG®{?· =p£× ?´záG®{?· =p£× ?´záG®{?Áë…¸Qì?´záG®{?±ë…¸Qì?±ë…¸Qì?´záG®{?´záG®{?À£× =p¤?· =p£× ?· =p£× ?Õ¸Që…?®¸Që…¸¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.485027] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.485065] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B2A-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303353459, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893479, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.485314] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284903} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.487215] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061835486815000, start_id=1729061835486815000, end_id=1729061835486815001) [2024-10-16 14:57:15.487311] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B2C-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303355706, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893480, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.487789] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:15.487807] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:15.487837] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=70411428, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:15.488553] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.488567] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.488574] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044748304384},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.488640] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=41][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.488831] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284904} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.494091] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1175420928, global_cache_size=1595949056, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:15.495549] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B2E-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303363942, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893481, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.497258] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.497276] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.497283] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044832190465},{"BIGINT":1729058400},{"VARCHAR":"?Ù™™™™™š?ØQë…¸R?Öffffff?ØõÂ\(ö?× =p£× ?ÕÂ\(õÃ?× =p£× ?×®záG®?ØQë…¸R?ØQë…¸R?ÜÌÌÌÌÌÍ?×®záG®?×®záG®?ØõÂ\(ö?Öffffff?Ù™™™™™š?ØQë…¸R?×®záG®?× =p£× ?ØõÂ\(ö?ØQë…¸R?Ú=p£× =?Öffffff?× =p£× ?×®záG®?×®záG®?ÕÂ\(õÃ?ÕÂ\(õÃ?à£× =p¤?× =p£× ?Öffffff?ØõÂ\(ö?×®záG®?Õ¸Që…?ÕÂ\(õÃ?Ù™™™™™š?Û…¸Që…?ÕÂ\(õÃ?à", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.497346] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.497674] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2127}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284906} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.499274] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:15.508545] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=9] replay status is not follower(min_unreplayed_lsn={lsn:45686897415}, base_lsn={lsn:45497032704}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:15.508612] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=66] get_replay_process success(id={id:1}, replayed_log_size=189864711, unreplayed_log_size=0) [2024-10-16 14:57:15.508631] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=6] dump tenant replay process(tenant_id=1001, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=181, last_replayed_log_size(MB)=181, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:15.510415] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B30-0-0] [lt=32][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303378806, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893482, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.510954] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061835510227000, start_id=1729061835510227000, end_id=1729061835510227001) [2024-10-16 14:57:15.510982] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061835510977], gts:1729061835510227000, latest_srr:[mts=1729061835510977]}) [2024-10-16 14:57:15.512946] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B32-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303381339, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893483, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.513179] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.513190] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.513197] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044962213888},{"BIGINT":1729058400},{"VARCHAR":"Aló÷ =qAlÿ€£× Al £× Al¸£× Al"dzáHAl-ú\)Al9¹™™šAlE^¸QìAlQ× >Al\°", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.513254] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.513625] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3346}, to_seq_no={branch:0, seq:1204}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284910} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.515710] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [22809][T1004_SSTableGC][T1004][YB4A0A000407-00061E997A9CB61B-0-0] [lt=11] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60379}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015311100270042}, old_min_reserved_snapshot=1729060031822855000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060031822855000) [2024-10-16 14:57:15.525752] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061835010555002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=515190, in_cluster_service=true, cluster_version={val:1729061834620560004, v:0}, min_cluster_version={val:1729061834620560004, v:0}, max_cluster_version={val:1729061834620560004, v:0}, get_cluster_version_err=0, cluster_version_delta=905185, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65243896, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=506855584, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:15.526720] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061835514617000, start_id=1729061835514617000, end_id=1729061835514617001) [2024-10-16 14:57:15.529760] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061835514617000, start_id=1729061835514617000, end_id=1729061835514617001) [2024-10-16 14:57:15.531583] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061835530744000, start_id=1729061835530744000, end_id=1729061835530744001) [2024-10-16 14:57:15.536776] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=906907648, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=166834176, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:15.536829] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=23] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:15.536837] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:15.536873] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=973893632, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=99848192, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:15.536920] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:15.536927] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:15.536971] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=942559232, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=131182592, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:15.537013] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:15.537018] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011850, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:15.537049] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=952647680, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=121094144, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:15.537090] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:15.537094] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:15.539278] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2248-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303407672, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893484, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.540942] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF224A-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303409338, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893485, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.542777] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106746][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC34-0-0] [lt=10] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:15.542794] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106746][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC34-0-0] [lt=13] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:15.542811] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106746][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC34-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:15.542906] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D53-0-0] [lt=21] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:15.542922] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D53-0-0] [lt=8] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061835119854003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061835119854003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:15.543828] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [22809][T1004_SSTableGC][T1004][YB4A0A000407-00061E997A9CB61B-0-0] [lt=16] SSTableGCTask(cost_ts=28307) [2024-10-16 14:57:15.546064] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D53-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061835545088000, start_id=1729061835545088000, end_id=1729061835545088001) [2024-10-16 14:57:15.549007] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.550188] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B34-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303418582, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893486, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.550593] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D53-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061835550202000, start_id=1729061835550202000, end_id=1729061835550202001) [2024-10-16 14:57:15.552938] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B36-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303421333, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893487, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.555563] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.555577] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.555583] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045088043008},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.555643] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.555856] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1204}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284912} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.556412] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B38-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303424806, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893488, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.559464] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:15.559484] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:15.559490] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.564134] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:15.564154] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:15.565800] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061835565319000, start_id=1729061835565319000, end_id=1729061835565319001) [2024-10-16 14:57:15.565931] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:57:15.565943] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:57:15.565946] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:15.566565] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061847008855000, latest_log_ts:{val:1729061830358352001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:15.566673] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=68] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=27, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:15.566689] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=5070) [2024-10-16 14:57:15.566696] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:15.567048] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=5070) [2024-10-16 14:57:15.567262] INFO [LIB] stat (utility.h:1165) [106635][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45AD9-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:15.567272] INFO [LIB] stat (utility.h:1165) [106635][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45AD9-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=906, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:15.567865] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.567880] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.567887] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045176123392},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.567955] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.568796] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.568797] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.568817] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.569073] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.569230] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3502}, to_seq_no={branch:0, seq:1336}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284915} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.571951] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:15.574592] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.574614] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.574620] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045402615808},{"BIGINT":1729058400},{"VARCHAR":"AAoŸÌÌÌÍAAo§J=p¤AAo®šáG®AAoµà", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.574682] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.575151] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1047}, to_seq_no={branch:0, seq:1040}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284917} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.575214] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061835574161000, start_id=1729061835574161000, end_id=1729061835574161001) [2024-10-16 14:57:15.578284] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1002}, {id:1003}]) [2024-10-16 14:57:15.578335] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=43] no need to update min/max version(new_version={val:1729061835176500000, v:0}, min_version={val:1729061835176500000, v:0}, max_version={val:1729061835176500000, v:0}) [2024-10-16 14:57:15.581175] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B3A-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303449567, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893489, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.585349] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B3C-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303453742, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893490, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.586937] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B3E-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303455332, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893491, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.588047] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:15.592836] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.592854] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.592870] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045536833537},{"BIGINT":1729058400},{"VARCHAR":"BŸ¹òµ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.592957] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.593208] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5324}, to_seq_no={branch:0, seq:1114}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284921} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.595208] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061835594388000, start_id=1729061835594388000, end_id=1729061835594388001) [2024-10-16 14:57:15.595503] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4443, mem_used=8318976) [2024-10-16 14:57:15.596782] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.596797] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.596803] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045566193664},{"BIGINT":1729058400},{"VARCHAR":"BŸ¹õh", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.596854] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.597051] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284922} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.604678] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B40-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303473072, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893492, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.604981] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=26] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:15.605003] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=66886) [2024-10-16 14:57:15.605224] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=257, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:15.605240] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=30, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:15.605273] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:15.605470] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=68510, hit_cnt=20, read_cnt=20, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:15.610823] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B42-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303479217, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893493, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.610974] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.610984] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.610990] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045654274048},{"BIGINT":1729058400},{"VARCHAR":"Aô_9XbAô_r° ÅAô_‡+ Aô_ãS÷ÏAôb¸QìAôixÔýôAôp‡+AôvÐå`BAôx¡ÊÀƒAôz/Ÿ¿Aô|", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.611049] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.611295] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2146}, to_seq_no={branch:0, seq:1080}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284925} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.614726] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B44-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303483121, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893494, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.620741] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.620763] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.620773] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045729771520},{"BIGINT":1729058400},{"VARCHAR":"A°«Ó+", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.620917] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=40][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.621029] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B51-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061835620865000, start_id=1729061835620865000, end_id=1729061835620865001) [2024-10-16 14:57:15.621269] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B52-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061835620865000, start_id=1729061835620865000, end_id=1729061835620865001) [2024-10-16 14:57:15.621283] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2121}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284927} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.621491] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B55-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061835620865000, start_id=1729061835620865000, end_id=1729061835620865001) [2024-10-16 14:57:15.623456] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B46-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303491850, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893495, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.626753] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B48-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303495147, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893496, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.628614] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B4A-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303497009, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893497, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.630163] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B4C-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303498559, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893498, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.630880] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get_max_applied_scn(scn={val:1729061835530744000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264458443}, palf_committed_end_scn_:{val:1729061835550202000, v:0}, last_check_scn_:{val:1729061835550202000, v:0}, max_applied_cb_scn_:{val:1729061835530744000, v:0}}) [2024-10-16 14:57:15.630901] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:15.640883] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=42] success to update min reserved snapshot(reserved_snapshot=1729060030183137000, duration=1800, snapshot_gc_ts=1729061830183137000) [2024-10-16 14:57:15.641455] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=17] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060030183137000) [2024-10-16 14:57:15.642479] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=16] submit reserved snapshot log success(ls_id={id:1001}, new_snapshot_version=1729060030183137000) [2024-10-16 14:57:15.642495] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=14] update reserved snapshot finished(cost_ts=1616, reserved_snapshot=1729060030183137000) [2024-10-16 14:57:15.645779] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:111799114, max_lsn:17493428722, max_scn:{val:1729061835510227001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7667642653}, reuse_lsn:{lsn:17493428722}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17451485682}}, last_submit_lsn:{lsn:17493428600}, last_submit_end_lsn:{lsn:17493428722}, last_submit_log_id:111799114, last_submit_log_pid:17, max_flushed_lsn:{lsn:17493428600}, max_flushed_end_lsn:{lsn:17493428722}, max_flushed_log_pid:17, committed_end_lsn:{lsn:17493428722}, last_slide_log_id:111799114, last_slide_scn:{val:1729061835510227001, v:0}, last_slide_lsn:{lsn:17493428600}, last_slide_end_lsn:{lsn:17493428722}, last_slide_log_pid:17, last_slide_log_accum_checksum:1457712300, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431d4530}, [StateMgr]={this:0x2b0e431d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:111799115, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261853177, palf_role_change_cb:0x2b0e431df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:18}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:18}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494261753074, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494261753074, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262053331, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728249494772393002, v:0}, min_block_id:241, min_block_min_scn:{val:1728207118648006001, v:0}, base_lsn_for_block_gc:{lsn:17380134912}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:17380134912}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:757760}, readable_log_tail:{lsn:757760}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17514344448}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/meta", dir_fd:193, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66347008, block_header_serialize_buf_:0x2b0e431da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17493428722}, readable_log_tail:{lsn:17493428722}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17447239680}, min_scn:{val:1729032503563174001, v:0}, curr_block_id:260, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/log", dir_fd:195, min_block_id:241, max_block_id:261, curr_writable_block_id:260}, logical_block_size_:67104768, curr_block_writable_size_:20915726, block_header_serialize_buf_:0x2b0e431dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261852794, this:0x2b0e431d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:7668488755}, saved_end_lsn:{lsn:7668488755}, last_submit_prepare_req_time_us:1722494261853360, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431d89b0}) [2024-10-16 14:57:15.645918] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=135] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061835613953, last_flushed_end_lsn:{lsn:17493428722}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061835614869, last_flushed_end_lsn:{lsn:17493428722}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061835614918, last_flushed_end_lsn:{lsn:17493428722}}]) [2024-10-16 14:57:15.645929] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=111799115, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061835613953, last_flushed_end_lsn:{lsn:17493428722}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061835614869, last_flushed_end_lsn:{lsn:17493428722}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061835614918, last_flushed_end_lsn:{lsn:17493428722}}]) [2024-10-16 14:57:15.648723] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:15.651284] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.656419] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=6] query dest_info_ succ(dest_info=[[0, "file:///mnt/oceanbasedata/bddb/archive"]], archive_dest_changed=false) [2024-10-16 14:57:15.656436] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=15] finish to resize log external storage handler(current_ts=1729061835656434, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:15.656779] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=215, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:15.656794] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=246, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:15.656819] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:15.659530] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.666780] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061835666546000, start_id=1729061835666546000, end_id=1729061835666546001) [2024-10-16 14:57:15.667164] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4596) [2024-10-16 14:57:15.668234] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.668251] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.668264] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046145007616},{"BIGINT":1729058400},{"VARCHAR":"B¯jnè", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.668343] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=35][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.668844] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2143}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284936} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.670628] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061831652120, last_first_register_time_us=1722494150998907, register_time_us=1729061831652120, curr_time_us=1729061835670627) [2024-10-16 14:57:15.670683] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=23] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061835670652, last_submit_register_req_time_us=1729061835670652, register_parent_reason=3) [2024-10-16 14:57:15.670693] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:15.671337] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=26, stat_interval=1000000, avg (count/cost)=263, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:15.671353] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=26, stat_interval=1000000, avg (count/cost)=75, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:15.671375] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=26, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:15.672020] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=332, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:15.672034] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=271, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:15.672062] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:15.672443] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [106960][][T1][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1, ret=0, thread_count_=241) [2024-10-16 14:57:15.672466] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [106960][][T1][Y0-0000000000000000-0-0] [lt=21] Init thread local success [2024-10-16 14:57:15.672481] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=7] worker thread created(tenant_->id()=1, group_id_=1, token=1) [2024-10-16 14:57:15.679412] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061835679300000, start_id=1729061835679300000, end_id=1729061835679300001) [2024-10-16 14:57:15.682321] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B52-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303550715, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893499, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.684331] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.684350] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.684360] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":674740107403186176},{"BIGINT":1729058400},{"VARCHAR":"B;›°", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.684433] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.685529] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2064}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284940} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.687917] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=46] get number(ret=0, service_type_=0, range=1, base_id=1729061835687704000, start_id=1729061835687704000, end_id=1729061835687704001) [2024-10-16 14:57:15.689822] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B53-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303558216, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893500, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.690421] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.690436] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.690445] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":680985887415848961},{"BIGINT":1729058400},{"VARCHAR":"B6,", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.690506] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.690768] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1058}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284941} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.694427] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1175420928, global_cache_size=1595949056, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:15.695093] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:15.698625] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B55-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303567018, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893501, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.701129] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B57-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303569523, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893502, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.707489] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B59-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303575882, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893503, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.712025] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B5B-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303580418, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893504, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.712945] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.712964] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.712974] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046623158272},{"BIGINT":1729058400},{"VARCHAR":"BHö€Ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.713042] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.713257] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2251}, to_seq_no={branch:0, seq:1189}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284946} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.713394] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:15.713412] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:15.715532] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061835715402000, start_id=1729061835715402000, end_id=1729061835715402001) [2024-10-16 14:57:15.720140] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:15.720756] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B5D-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303589150, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893505, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.721105] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:15.721123] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=209) [2024-10-16 14:57:15.725860] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1002, min_version={val:1729061835010555004, v:0}, base_version={val:1729061835010555002, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061835010555004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061835725847, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061835482557000, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061835662122, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061835471806001, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061835666687, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:15.726794] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061835724940000, start_id=1729061835724940000, end_id=1729061835724940001) [2024-10-16 14:57:15.729531] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B5F-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303597925, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893506, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.730445] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061835730185000, start_id=1729061835730185000, end_id=1729061835730185001) [2024-10-16 14:57:15.730721] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.730777] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=53][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.730785] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046799319040},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.730841] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.731813] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=42] get number(ret=0, service_type_=0, range=1, base_id=1729061835731245000, start_id=1729061835731245000, end_id=1729061835731245001) [2024-10-16 14:57:15.732077] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4323}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284950} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.735594] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061835735492000, start_id=1729061835735492000, end_id=1729061835735492001) [2024-10-16 14:57:15.737112] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B61-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303605506, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893507, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.739133] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B63-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303607526, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893508, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.739502] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257650, offset=39001419}, end_cursor:ObLogCursor{file_id=197, log_id=49257651, offset=39001702}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a60050, len:693, log_data_len:283, seq:49257650, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:15.741778] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB355-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061835741746000, start_id=1729061835741746000, end_id=1729061835741746001) [2024-10-16 14:57:15.742949] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=24] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257651, offset=39001702}, end_cursor:ObLogCursor{file_id=197, log_id=49257652, offset=39001984}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a94050, len:410, log_data_len:282, seq:49257651, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:15.745507] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257652, offset=39001984}, end_cursor:ObLogCursor{file_id=197, log_id=49257653, offset=39002266}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2984050, len:4224, log_data_len:282, seq:49257652, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:15.745757] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257653, offset=39002266}, end_cursor:ObLogCursor{file_id=197, log_id=49257654, offset=39002548}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a2c050, len:3942, log_data_len:282, seq:49257653, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:15.745818] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62649957, miss_cnt:32, cache_read_size:8050385782, hit_ratio:"9.999994892257683121e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:15.745835] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656692, miss_cnt:17, cache_read_size:8050133163, hit_ratio:"9.999997286802918550e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:15.747294] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA866-0-0] [lt=23] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:15.747315] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA866-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061835741746000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061835741746000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:15.747744] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=19] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}) [2024-10-16 14:57:15.747937] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B65-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303616331, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893509, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.748451] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:15.748476] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:15.748486] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=8] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:15.748514] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA866-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061835747989000, start_id=1729061835747989000, end_id=1729061835747989001) [2024-10-16 14:57:15.751344] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.751774] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.751786] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.751793] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047017422850},{"BIGINT":1729058400},{"VARCHAR":"BMå€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.751844] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.751912] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA866-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061835751179000, start_id=1729061835751179000, end_id=1729061835751179001) [2024-10-16 14:57:15.751975] INFO [LIB] stat (utility.h:1165) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA866-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:15.751986] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA866-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=9929, avg_group_log_size=522) [2024-10-16 14:57:15.751994] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA866-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9929) [2024-10-16 14:57:15.752000] INFO [LIB] stat (utility.h:1165) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA866-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:15.752166] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2133}, to_seq_no={branch:0, seq:1071}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284953} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.752296] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=626, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:15.752306] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9929) [2024-10-16 14:57:15.752315] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=203, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:15.752345] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9929) [2024-10-16 14:57:15.752352] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:15.752365] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601284, offset=9596418}, end_cursor:ObLogCursor{file_id=192, log_id=47601285, offset=9596702}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b00050, len:510, log_data_len:284, seq:47601284, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:15.752484] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5663-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:15.752489] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5663-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=1814, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:15.752493] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5663-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=1007, avg_log_submit_to_slide_time=1782) [2024-10-16 14:57:15.752558] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:15.753723] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B67-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303622116, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893510, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.755248] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48A-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061835754310000, start_id=1729061835754310000, end_id=1729061835754310001) [2024-10-16 14:57:15.755757] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A4D-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:57:15.756593] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601285, offset=9596702}, end_cursor:ObLogCursor{file_id=192, log_id=47601286, offset=9596976}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b3c050, len:4322, log_data_len:274, seq:47601285, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:15.756817] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:15.756834] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:15.757031] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=77668) [2024-10-16 14:57:15.757051] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=76166) [2024-10-16 14:57:15.757776] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B68-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303626170, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893511, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.759340] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=15] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:15.760274] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B6A-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303628669, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893512, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.760387] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.760891] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=27, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:15.760943] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601286, offset=9596976}, end_cursor:ObLogCursor{file_id=192, log_id=47601287, offset=9597250}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2960050, len:4048, log_data_len:274, seq:47601286, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:15.761195] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601287, offset=9597250}, end_cursor:ObLogCursor{file_id=192, log_id=47601288, offset=9597524}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2974050, len:3774, log_data_len:274, seq:47601287, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:15.761283] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560333, miss_cnt:1, cache_read_size:152589319452, hit_ratio:"9.999999928346402500e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:15.761543] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137597934, miss_cnt:9, cache_read_size:149776588270, hit_ratio:"9.999999345920455029e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:15.764585] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B6C-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303632980, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893513, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.767034] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.767051] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.767057] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047189389312},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.767139] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=51][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.767354] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3178}, to_seq_no={branch:0, seq:2119}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284956} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.767975] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=26] gc stale ls task succ [2024-10-16 14:57:15.769921] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B251ED09E-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303638315, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893514, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.772211] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061835771319000, start_id=1729061835771319000, end_id=1729061835771319001) [2024-10-16 14:57:15.775285] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061835773443000, start_id=1729061835773443000, end_id=1729061835773443001) [2024-10-16 14:57:15.779063] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=18] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:15.783117] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B6F-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303651511, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893515, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.784347] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.784363] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.784369] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047403298816},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.784433] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.784697] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1155}, to_seq_no={branch:0, seq:1150}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284960} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.788024] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B71-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303656417, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893516, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.788130] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:15.788438] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.788626] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=3] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.788635] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.788649] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.788660] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.788951] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.788956] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.789632] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.789662] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.790126] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.790146] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:15.793749] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B73-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303662143, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893517, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.795797] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B74-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303664192, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893518, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.805899] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A7E-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=36, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:15.805914] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3A7E-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=36, stat_interval=1000000, avg (count/cost)=7274, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:15.807054] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B76-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303675447, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893519, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.808720] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B78-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303677115, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893520, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.809592] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.809615] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.809622] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047663345665},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.809709] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.811815] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6390}, to_seq_no={branch:0, seq:3217}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284966} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.813660] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=361, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:15.813674] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=204, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:15.815204] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1003}, {id:1002}, {id:1}]) [2024-10-16 14:57:15.816319] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF224E-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303684712, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893521, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.817001] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.817017] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.817024] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047730454528},{"BIGINT":1729058400},{"VARCHAR":"?ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.817077] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.817403] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1853}, to_seq_no={branch:0, seq:1848}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284967} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.818109] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B7B-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303686504, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893522, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.818717] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061835818566000, start_id=1729061835818566000, end_id=1729061835818566001) [2024-10-16 14:57:15.822118] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B7D-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303690512, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893523, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.826063] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B7F-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303694458, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893524, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.826905] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:15.826920] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:15.826932] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52146508, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:15.827484] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B81-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303695879, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893525, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.829903] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B83-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303698299, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893526, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.831968] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=18] LogLoopThread round_cost_time(us)(round_cost_time=8) [2024-10-16 14:57:15.835177] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B85-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303703571, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893527, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.843470] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=33, stat_interval=1000000, avg (count/cost)=283, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:15.843492] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=33, stat_interval=1000000, avg (count/cost)=146, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:15.843539] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=33, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:15.845340] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.845355] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.845362] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041212506112},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.845413] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.846694] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF224F-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303715087, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893528, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.847096] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4449}, to_seq_no={branch:0, seq:1071}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284971} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.849128] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2817, mem_used=6239232) [2024-10-16 14:57:15.850836] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B8D-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303719229, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893529, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.851347] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.851371] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.851378] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041300586496},{"BIGINT":1729058400},{"VARCHAR":"?Ã333333?¼(õÂ\)?¹™™™™™š?¾¸Që…¸?· =p£× ?· =p£× ?¹™™™™™š?· =p£× ?¼(õÂ\)?· =p£× ?¼(õÂ\)?¼(õÂ\)?¹™™™™™š?¹™™™™™š?¾¸Që…¸?¼(õÂ\)?¼(õÂ\)?Ã333333?À£× =p¤?· =p£× ?· =p£× ?´záG®{?¹™™™™™š?¼(õÂ\)?äzáG®{?´záG®{?Ã333333?¼(õÂ\)?¼(õÂ\)?ÈQë…¸R?À£× =p¤?¼(õÂ\)?Ã333333?¼(õÂ\)?¼(õÂ\)?À£× =p¤?ú=p£× =?À£× =p¤?õ™™™™™š?¼(õÂ\)?Ñë…¸Qì?· =p£× ?Ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.851439] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.851509] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.851726] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2110}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284972} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.854171] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061835853229000, start_id=1729061835853229000, end_id=1729061835853229001) [2024-10-16 14:57:15.854231] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=35, avg_log_batch_cnt=1, total_group_log_size=87289, avg_group_log_size=2493) [2024-10-16 14:57:15.854240] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=85787) [2024-10-16 14:57:15.854552] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B8F-0-0] [lt=30][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303722945, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893530, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.855542] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.855541] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A4F-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:15.855554] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.855561] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041338335232},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.855622] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.855848] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1077}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284973} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.857557] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=14] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:15.860072] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.860085] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.860092] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041476747264},{"BIGINT":1729058400},{"VARCHAR":"?ã× =p£×?âáG®zá?â\(õÂ?ç\(õÂ\?á™™™™™š?áë…¸Qì?â=p£× =?áG®záH?äÌÌÌÌÌÍ?áë…¸Qì?äÌÌÌÌÌÍ?ä(õÂ\)?â\(õÂ?ã333333?â\(õÂ?áG®záH?á™™™™™š?á™™™™™š?åp£× =q?áG®záH?â=p£× =?à", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.860172] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.860210] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B91-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303728603, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893531, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.860442] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.860479] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1124}, to_seq_no={branch:0, seq:1119}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284975} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.862106] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B93-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303730500, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893532, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.868242] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B95-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303736636, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893533, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.869389] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.869406] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.869413] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041652908032},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.869481] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=35][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.869678] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284978} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.871760] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061835871402000, start_id=1729061835871402000, end_id=1729061835871402001) [2024-10-16 14:57:15.874326] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B97-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303742719, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893534, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.876129] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B99-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303744523, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893535, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.876321] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061835876319, bytes: 0, bw: 0.000000 MB/s, add_ts: 1000172, add_bytes: 0 [2024-10-16 14:57:15.877867] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B9B-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303746250, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893536, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.878482] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.878496] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.878503] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041740988416},{"BIGINT":1729058400},{"VARCHAR":"@êYáG®{@êYéG®z@êYñG®{@êYøõÂ\@êZ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.878561] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.878856] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2189}, to_seq_no={branch:0, seq:2184}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284980} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.880005] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061835878990000, start_id=1729061835878990000, end_id=1729061835878990001) [2024-10-16 14:57:15.880884] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061835679300000, v:0}, new_max_version={val:1729061835679300000, v:0}, last_min_version={val:1729061835176500001, v:0}, last_max_version={val:1729061835176500001, v:0}, total_time_us=1671, delta_us=201583) [2024-10-16 14:57:15.880901] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061835679300000, v:0}, min_version={val:1729061835679300000, v:0}, max_version={val:1729061835679300000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:15.882752] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:15.883806] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.883820] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.883827] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041803902976},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.883891] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=35][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.884083] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1051}, to_seq_no={branch:0, seq:1046}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284981} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.884148] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.98, rt: 223.25; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 16025.60, iops: 238.80, rt: 382.11; ) [2024-10-16 14:57:15.884170] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:15.884211] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.98, rt: 545.50; ) [2024-10-16 14:57:15.884220] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:15.884231] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:15.885347] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:15.885525] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D42-0-0] [lt=13] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061835735492000, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833508923001, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850098662000, latest_log_ts:{val:1729061833508923001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:15.885670] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D42-0-0] [lt=100] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061835818566000, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833508923001, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:15.885735] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D42-0-0] [lt=32] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061835868225000, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833508923001, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:15.885799] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D42-0-0] [lt=32] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061835874670000, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833508923001, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:15.885829] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D42-0-0] [lt=28] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:15.886362] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B9D-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303754756, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893537, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.889058] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.889072] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.889079] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041866817536},{"BIGINT":1729058400},{"VARCHAR":"@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸@1¸Që…¸¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.889132] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.889362] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2048}, to_seq_no={branch:0, seq:2043}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284982} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.892532] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0B9F-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303760925, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893538, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.894727] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1171226624, global_cache_size=1600110592, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:15.895016] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash memory, (tenant_id=1002, cache_size=106119168, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:57:15.895143] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash time detail, (compute_wash_size_time=97, refresh_score_time=80, wash_time=203) [2024-10-16 14:57:15.898114] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=7] Cache wash clean map node details(ret=0, clean_node_count=45, clean_time=0, clean_start_pos=62914, clean_num=62914) [2024-10-16 14:57:15.899424] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.899439] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.899449] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042017812480},{"BIGINT":1729058400},{"VARCHAR":"APv¡O\(öAPv§Ü(õÂAPv®Ç®{APvµ$záHAPv»¦fffAPvÁäzáHAPvÈ ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.899514] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.899751] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3163}, to_seq_no={branch:0, seq:2103}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284986} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.900734] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BA1-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303769128, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893539, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.902829] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.902852] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.902858] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042055561216},{"BIGINT":1729058400},{"VARCHAR":"A(¯6Rñ©üA(¯:k…¸A(¯E6È´9A(¯IÃn˜A(¯OÍÒñªA(¯S Ä›¦A(¯W»dZA(¯[¤Ý/A(¯_ò° ÅA(¯cÿ|í’A(¯l$Ý/A(¯q…¸RA(¯u‰7KÇA(¯ys¶E¢A(¯~§ïžA(¯‚ž¸QìA(¯†„“uA(¯•’n—A(¯™Òn—A(¯Ÿ¯²-A(¯¥)xÔþA(¯©A(¯É¦éxÕA(¯ÍøÔýôA(¯Ò¼j~úA(¯â!ÊÀƒA(¯ë÷KƨA(¯ï³¶E¢A(¯ôp ÄœA(¯øMÒñªA(¯ü§l‹DA(°", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.902939] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=52][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.904242] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2121}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284987} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.905463] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BA3-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303773857, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893540, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.906973] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:15.911140] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BA5-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303779535, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893541, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.919203] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:15.919735] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=78, stat_interval=1000000, avg (count/cost)=96, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:15.919751] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=78, stat_interval=1000000, avg (count/cost)=124, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:15.920470] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.920485] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.920492] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042168807424},{"BIGINT":1729058400},{"VARCHAR":"?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.920557] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.920788] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1084}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284989} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.923977] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BA7-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303792370, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893542, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.925884] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BA9-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303794279, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893543, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.927166] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061835925135000, start_id=1729061835925135000, end_id=1729061835925135001) [2024-10-16 14:57:15.928975] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BAB-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303797370, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893544, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.930548] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BAD-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303798944, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893545, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.930868] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061835929377000, start_id=1729061835929377000, end_id=1729061835929377001) [2024-10-16 14:57:15.931801] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061835931053000, start_id=1729061835931053000, end_id=1729061835931053001) [2024-10-16 14:57:15.932737] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.932749] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.932756] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042307219456},{"BIGINT":1729058400},{"VARCHAR":"B²®üÚ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.932808] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.932872] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BAF-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303801268, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893546, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.933418] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5932}, to_seq_no={branch:0, seq:2135}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284991} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.936687] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BB1-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303805083, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893547, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.936726] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=725, mem_used=2079744) [2024-10-16 14:57:15.936991] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061835936806000, start_id=1729061835936806000, end_id=1729061835936806001) [2024-10-16 14:57:15.938405] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BB3-0-0] [lt=9][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303806800, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893548, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.940749] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=66372665, this={dio_aligned_buf:{buf_write_offset:2705, buf_padding_size:1391, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111159383, truncate_used_ts:41410636}, log_block_size:67108864, dir_fd:177, io_fd:142}, aligned_buf_len=4096, aligned_block_offset=66371584, buf_len=1624, total_write_size_=166563, total_write_size_after_dio_=708608, ob_pwrite_used_ts=35062, count=132, each_pwrite_cost=265) [2024-10-16 14:57:15.941883] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BB5-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303810278, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893549, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.943354] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BB7-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303811750, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893550, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.944980] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BB9-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303813376, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893551, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.945781] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:15.945801] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:15.945808] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=144) [2024-10-16 14:57:15.949910] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO [ratelimit] time: 1729061835949908, bytes: 4446787781133, bw: 1.187777 MB/s, add_ts: 1001041, add_bytes: 1246771 [2024-10-16 14:57:15.951050] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.951068] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.951075] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042495963136},{"BIGINT":1729058400},{"VARCHAR":"?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/?•$Ý/¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:15.951139] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:15.951396] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1032}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392284995} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:15.951538] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.956021] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A51-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:15.957474] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061835956737000, start_id=1729061835956737000, end_id=1729061835956737001) [2024-10-16 14:57:15.960509] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:15.961054] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2251-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303829448, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893552, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.961655] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106956][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79CF-0-0] [lt=10] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061835952875, keepalive_ts:-1}) [2024-10-16 14:57:15.961672] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106956][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79CF-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061835952875, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:15.965970] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BBB-0-0] [lt=31][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303834365, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893553, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.967294] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BBD-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303835688, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893554, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.970975] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BBF-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303839370, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893555, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.971936] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15C4-0-0] [lt=7] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:15.974904] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061835974325000, start_id=1729061835974325000, end_id=1729061835974325001) [2024-10-16 14:57:15.975391] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061835974325000, start_id=1729061835974325000, end_id=1729061835974325001) [2024-10-16 14:57:15.978148] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BC1-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303846542, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893556, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.979858] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BC3-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303848249, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893557, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:15.983850] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:15.983867] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:15.983873] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042735038464},{"BIGINT":1729058400},{"VARCHAR":"AD‘ì‹C–AD—3333ADžÆ§ïžAD¤‡+ADªD›¥ãAD¯„“uAD´âMÒòADºE¡ÊÁAD¿Þ5?}ADÅVAD̲-ADÑÔýó¶AD×/Ÿ¿ADÜ$ÝAD⟾wADçC•ADì~ùÛ#ADóê~ùÛADù(õÂADþget_logical_del_time()=9510303865232, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893558, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.005977] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BC6-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303874371, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893559, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.006030] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=18] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:16.011580] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061836011099000, start_id=1729061836011099000, end_id=1729061836011099001) [2024-10-16 14:57:16.011612] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=17] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:16.011628] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061836011624], gts:1729061836011099000, latest_srr:[mts=1729061836011624]}) [2024-10-16 14:57:16.013490] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:16.013512] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:16.013731] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:16.013744] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:16.013752] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:16.018408] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.018423] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.018430] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043200606208},{"BIGINT":1729058400},{"VARCHAR":"A…»€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.018490] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.018715] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2119}, to_seq_no={branch:0, seq:2114}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285008} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.020360] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BCA-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303888752, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893560, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.024889] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061836024598000, start_id=1729061836024598000, end_id=1729061836024598001) [2024-10-16 14:57:16.034591] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BCC-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303902983, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893561, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.034907] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.034919] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.034926] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043288686592},{"BIGINT":1729058400},{"VARCHAR":"A¨• ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.034980] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.035199] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2698}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285011} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.040385] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BCE-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303908778, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893562, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.041750] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.041767] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.041774] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043339018240},{"BIGINT":1729058400},{"VARCHAR":"BHö€Ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.041832] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.042987] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BD0-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303911381, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893563, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.043239] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22267][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2364}, to_seq_no={branch:0, seq:1306}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285013} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.045812] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061836045430000, start_id=1729061836045430000, end_id=1729061836045430001) [2024-10-16 14:57:16.051743] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.052435] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.052450] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.052456] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043410321409},{"BIGINT":1729058400},{"VARCHAR":"@G®záH@G®záH@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.052514] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.053787] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2253-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303922180, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893564, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.053956] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3865}, to_seq_no={branch:0, seq:1864}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285015} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.054876] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:16.054894] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:16.055119] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=325, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:16.055131] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1210) [2024-10-16 14:57:16.055142] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=239, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:16.055156] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:16.055167] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:16.055391] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3AA0-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:16.055398] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3AA0-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=905, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:16.057638] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061836057173000, start_id=1729061836057173000, end_id=1729061836057173001) [2024-10-16 14:57:16.057738] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A53-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:16.057749] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A53-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30213, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:16.057753] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A53-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=479, avg_log_submit_to_slide_time=30208) [2024-10-16 14:57:16.057760] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A53-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178200, log_task={header:{begin_lsn:{lsn:14538456941}, end_lsn:{lsn:14538457063}, log_id:115178200, min_scn:{val:1729061835620865003, v:0}, max_scn:{val:1729061835620865003, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538456819}, prev_proposal_id:23, committed_end_lsn:{lsn:14538456941}, data_checksum:1715074492, accum_checksum:3198071202, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061835956016, freeze_ts:1729061835956016, submit_ts:1729061835956033, flushed_ts:1729061835956231, gen_to_freeze cost time:0, gen_to_submit cost time:17, submit_to_flush cost time:198}, fs_cb_begin_ts=1729061836057732, log_life_time=101716) [2024-10-16 14:57:16.057771] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:16.059583] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BD3-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303927977, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893565, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.061199] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BD5-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303929593, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893566, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.061245] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.062934] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BD7-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303931329, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893567, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.064447] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BD9-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303932843, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893568, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.067446] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F15-0-0] [lt=4] init point thread id with(&point=0x55e3435c2280, idx_=464, point=[thread id=106960, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=106960) [2024-10-16 14:57:16.067759] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BDB-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303936154, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893569, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.075124] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061836074330000, start_id=1729061836074330000, end_id=1729061836074330001) [2024-10-16 14:57:16.075584] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.075598] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.075617] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043590676480},{"BIGINT":1729058400},{"VARCHAR":"BUæ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.075671] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.075863] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2114}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285020} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.076809] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BDD-0-0] [lt=8][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303945203, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893570, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.080373] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BDF-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303948767, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893571, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.080998] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=6] ====== traversal_flush timer task ====== [2024-10-16 14:57:16.081178] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D40-0-0] [lt=10] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061836011099000, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833619065004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850721265000, latest_log_ts:{val:1729061834119131001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:16.081330] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061836080698000, start_id=1729061836080698000, end_id=1729061836080698001) [2024-10-16 14:57:16.081352] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D40-0-0] [lt=91] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061836011099000, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833619065004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:16.081496] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D40-0-0] [lt=65] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061835620865003, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833619065004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:16.081646] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D40-0-0] [lt=59] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061835621990004, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833619065004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:16.081706] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D40-0-0] [lt=59] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:16.091627] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BE1-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303960020, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893572, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.095333] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BE3-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303963727, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893573, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.098433] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1175420928, global_cache_size=1593868288, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:16.098651] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] [CACHE] tenant_id= 1 cache memory info: [CACHE] tenant_id= 1 | cache_name= schema_cache | cache_size= 91552768 | cache_store_size= 89473024 | cache_map_size= 2079744 | kv_cnt= 5669 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= tablet_table_cache | cache_size= 12481536 | cache_store_size= 6242304 | cache_map_size= 6239232 | kv_cnt= 43492 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= vtable_cache | cache_size= 147733504 | cache_store_size= 145653760 | cache_map_size= 2079744 | kv_cnt= 76 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= index_block_cache | cache_size= 68662272 | cache_store_size= 62423040 | cache_map_size= 6239232 | kv_cnt= 31222 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= user_block_cache | cache_size= 222639104 | cache_store_size= 216399872 | cache_map_size= 6239232 | kv_cnt= 32894 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= user_row_cache | cache_size= 12483584 | cache_store_size= 10403840 | cache_map_size= 2079744 | kv_cnt= 2891 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= bf_cache | cache_size= 10402816 | cache_store_size= 8323072 | cache_map_size= 2079744 | kv_cnt= 193 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= fuse_row_cache | cache_size= 14564352 | cache_store_size= 12484608 | cache_map_size= 2079744 | kv_cnt= 4381 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= storage_meta_cache | cache_size= 282978304 | cache_store_size= 270499840 | cache_map_size= 12478464 | kv_cnt= 81029 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 17 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 25 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 234 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_column_stat_cache | cache_size= 16645120 | cache_store_size= 14565376 | cache_map_size= 2079744 | kv_cnt= 1027 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_ds_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 9 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= BACKUP_INDEX_CACHE | cache_size= 0 | cache_store_size= 0 | cache_map_size= 0 | kv_cnt= 0 | hold_size= 0 [2024-10-16 14:57:16.098694] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] [CACHE] tenant_id= 1001 cache memory info: [CACHE] tenant_id= 1001 | cache_name= index_block_cache | cache_size= 22887424 | cache_store_size= 20807680 | cache_map_size= 2079744 | kv_cnt= 5229 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= user_block_cache | cache_size= 187268096 | cache_store_size= 185188352 | cache_map_size= 2079744 | kv_cnt= 10159 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= user_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1229 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= bf_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 28 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= fuse_row_cache | cache_size= 10402816 | cache_store_size= 8323072 | cache_map_size= 2079744 | kv_cnt= 2520 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= storage_meta_cache | cache_size= 41614336 | cache_store_size= 39534592 | cache_map_size= 2079744 | kv_cnt= 10145 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 39 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 6 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 133 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= opt_column_stat_cache | cache_size= 8322048 | cache_store_size= 6242304 | cache_map_size= 2079744 | kv_cnt= 362 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [2024-10-16 14:57:16.098738] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] [CACHE] tenant_id= 1002 cache memory info: [CACHE] tenant_id= 1002 | cache_name= index_block_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 433 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= user_block_cache | cache_size= 49937408 | cache_store_size= 47857664 | cache_map_size= 2079744 | kv_cnt= 1128 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= user_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1406 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= bf_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 19 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= fuse_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 2419 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= storage_meta_cache | cache_size= 16645120 | cache_store_size= 14565376 | cache_map_size= 2079744 | kv_cnt= 2728 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= tmp_block_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 3691 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 24 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1078 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= opt_column_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 624 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [2024-10-16 14:57:16.098774] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=7] [CACHE] tenant_id= 1003 cache memory info: [CACHE] tenant_id= 1003 | cache_name= index_block_cache | cache_size= 24968192 | cache_store_size= 22888448 | cache_map_size= 2079744 | kv_cnt= 5100 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= user_block_cache | cache_size= 164379648 | cache_store_size= 162299904 | cache_map_size= 2079744 | kv_cnt= 11493 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= user_row_cache | cache_size= 8322048 | cache_store_size= 6242304 | cache_map_size= 2079744 | kv_cnt= 5065 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= bf_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 49 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= fuse_row_cache | cache_size= 8322048 | cache_store_size= 6242304 | cache_map_size= 2079744 | kv_cnt= 2220 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= storage_meta_cache | cache_size= 49937408 | cache_store_size= 47857664 | cache_map_size= 2079744 | kv_cnt= 10774 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= tmp_block_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 31 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 24 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 134 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= opt_column_stat_cache | cache_size= 12483584 | cache_store_size= 10403840 | cache_map_size= 2079744 | kv_cnt= 764 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [2024-10-16 14:57:16.098818] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=8] [CACHE] tenant_id= 1004 cache memory info: [CACHE] tenant_id= 1004 | cache_name= index_block_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 747 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= user_block_cache | cache_size= 14564352 | cache_store_size= 12484608 | cache_map_size= 2079744 | kv_cnt= 388 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= user_row_cache | cache_size= 10402816 | cache_store_size= 8323072 | cache_map_size= 2079744 | kv_cnt= 573 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= fuse_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1098 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= storage_meta_cache | cache_size= 24968192 | cache_store_size= 22888448 | cache_map_size= 2079744 | kv_cnt= 5924 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= tmp_block_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 6 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 27 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 161 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_column_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 461 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_ds_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 9 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= BACKUP_INDEX_CACHE | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 3 | hold_size= 0 [2024-10-16 14:57:16.098836] INFO print_current_status (ob_kvcache_hazard_version.cpp:328) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] [KVCACHE-HAZARD] hazard version status info: current_version: 207904782 | min_version=207904782 | total_nodes_count: 59 | [KVCACHE-HAZARD] i= 25 | acquire_version=18446744073709551615 | waiting_nodes_count= 45 | last_retire_version=207904723 | [KVCACHE-HAZARD] i= 132 | acquire_version=18446744073709551615 | waiting_nodes_count= 9 | last_retire_version=207904723 | [KVCACHE-HAZARD] i= 156 | acquire_version=18446744073709551615 | waiting_nodes_count= 1 | last_retire_version=207904723 | [KVCACHE-HAZARD] i= 732 | acquire_version=18446744073709551615 | waiting_nodes_count= 4 | last_retire_version=207904723 | [2024-10-16 14:57:16.100099] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BE5-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303968493, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893574, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.102690] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.102704] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.102711] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043892666368},{"BIGINT":1729058400},{"VARCHAR":"B¢ÿ໤", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.102773] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.104228] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BE6-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303972622, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893575, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.104228] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3175}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285028} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.107719] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21668][T1_L0_G0][T1][YB4A0A000407-00061E99793CF610-0-0] [lt=8][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303976107, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410503, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.108934] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AA-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:16.108958] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AA-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061835621990004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061835621990004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:16.109761] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:16.109778] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:16.110186] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.110196] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.110214] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043980746752},{"BIGINT":1729058400},{"VARCHAR":"B{nþ˜", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.110267] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.110528] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3525}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285029} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.111382] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] [COMPACTION DAG STATUS] (start_time=1729061826109322, end_time=1729061836111381, dag_status={}) [2024-10-16 14:57:16.111394] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.111405] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.111411] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.111415] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.111420] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.111423] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.111430] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.111434] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.111437] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.111443] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111449] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111453] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111457] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111461] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111466] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111469] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111473] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111477] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111481] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111485] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111488] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111492] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111496] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111500] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111504] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111507] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111511] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111514] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111518] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111521] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111525] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111529] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111534] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111538] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111542] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111546] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111549] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111553] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111562] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111566] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111570] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111574] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111578] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111582] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111585] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111589] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111593] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111597] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111607] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111611] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111615] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111619] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111622] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111627] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111630] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111634] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111638] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111642] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111645] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111649] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111653] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111656] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111660] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111664] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111668] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111672] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111676] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.111680] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:16.111683] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:16.111687] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:16.111689] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:16.111692] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:16.111694] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:16.111697] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:16.111699] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:16.111702] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:16.111704] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:16.111707] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:16.113548] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:16.113561] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:16.113586] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:16.113591] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:16.114316] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=291, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:16.114337] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=236, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:16.114422] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:16.114433] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1175, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:16.114437] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=13, avg_log_submit_to_flush_time=539, avg_log_submit_to_slide_time=1162) [2024-10-16 14:57:16.114450] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:16.114454] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1187, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:16.114457] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=2] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:16.114549] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AA-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061836114143000, start_id=1729061836114143000, end_id=1729061836114143001) [2024-10-16 14:57:16.114634] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AA-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9699, avg_group_log_size=570) [2024-10-16 14:57:16.114666] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AA-0-0] [lt=30] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9699) [2024-10-16 14:57:16.114809] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=363, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:16.114820] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9699) [2024-10-16 14:57:16.114830] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=637, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:16.114844] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=26] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10292) [2024-10-16 14:57:16.114855] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:16.115370] INFO [LIB] stat (utility.h:1165) [106956][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCF76-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:16.115382] INFO [LIB] stat (utility.h:1165) [106956][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCF76-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=2466, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:16.115407] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106956][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCF76-0-0] [lt=20] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=66, avg_log_gen_to_submit_time=93, avg_log_submit_to_flush_time=1065, avg_log_submit_to_slide_time=2373) [2024-10-16 14:57:16.115454] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=2] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:16.118483] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BE8-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303986877, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893576, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.120076] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BEA-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510303988472, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893577, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.121767] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B56-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061836121545000, start_id=1729061836121545000, end_id=1729061836121545001) [2024-10-16 14:57:16.121802] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B5A-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061836121545000, start_id=1729061836121545000, end_id=1729061836121545001) [2024-10-16 14:57:16.122246] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B57-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061836121545000, start_id=1729061836121545000, end_id=1729061836121545001) [2024-10-16 14:57:16.122837] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:16.123110] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.123127] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.123134] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044182073344},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.123197] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.123501] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2170}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285032} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.124643] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=17326, mem_used=31196160) [2024-10-16 14:57:16.127307] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061836126897000, start_id=1729061836126897000, end_id=1729061836126897001) [2024-10-16 14:57:16.127550] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [21543][T1_CdcSrv][T1][Y0-0000000000000000-0-0] [lt=12] finish to resize log external storage handler(current_ts=1729061836127548, tenant_max_cpu=3, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:16.131068] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061836129114000, start_id=1729061836129114000, end_id=1729061836129114001) [2024-10-16 14:57:16.131641] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1003, min_version={val:1729061835931053000, v:0}, base_version={val:1729061835731245000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061835931053000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061836131628, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061835931053000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061836066180, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061835931053000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061836092208, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:16.132534] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061836132313000, start_id=1729061836132313000, end_id=1729061836132313001) [2024-10-16 14:57:16.132937] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=126356) [2024-10-16 14:57:16.135570] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106793][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC72-0-0] [lt=9] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:16.135590] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106793][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC72-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:16.135618] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106793][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC72-0-0] [lt=26] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:16.140943] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=127790) [2024-10-16 14:57:16.142038] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=129301) [2024-10-16 14:57:16.142391] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.142412] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.142422] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192034107392},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.142490] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.142978] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1152}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285037} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.148083] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.148101] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.148109] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618574290604032},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.148183] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=38][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.148427] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285039} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.151028] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061836150818000, start_id=1729061836150818000, end_id=1729061836150818001) [2024-10-16 14:57:16.151814] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.154954] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1210, avg_group_log_size=121) [2024-10-16 14:57:16.154970] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1210) [2024-10-16 14:57:16.155462] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BEC-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304023855, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893578, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.160938] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5012-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:16.160959] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5012-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:16.161318] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.166671] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.166694] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.166704] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618574592593920},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.166796] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.167072] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3188}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285044} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.169176] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:16.169202] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=18] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:16.169216] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:16.169225] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:16.169234] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=8] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:16.169240] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=5] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16720) [2024-10-16 14:57:16.170322] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3AB4-0-0] [lt=37] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=57, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:16.170334] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3AB4-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=57, stat_interval=1000000, avg (count/cost)=1064, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:16.172671] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BD6-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:16.172691] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BD6-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:16.176089] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061836175334000, start_id=1729061836175334000, end_id=1729061836175334001) [2024-10-16 14:57:16.176395] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061836176391000, start_id=1729061836176391000, end_id=1729061836176391001) [2024-10-16 14:57:16.177843] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BEE-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304046236, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893579, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.179531] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BF0-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304047926, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893580, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.186971] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BF2-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304055361, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893581, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.188226] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:16.193469] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BF4-0-0] [lt=36][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304061863, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893582, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.196912] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21659][T1_L0_G0][T1][YB4A0A000408-00061E99858D2FD0-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304065306, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410504, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.199907] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.199929] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.199935] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044899299328},{"BIGINT":1729058400},{"VARCHAR":"AWÏ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.200018] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.200374] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1069}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285052} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.213893] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:16.214525] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3188}, to_seq_no={branch:0, seq:2122}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285054} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.215747] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5871-0-0] [lt=10] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:16.215843] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5871-0-0] [lt=12][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:16.218344] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:57:16.218364] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=16] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=82, cache_node_num=82) [2024-10-16 14:57:16.218373] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=8] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=82, cache_node_num=82) [2024-10-16 14:57:16.218381] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:16.220876] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=82, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}]) [2024-10-16 14:57:16.221036] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=157] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:16.222704] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BF5-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304091091, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893583, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.225370] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1156}, to_seq_no={branch:0, seq:1151}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285056} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.226543] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=20] get_max_applied_scn(scn={val:1729061836045430000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959661683}, palf_committed_end_scn_:{val:1729061836150818000, v:0}, last_check_scn_:{val:1729061836150818000, v:0}, max_applied_cb_scn_:{val:1729061836045430000, v:0}}) [2024-10-16 14:57:16.226567] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:16.227161] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=51] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:57:16.227180] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info(lsn={lsn:213056941173}, scn={val:1729061836223243001, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061836224135, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213056941173}, next_to_submit_scn_:{val:1729061836223243001, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:1624, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1624, log_storage_:{IteratorStorage:{start_lsn:{lsn:213056939549}, end_lsn:{lsn:213056941173}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061836223243000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:1536, scn_:{val:1729061836223243000, v:0}, data_checksum:2258789020, flag:1}}, init_mode_version:0, accumulate_checksum:4217668385, curr_entry_is_padding:0, padding_entry_size:1568, padding_entry_scn:{val:1729061836223243000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560371, miss_cnt:1, cache_read_size:152589381155, hit_ratio:"9.999999928346422484e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:16.227440] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:57:16.228310] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061836226516000, start_id=1729061836226516000, end_id=1729061836226516001) [2024-10-16 14:57:16.230343] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:16.231979] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BF7-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304100372, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893584, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.232300] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3307}, to_seq_no={branch:0, seq:1077}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285057} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.233865] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2255-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304102258, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893585, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.235674] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BFA-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304104069, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893586, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.237450] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BFC-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304105834, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893587, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.247345] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD02-0-0] [lt=5][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:16.252462] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.254221] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2257-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304122615, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893588, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.255644] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5760}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285061} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.256190] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BFD-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304124584, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893589, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.258539] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0BFF-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304126933, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893590, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.258595] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061836258043000, start_id=1729061836258043000, end_id=1729061836258043001) [2024-10-16 14:57:16.260931] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1071}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285062} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.261391] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.264219] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061836263309000, start_id=1729061836263309000, end_id=1729061836263309001) [2024-10-16 14:57:16.267001] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3657, avg_group_log_size=332) [2024-10-16 14:57:16.267232] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:16.269221] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106635][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45AE0-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=40, avg_log_submit_to_flush_time=665, avg_log_submit_to_slide_time=1286) [2024-10-16 14:57:16.270056] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3318}, to_seq_no={branch:0, seq:2106}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285063} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.270469] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C01-0-0] [lt=31][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304138862, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893591, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.271878] INFO [COMMON] generate_mod_stat_task (memory_dump.cpp:218) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] task info(*task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:16.271893] INFO [COMMON] handle (memory_dump.cpp:550) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=14] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:16.272746] INFO [COMMON] update_check_range (ob_sql_mem_leak_checker.cpp:62) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] update_check_range(min_check_version=652896, max_check_version=652897, global_version=652898) [2024-10-16 14:57:16.274989] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] [WRS] update tenant weak read server version(tenant_id=1001, server_version={version:{val:1729061836175334000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061836274961}, version_delta=99654) [2024-10-16 14:57:16.276943] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB90-0-0] [lt=7] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12287, add_timestamp:1729061836276923}) [2024-10-16 14:57:16.281764] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=23] LogLoopThread round_cost_time(us)(round_cost_time=12) [2024-10-16 14:57:16.282585] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3ACEB8-0-0] [lt=12] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.704990520477294922e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=852, Throttle Trigger(MB)=511, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.368411119626076555e+01) [2024-10-16 14:57:16.282786] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061836275385000, start_id=1729061836275385000, end_id=1729061836275385001) [2024-10-16 14:57:16.283071] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E05-0-0] [lt=8] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12287, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:16.285302] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9B3769AB90-0-0] [lt=15][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:16.291282] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C03-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304159675, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893592, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.294813] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061836293815000, start_id=1729061836293815000, end_id=1729061836293815001) [2024-10-16 14:57:16.294946] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E05-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12287, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061836294932, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061836294932, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061836294932, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:16.294974] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E05-0-0] [lt=27] success to process renew task(task={tenant_id:1, table_id:12287, add_timestamp:1729061836276923}, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061836294932, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061836294932, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061836294932, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:16.297406] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C05-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304165801, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893593, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.298071] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=15] ls blacklist refresh finish(cost_time=21859) [2024-10-16 14:57:16.299110] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=6] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1169129472, global_cache_size=1598029824, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:16.299379] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash memory, (tenant_id=1002, cache_size=104038400, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:57:16.299506] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Wash time detail, (compute_wash_size_time=95, refresh_score_time=70, wash_time=192) [2024-10-16 14:57:16.302200] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=5] Cache wash clean map node details(ret=0, clean_node_count=27, clean_time=0, clean_start_pos=125828, clean_num=62914) [2024-10-16 14:57:16.306175] INFO handle (memory_dump.cpp:677) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 215 258 0 1 2 117 1 0 1 4 1 1 0 1 5 38 4 0 1 6 1 1 0 1 7 1 2 0 1 8 65 1 0 1 12 18 3 0 1 13 6 1 0 1 14 1 2 0 1 15 1 1 0 1 18 26 3 0 1 20 2 2 0 1 22 1 1 0 500 0 140 257 0 500 7 7 4 0 500 8 50 1 0 500 9 2 1 0 500 10 10 2 0 500 16 3 3 0 500 17 10 7 0 500 22 111 63 0 500 23 61 11 0 508 0 4 13 0 508 8 5 1 0 509 0 3 8 0 509 8 5 1 0 1001 0 128 232 0 1001 2 21 1 0 1001 4 1 1 0 1001 5 23 4 0 1001 8 44 1 0 1001 12 10 3 0 1001 13 3 1 0 1001 14 1 2 0 1001 18 14 3 0 1001 20 1 2 0 1002 0 171 258 0 1002 2 19 1 0 1002 4 2 1 0 1002 5 24 4 0 1002 8 49 1 0 1002 12 19 3 0 1002 13 7 1 0 1002 14 1 2 0 1002 16 2 2 0 1002 18 14 3 0 1002 20 7 2 0 1003 0 140 236 0 1003 2 29 1 0 1003 4 1 1 0 1003 5 23 4 0 1003 8 45 1 0 1003 12 10 3 0 1003 13 3 1 0 1003 14 1 2 0 1003 18 13 3 0 1003 20 1 2 0 1004 0 289 256 0 1004 2 19 1 0 1004 4 2 1 0 1004 5 34 4 0 1004 8 51 1 0 1004 12 19 3 0 1004 13 10 1 0 1004 14 2 2 0 1004 15 2 1 0 1004 18 25 3 0 1004 20 11 2 0 cost_time: 33462 [2024-10-16 14:57:16.306202] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=11] [MEMORY] tenant: 1, limit: 2,147,483,648 hold: 2,005,372,928 rpc_hold: 0 cache_hold: 861,929,472 cache_used: 861,929,472 cache_item_count: 411 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 562,511,872 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MEMSTORE_CTX_ID hold_bytes= 245,366,784 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TRANS_CTX_MGR_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PLAN_CACHE_CTX_ID hold_bytes= 79,691,776 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 136,314,880 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= META_OBJ_CTX_ID hold_bytes= 37,748,736 limit= 429,496,720 [MEMORY] ctx_id= TX_CALLBACK_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOB_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PS_CACHE_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TX_DATA_TABLE hold_bytes= 54,546,432 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MDS_DATA_ID hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:16.306421] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= DEFAULT_CTX_ID hold= 562,511,872 used= 449,784,000 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 32 [MEMORY] hold= 54,099,968 used= 54,073,344 count= 26 avg_used= 2,079,744 block_cnt= 26 chunk_cnt= 26 mod=CACHE_MAP_NODE [MEMORY] hold= 41,963,520 used= 41,943,040 count= 1 avg_used= 41,943,040 block_cnt= 1 chunk_cnt= 1 mod=LogGroupBuffer [MEMORY] hold= 32,030,720 used= 32,007,104 count= 17 avg_used= 1,882,770 block_cnt= 17 chunk_cnt= 17 mod=MysqlRequesReco [MEMORY] hold= 30,965,760 used= 27,525,120 count= 420 avg_used= 65,536 block_cnt= 420 chunk_cnt= 50 mod=MinorMergeMgr [MEMORY] hold= 23,511,040 used= 23,433,792 count= 8 avg_used= 2,929,224 block_cnt= 8 chunk_cnt= 7 mod=IoControl [MEMORY] hold= 22,880,256 used= 21,568,040 count= 161 avg_used= 133,962 block_cnt= 161 chunk_cnt= 38 mod=SqlPlanMon [MEMORY] hold= 18,874,368 used= 16,967,424 count= 256 avg_used= 66,279 block_cnt= 256 chunk_cnt= 63 mod=SqlDtlBuf [MEMORY] hold= 17,768,448 used= 17,012,672 count= 241 avg_used= 70,592 block_cnt= 241 chunk_cnt= 30 mod=[T]ObSessionDIB [MEMORY] hold= 16,797,696 used= 16,777,336 count= 1 avg_used= 16,777,336 block_cnt= 1 chunk_cnt= 1 mod=MrgMemPoolChk [MEMORY] hold= 15,034,848 used= 14,886,881 count= 98 avg_used= 151,906 block_cnt= 31 chunk_cnt= 7 mod=OmtTenant [MEMORY] hold= 11,845,632 used= 9,904,128 count= 237 avg_used= 41,789 block_cnt= 237 chunk_cnt= 61 mod=[T]char [MEMORY] hold= 10,911,744 used= 9,699,328 count= 148 avg_used= 65,536 block_cnt= 148 chunk_cnt= 38 mod=MajorMergeMgr [MEMORY] hold= 8,777,728 used= 8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv [MEMORY] hold= 8,409,088 used= 8,388,640 count= 1 avg_used= 8,388,640 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 6,094,848 used= 6,033,408 count= 8 avg_used= 754,176 block_cnt= 8 chunk_cnt= 7 mod=ResourceGroup [MEMORY] hold= 4,964,352 used= 4,959,488 count= 3 avg_used= 1,653,162 block_cnt= 3 chunk_cnt= 3 mod=SqlFltSpanRec [MEMORY] hold= 4,943,872 used= 4,915,456 count= 2 avg_used= 2,457,728 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe [MEMORY] hold= 4,235,264 used= 4,194,304 count= 2 avg_used= 2,097,152 block_cnt= 2 chunk_cnt= 2 mod=SHARE_BLOCK [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 12 mod=CkptDgnMemCU [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMem [MEMORY] hold= 3,772,416 used= 3,751,936 count= 1 avg_used= 3,751,936 block_cnt= 1 chunk_cnt= 1 mod=DiskIteratorSto [MEMORY] hold= 3,723,264 used= 3,698,688 count= 2 avg_used= 1,849,344 block_cnt= 2 chunk_cnt= 2 mod=LogDIOAligned [MEMORY] hold= 3,162,112 used= 3,145,808 count= 2 avg_used= 1,572,904 block_cnt= 2 chunk_cnt= 2 mod=TenantDLMgr [MEMORY] hold= 3,145,728 used= 2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 5 mod=SqlDtlQueue [MEMORY] hold= 2,828,928 used= 2,708,352 count= 604 avg_used= 4,484 block_cnt= 576 chunk_cnt= 72 mod=WorkerMap [MEMORY] hold= 2,777,088 used= 2,760,192 count= 1 avg_used= 2,760,192 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo [MEMORY] hold= 2,415,648 used= 2,387,720 count= 104 avg_used= 22,958 block_cnt= 104 chunk_cnt= 4 mod=TabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=FastFrezCkr [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap [MEMORY] hold= 2,375,680 used= 2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta [MEMORY] hold= 2,248,704 used= 2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb [MEMORY] hold= 2,198,864 used= 416,700 count= 25,277 avg_used= 16 block_cnt= 270 chunk_cnt= 2 mod=Number [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=LogReplayTask [MEMORY] hold= 1,843,200 used= 1,770,696 count= 9 avg_used= 196,744 block_cnt= 9 chunk_cnt= 2 mod=DagMap [MEMORY] hold= 1,777,664 used= 1,770,696 count= 1 avg_used= 1,770,696 block_cnt= 1 chunk_cnt= 1 mod=TableProc [MEMORY] hold= 1,548,288 used= 1,507,118 count= 5 avg_used= 301,423 block_cnt= 5 chunk_cnt= 1 mod=LDIOSetup [MEMORY] hold= 1,376,256 used= 1,322,752 count= 8 avg_used= 165,344 block_cnt= 8 chunk_cnt= 7 mod=SQLSessionInfo [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=XATimeWheel [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=TransTimeWheel [MEMORY] hold= 1,261,568 used= 681,912 count= 77 avg_used= 8,856 block_cnt= 77 chunk_cnt= 31 mod=[T]ObDSActionAr [MEMORY] hold= 1,196,032 used= 1,182,720 count= 2 avg_used= 591,360 block_cnt= 2 chunk_cnt= 2 mod=ServerObjecPool [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=RewriteRuleMap [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPlanCac [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsInfo [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsCache [MEMORY] hold= 1,024,000 used= 1,015,808 count= 1 avg_used= 1,015,808 block_cnt= 1 chunk_cnt= 1 mod=CommitVersions [MEMORY] hold= 983,488 used= 960,256 count= 121 avg_used= 7,936 block_cnt= 121 chunk_cnt= 9 mod=TltMemtablMgr [MEMORY] hold= 958,464 used= 858,728 count= 13 avg_used= 66,056 block_cnt= 13 chunk_cnt= 12 mod=CommSysVarFac [MEMORY] hold= 958,464 used= 950,272 count= 1 avg_used= 950,272 block_cnt= 1 chunk_cnt= 1 mod=IOWorkerLQ [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=TransService [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=ArcSenderQueue [MEMORY] hold= 819,200 used= 402,400 count= 50 avg_used= 8,048 block_cnt= 50 chunk_cnt= 23 mod=[T]ObPerfEventR [MEMORY] hold= 798,400 used= 779,200 count= 100 avg_used= 7,792 block_cnt= 100 chunk_cnt= 20 mod=CkptDgnMemNode [MEMORY] hold= 798,112 used= 789,624 count= 3 avg_used= 263,208 block_cnt= 2 chunk_cnt= 2 mod=PartitTableTask [MEMORY] hold= 786,688 used= 524,352 count= 33 avg_used= 15,889 block_cnt= 33 chunk_cnt= 4 mod=LogAlloc [MEMORY] hold= 765,440 used= 730,112 count= 184 avg_used= 3,968 block_cnt= 184 chunk_cnt= 44 mod=CkpDgn [MEMORY] hold= 663,552 used= 655,360 count= 1 avg_used= 655,360 block_cnt= 1 chunk_cnt= 1 mod=FetchLog [MEMORY] hold= 663,552 used= 659,200 count= 1 avg_used= 659,200 block_cnt= 1 chunk_cnt= 1 mod=MulLevelQueue [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MdsT [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=FrzTrigger [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DupTbLease [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DetectorTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MultiVersionGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTR [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=OBJLockGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTF [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=ElectTimer [MEMORY] hold= 598,016 used= 590,232 count= 1 avg_used= 590,232 block_cnt= 1 chunk_cnt= 1 mod=DagNetIdMap [MEMORY] hold= 458,752 used= 440,928 count= 3 avg_used= 146,976 block_cnt= 3 chunk_cnt= 3 mod=bf_queue [MEMORY] hold= 426,240 used= 408,960 count= 90 avg_used= 4,544 block_cnt= 90 chunk_cnt= 32 mod=[T]ObTraceEvent [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ApplySrv [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ReplaySrv [MEMORY] hold= 406,400 used= 396,800 count= 50 avg_used= 7,936 block_cnt= 50 chunk_cnt= 29 mod=LinearHashMapNo [MEMORY] hold= 399,200 used= 389,600 count= 50 avg_used= 7,792 block_cnt= 50 chunk_cnt= 32 mod=CkptDgnCUNode [MEMORY] hold= 385,024 used= 377,120 count= 1 avg_used= 377,120 block_cnt= 1 chunk_cnt= 1 mod=ClogGe [MEMORY] hold= 338,112 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 44 chunk_cnt= 2 mod=LSMap [MEMORY] hold= 311,296 used= 262,144 count= 256 avg_used= 1,024 block_cnt= 44 chunk_cnt= 1 mod=TabletToLS [MEMORY] hold= 311,104 used= 303,904 count= 6 avg_used= 50,650 block_cnt= 6 chunk_cnt= 2 mod=PoolFreeList [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=DmlStatHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompactMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompStMap [MEMORY] hold= 294,912 used= 261,632 count= 4 avg_used= 65,408 block_cnt= 4 chunk_cnt= 3 mod=MemTblObj [MEMORY] hold= 268,224 used= 261,888 count= 33 avg_used= 7,936 block_cnt= 33 chunk_cnt= 18 mod=CtxBlockPl [MEMORY] hold= 260,224 used= 253,952 count= 31 avg_used= 8,192 block_cnt= 31 chunk_cnt= 22 mod=ParamCharsType [MEMORY] hold= 212,992 used= 212,160 count= 1 avg_used= 212,160 block_cnt= 1 chunk_cnt= 1 mod=MetaMemMgr [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 19 mod=QueryAllocator [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 21 mod=SqlSession [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=TenDLBucket [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=ResourMapLock [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=T3MBucket [MEMORY] hold= 204,800 used= 196,744 count= 1 avg_used= 196,744 block_cnt= 1 chunk_cnt= 1 mod=DagNetMap [MEMORY] hold= 163,776 used= 156,741 count= 2 avg_used= 78,370 block_cnt= 2 chunk_cnt= 2 mod=CsSstableReader [MEMORY] hold= 147,456 used= 139,264 count= 1 avg_used= 139,264 block_cnt= 1 chunk_cnt= 1 mod=RFLTaskQueue [MEMORY] hold= 147,456 used= 138,240 count= 2 avg_used= 69,120 block_cnt= 2 chunk_cnt= 1 mod=LDBlockBitMap [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 11 mod=RawPl [MEMORY] hold= 129,104 used= 33,296 count= 473 avg_used= 70 block_cnt= 177 chunk_cnt= 34 mod=Coro [MEMORY] hold= 128,000 used= 124,928 count= 16 avg_used= 7,808 block_cnt= 16 chunk_cnt= 2 mod=TabletIDSetNode [MEMORY] hold= 122,880 used= 110,608 count= 2 avg_used= 55,304 block_cnt= 2 chunk_cnt= 1 mod=TabletIDSetBkt [MEMORY] hold= 114,688 used= 106,496 count= 1 avg_used= 106,496 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlMgr [MEMORY] hold= 114,688 used= 111,096 count= 1 avg_used= 111,096 block_cnt= 1 chunk_cnt= 1 mod=AsyncFreezeTabl [MEMORY] hold= 106,496 used= 98,432 count= 1 avg_used= 98,432 block_cnt= 1 chunk_cnt= 1 mod=tenant_mf_mgr [MEMORY] hold= 106,496 used= 98,528 count= 1 avg_used= 98,528 block_cnt= 1 chunk_cnt= 1 mod=LSSvr [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=PlanVaIdx [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=SqlSessiVarMap [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=DictPl [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=RlePl [MEMORY] hold= 81,920 used= 73,896 count= 1 avg_used= 73,896 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairCache [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 10 mod=ConstPl [MEMORY] hold= 77,824 used= 65,536 count= 64 avg_used= 1,024 block_cnt= 12 chunk_cnt= 3 mod=TxCtxMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=DEVICE_MANAGER [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushLog [MEMORY] hold= 73,728 used= 66,176 count= 1 avg_used= 66,176 block_cnt= 1 chunk_cnt= 1 mod=Rpc [MEMORY] hold= 73,728 used= 65,536 count= 1 avg_used= 65,536 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LockMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxCtxMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxDataMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=StorMetaCacheIO [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushMeta [MEMORY] hold= 73,728 used= 72,736 count= 1 avg_used= 72,736 block_cnt= 1 chunk_cnt= 1 mod=LogSharedQueueT [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 9 mod=IntDiffPl [MEMORY] hold= 72,240 used= 1,024 count= 896 avg_used= 1 block_cnt= 10 chunk_cnt= 1 mod=CharsetUtil [MEMORY] hold= 65,216 used= 60,712 count= 4 avg_used= 15,178 block_cnt= 4 chunk_cnt= 4 mod=HB_SERVICE [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 8 mod=MdsTableMgr [MEMORY] hold= 64,000 used= 62,464 count= 8 avg_used= 7,808 block_cnt= 8 chunk_cnt= 8 mod=HashNodeLCSta [MEMORY] hold= 58,400 used= 55,296 count= 16 avg_used= 3,456 block_cnt= 11 chunk_cnt= 8 mod=CACHE_INST [MEMORY] hold= 57,344 used= 50,496 count= 1 avg_used= 50,496 block_cnt= 1 chunk_cnt= 1 mod=palf_env [MEMORY] hold= 54,240 used= 5,784 count= 241 avg_used= 24 block_cnt= 147 chunk_cnt= 28 mod=[T]MemoryContex [MEMORY] hold= 48,960 used= 44,840 count= 2 avg_used= 22,420 block_cnt= 2 chunk_cnt= 2 mod=DynamicFactor [MEMORY] hold= 48,880 used= 44,760 count= 2 avg_used= 22,380 block_cnt= 2 chunk_cnt= 2 mod=Autoincrement [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 5 mod=ColEqualPl [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 4 mod=HexStrPl [MEMORY] hold= 46,656 used= 36,288 count= 54 avg_used= 672 block_cnt= 52 chunk_cnt= 24 mod=[T]ObWarningBuf [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DagWarnHisBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoBkt [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 1 mod=ObDMMDL [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=StrDiffPl [MEMORY] hold= 33,536 used= 22,880 count= 55 avg_used= 416 block_cnt= 24 chunk_cnt= 7 mod=OMT_Worker [MEMORY] hold= 32,768 used= 17,408 count= 2 avg_used= 8,704 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTTP [MEMORY] hold= 32,768 used= 25,664 count= 1 avg_used= 25,664 block_cnt= 1 chunk_cnt= 1 mod=TSQLSessionMgr [MEMORY] hold= 32,768 used= 24,688 count= 2 avg_used= 12,344 block_cnt= 2 chunk_cnt= 2 mod=TLD_AssignedMgr [MEMORY] hold= 32,704 used= 26,079 count= 2 avg_used= 13,039 block_cnt= 2 chunk_cnt= 2 mod=CGReadInfoMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 3 mod=StrPrefixPl [MEMORY] hold= 32,000 used= 31,232 count= 4 avg_used= 7,808 block_cnt= 4 chunk_cnt= 4 mod=HashNodePlanCac [MEMORY] hold= 24,576 used= 22,592 count= 1 avg_used= 22,592 block_cnt= 1 chunk_cnt= 1 mod=IO_MGR [MEMORY] hold= 24,576 used= 16,384 count= 1 avg_used= 16,384 block_cnt= 1 chunk_cnt= 1 mod=SlogWriteBuffer [MEMORY] hold= 24,368 used= 20,136 count= 2 avg_used= 10,068 block_cnt= 2 chunk_cnt= 2 mod=TabletStats [MEMORY] hold= 24,336 used= 20,104 count= 2 avg_used= 10,052 block_cnt= 2 chunk_cnt= 2 mod=TLD_ResourceMgr [MEMORY] hold= 21,152 used= 16,896 count= 22 avg_used= 768 block_cnt= 15 chunk_cnt= 12 mod=TGTimer [MEMORY] hold= 17,056 used= 8,448 count= 3 avg_used= 2,816 block_cnt= 2 chunk_cnt= 2 mod=BaseLogWriter [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=SlogNopLog [MEMORY] hold= 16,384 used= 13,824 count= 1 avg_used= 13,824 block_cnt= 1 chunk_cnt= 1 mod=LogReplayStatus [MEMORY] hold= 16,384 used= 11,584 count= 1 avg_used= 11,584 block_cnt= 1 chunk_cnt= 1 mod=LogApplyStatus [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_TableCtxMgr [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MedCheckSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_ClientTask [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=DiaTabletMap [MEMORY] hold= 16,384 used= 8,448 count= 1 avg_used= 8,448 block_cnt= 1 chunk_cnt= 1 mod=PalfEnv [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TabletSvrBucket [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=ColSubStrPl [MEMORY] hold= 15,648 used= 15,264 count= 2 avg_used= 7,632 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoNode [MEMORY] hold= 11,984 used= 11,592 count= 2 avg_used= 5,796 block_cnt= 2 chunk_cnt= 1 mod=StopWordSet [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=Scheduler [MEMORY] hold= 9,488 used= 6,979 count= 13 avg_used= 536 block_cnt= 13 chunk_cnt= 10 mod=SqlSessiQuerSql [MEMORY] hold= 9,376 used= 8,984 count= 2 avg_used= 4,492 block_cnt= 2 chunk_cnt= 1 mod=FTPluginMap [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapCn [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapDi [MEMORY] hold= 8,192 used= 7,760 count= 2 avg_used= 3,880 block_cnt= 1 chunk_cnt= 1 mod=HashBucPsSessIn [MEMORY] hold= 8,176 used= 5,512 count= 13 avg_used= 424 block_cnt= 10 chunk_cnt= 6 mod=PackStateMap [MEMORY] hold= 8,144 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 8 mod=SequenceIdMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSIter [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LsElectRefInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LockWaitMgr [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=PurgeThrottle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=APPLY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=REPLAY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MatchLsnMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FetchLogTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ls_id [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MdsDebugMap [MEMORY] hold= 8,128 used= 5,512 count= 13 avg_used= 424 block_cnt= 12 chunk_cnt= 8 mod=SequenceMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=HandleSubmit [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LCLSender [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=IORunners [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompCkmItems [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairs [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ShareBlksMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FTPluginHandle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=DASIDCache [MEMORY] hold= 8,112 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 7 mod=ContextsMap [MEMORY] hold= 8,096 used= 7,904 count= 1 avg_used= 7,904 block_cnt= 1 chunk_cnt= 1 mod=PsSessionInfo [MEMORY] hold= 8,000 used= 7,808 count= 1 avg_used= 7,808 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsInfo [MEMORY] hold= 7,984 used= 7,792 count= 1 avg_used= 7,792 block_cnt= 1 chunk_cnt= 1 mod=ColUsagNode [MEMORY] hold= 7,920 used= 7,728 count= 1 avg_used= 7,728 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsCache [MEMORY] hold= 7,904 used= 7,712 count= 1 avg_used= 7,712 block_cnt= 1 chunk_cnt= 1 mod=DmlStatNode [MEMORY] hold= 7,824 used= 7,632 count= 1 avg_used= 7,632 block_cnt= 1 chunk_cnt= 1 mod=CheckMapNode [MEMORY] hold= 7,136 used= 66 count= 33 avg_used= 2 block_cnt= 31 chunk_cnt= 11 mod=[T]TSIUseWeak [MEMORY] hold= 6,800 used= 6,216 count= 3 avg_used= 2,072 block_cnt= 2 chunk_cnt= 2 mod=DetectManager [MEMORY] hold= 5,056 used= 4,656 count= 2 avg_used= 2,328 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResGrpIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResRuleIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=PxPoolBkt [MEMORY] hold= 4,288 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 4,160 used= 3,968 count= 1 avg_used= 3,968 block_cnt= 1 chunk_cnt= 1 mod=TinyAllocator [MEMORY] hold= 3,328 used= 256 count= 16 avg_used= 16 block_cnt= 16 chunk_cnt= 14 mod=[T]TSILastOper [MEMORY] hold= 3,328 used= 1,792 count= 8 avg_used= 224 block_cnt= 5 chunk_cnt= 2 mod=LogIOTask [MEMORY] hold= 3,200 used= 3,008 count= 1 avg_used= 3,008 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlDfc [MEMORY] hold= 2,944 used= 1,920 count= 5 avg_used= 384 block_cnt= 5 chunk_cnt= 3 mod=TableLockNode [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=IRMMemHashBuck [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=MedCheckMap [MEMORY] hold= 2,048 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=LogIOWS [MEMORY] hold= 2,000 used= 1,800 count= 1 avg_used= 1,800 block_cnt= 1 chunk_cnt= 1 mod=PxResMgr [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=MSTXCTX [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TXBFRetryId [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TabStatMgrLock [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GROUP_INDEX_MAP [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GCMemtableMap [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=DUP_LS_SET [MEMORY] hold= 1,344 used= 960 count= 2 avg_used= 480 block_cnt= 1 chunk_cnt= 1 mod=PoolArenaArray [MEMORY] hold= 1,280 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=ModuleInitCtx [MEMORY] hold= 1,248 used= 1,056 count= 1 avg_used= 1,056 block_cnt= 1 chunk_cnt= 1 mod=LOG_HASH_MAP [MEMORY] hold= 1,216 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=LockMemtableMgr [MEMORY] hold= 1,152 used= 960 count= 1 avg_used= 960 block_cnt= 1 chunk_cnt= 1 mod=FTParserMgr [MEMORY] hold= 1,120 used= 120 count= 5 avg_used= 24 block_cnt= 2 chunk_cnt= 1 mod=FreezeTask [MEMORY] hold= 976 used= 776 count= 1 avg_used= 776 block_cnt= 1 chunk_cnt= 1 mod=DepTabletSet [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=CheckPointTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=FlushTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TLD_TIMER [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TabShellTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=DiskUsageTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=tenant_rm_mgr [MEMORY] hold= 848 used= 648 count= 1 avg_used= 648 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfo [MEMORY] hold= 752 used= 552 count= 1 avg_used= 552 block_cnt= 1 chunk_cnt= 1 mod=LSFreeze [MEMORY] hold= 672 used= 288 count= 2 avg_used= 144 block_cnt= 2 chunk_cnt= 2 mod=ObTxLSLogCb [MEMORY] hold= 672 used= 472 count= 1 avg_used= 472 block_cnt= 1 chunk_cnt= 1 mod=Election [MEMORY] hold= 576 used= 384 count= 1 avg_used= 384 block_cnt= 1 chunk_cnt= 1 mod=HAScheduler [MEMORY] hold= 544 used= 144 count= 2 avg_used= 72 block_cnt= 1 chunk_cnt= 1 mod=TntSrvObjPool [MEMORY] hold= 512 used= 120 count= 2 avg_used= 60 block_cnt= 2 chunk_cnt= 2 mod=UserResourceMgr [MEMORY] hold= 448 used= 64 count= 2 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfFSCbNode [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTHandler [MEMORY] hold= 400 used= 208 count= 1 avg_used= 208 block_cnt= 1 chunk_cnt= 1 mod=[T]ObMacroBlock [MEMORY] hold= 320 used= 102 count= 1 avg_used= 102 block_cnt= 1 chunk_cnt= 1 mod=TxLSLogBuf [MEMORY] hold= 304 used= 112 count= 1 avg_used= 112 block_cnt= 1 chunk_cnt= 1 mod=Coordinator [MEMORY] hold= 304 used= 104 count= 1 avg_used= 104 block_cnt= 1 chunk_cnt= 1 mod=RedoLogBuckLock [MEMORY] hold= 256 used= 48 count= 1 avg_used= 48 block_cnt= 1 chunk_cnt= 1 mod=[T]TSITabletTab [MEMORY] hold= 256 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfoMap [MEMORY] hold= 256 used= 64 count= 1 avg_used= 64 block_cnt= 1 chunk_cnt= 1 mod=NegParamIdex [MEMORY] hold= 240 used= 41 count= 1 avg_used= 41 block_cnt= 1 chunk_cnt= 1 mod=KeepAliveBuf [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfRCCbNode [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=RebuildCbNode [MEMORY] hold= 208 used= 16 count= 1 avg_used= 16 block_cnt= 1 chunk_cnt= 1 mod=logservice [MEMORY] hold= 449,784,000 used= 432,566,115 count= 33,098 avg_used= 13,069 mod=SUMMARY [2024-10-16 14:57:16.306486] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=57] [MEMORY] tenant_id= 1 ctx_id= MEMSTORE_CTX_ID hold= 245,366,784 used= 243,449,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 block_cnt= 117 chunk_cnt= 117 mod=Memstore [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 mod=SUMMARY [2024-10-16 14:57:16.306471] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:21892}, to_seq_no={branch:0, seq:10421}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285064} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.306497] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 1 ctx_id= TRANS_CTX_MGR_ID hold= 2,097,152 used= 729,088 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 block_cnt= 1 chunk_cnt= 1 mod=PartTranCtxMgr [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 mod=SUMMARY [2024-10-16 14:57:16.306507] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= PLAN_CACHE_CTX_ID hold= 79,691,776 used= 73,331,136 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 68,663,616 used= 62,143,518 count= 2,832 avg_used= 21,943 block_cnt= 2,832 chunk_cnt= 37 mod=SqlPhyPlan [MEMORY] hold= 3,414,336 used= 3,146,720 count= 326 avg_used= 9,652 block_cnt= 326 chunk_cnt= 37 mod=SqlPlanCache [MEMORY] hold= 1,251,200 used= 1,140,800 count= 575 avg_used= 1,984 block_cnt= 192 chunk_cnt= 37 mod=CreateContext [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 73,331,136 used= 66,432,638 count= 3,735 avg_used= 17,786 mod=SUMMARY [2024-10-16 14:57:16.306515] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= WORK_AREA hold= 0 used= 50,368 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 50,368 used= 44,045 count= 32 avg_used= 1,376 block_cnt= 9 chunk_cnt= 1 mod=ArenaHashJoin [MEMORY] hold= 50,368 used= 44,045 count= 32 avg_used= 1,376 mod=SUMMARY [2024-10-16 14:57:16.306528] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=9] [MEMORY] tenant_id= 1 ctx_id= GLIBC hold= 2,097,152 used= 81,040 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 80,944 used= 59,018 count= 263 avg_used= 224 block_cnt= 20 chunk_cnt= 1 mod=PlJit [MEMORY] hold= 96 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PlCodeGen [MEMORY] hold= 81,040 used= 59,050 count= 264 avg_used= 223 mod=SUMMARY [2024-10-16 14:57:16.306536] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= CO_STACK hold= 136,314,880 used= 134,184,960 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 134,184,960 used= 133,935,360 count= 260 avg_used= 515,136 block_cnt= 260 chunk_cnt= 65 mod=CoStack [MEMORY] hold= 134,184,960 used= 133,935,360 count= 260 avg_used= 515,136 mod=SUMMARY [2024-10-16 14:57:16.306548] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= KVSTORE_CACHE_ID hold= 861,929,472 used= 861,929,472 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 mod=SUMMARY [2024-10-16 14:57:16.306557] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= META_OBJ_CTX_ID hold= 37,748,736 used= 36,290,496 limit= 429,496,720 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 29,491,200 used= 26,236,400 count= 400 avg_used= 65,591 block_cnt= 400 chunk_cnt= 18 mod=L_TabletPool [MEMORY] hold= 6,709,248 used= 5,952,128 count= 91 avg_used= 65,408 block_cnt= 91 chunk_cnt= 15 mod=N_TabletPool [MEMORY] hold= 90,048 used= 84,768 count= 2 avg_used= 42,384 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 36,290,496 used= 32,273,296 count= 493 avg_used= 65,463 mod=SUMMARY [2024-10-16 14:57:16.306565] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TX_CALLBACK_CTX_ID hold= 12,582,912 used= 6,234,176 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 block_cnt= 767 chunk_cnt= 6 mod=MemtableCallbac [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:16.306573] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= LOB_CTX_ID hold= 2,097,152 used= 81,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LobAllocator [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 81,856 used= 73,344 count= 2 avg_used= 36,672 mod=SUMMARY [2024-10-16 14:57:16.306581] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= PS_CACHE_CTX_ID hold= 2,097,152 used= 16,288 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 block_cnt= 4 chunk_cnt= 1 mod=SqlPsCache [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 mod=SUMMARY [2024-10-16 14:57:16.306595] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TX_DATA_TABLE hold= 54,546,432 used= 52,738,944 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 49,548,288 used= 48,377,856 count= 6,096 avg_used= 7,936 block_cnt= 6,096 chunk_cnt= 25 mod=TX_DATA_SLICE [MEMORY] hold= 3,182,528 used= 3,153,728 count= 3 avg_used= 1,051,242 block_cnt= 3 chunk_cnt= 3 mod=MEMTABLE_ARENA [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 52,738,944 used= 51,539,520 count= 6,100 avg_used= 8,449 mod=SUMMARY [2024-10-16 14:57:16.306611] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= MDS_DATA_ID hold= 4,194,304 used= 398,272 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 260,096 used= 253,952 count= 32 avg_used= 7,936 block_cnt= 32 chunk_cnt= 2 mod=MdsTable [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 2 mod=TX_OP [MEMORY] hold= 398,272 used= 388,864 count= 49 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:16.306621] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= SCHEMA_SERVICE hold= 2,097,152 used= 8,128 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=NonPartTenCac [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:16.306639] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,082,318,848 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 371,380,224 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 14,680,064 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 104,857,600 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 20,971,520 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= RPC_CTX_ID hold_bytes= 6,291,456 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 26,357,760 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 237,785,088 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= UNEXPECTED_IN_500 hold_bytes= 295,800,832 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:16.306844] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 371,380,224 used= 313,562,976 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 66,584,784 used= 66,551,824 count= 33 avg_used= 2,016,721 block_cnt= 33 chunk_cnt= 33 mod=Compressor [MEMORY] hold= 31,477,760 used= 31,458,304 count= 1 avg_used= 31,458,304 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 23,584,768 used= 23,499,200 count= 9 avg_used= 2,611,022 block_cnt= 9 chunk_cnt= 8 mod=IoControl [MEMORY] hold= 15,925,248 used= 15,247,872 count= 216 avg_used= 70,592 block_cnt= 216 chunk_cnt= 42 mod=[T]ObSessionDIB [MEMORY] hold= 12,873,728 used= 12,856,352 count= 1 avg_used= 12,856,352 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 9,457,664 used= 9,437,784 count= 1 avg_used= 9,437,784 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 5,505,024 used= 5,238,784 count= 32 avg_used= 163,712 block_cnt= 32 chunk_cnt= 22 mod=Iteratorget_logical_del_time()=9510304176555, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893594, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.308232] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 1003 ctx_id= DEFAULT_CTX_ID hold= 395,182,080 used= 301,135,824 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 25 [MEMORY] hold= 41,963,520 used= 41,943,040 count= 1 avg_used= 41,943,040 block_cnt= 1 chunk_cnt= 1 mod=LogGroupBuffer [MEMORY] hold= 24,969,216 used= 24,956,928 count= 12 avg_used= 2,079,744 block_cnt= 12 chunk_cnt= 12 mod=CACHE_MAP_NODE [MEMORY] hold= 23,511,040 used= 23,433,792 count= 8 avg_used= 2,929,224 block_cnt= 8 chunk_cnt= 6 mod=IoControl [MEMORY] hold= 16,883,712 used= 15,007,744 count= 229 avg_used= 65,536 block_cnt= 229 chunk_cnt= 26 mod=MinorMergeMgr [MEMORY] hold= 16,797,696 used= 16,777,336 count= 1 avg_used= 16,777,336 block_cnt= 1 chunk_cnt= 1 mod=MrgMemPoolChk [MEMORY] hold= 15,034,800 used= 14,886,881 count= 98 avg_used= 151,906 block_cnt= 35 chunk_cnt= 6 mod=OmtTenant [MEMORY] hold= 12,754,944 used= 12,212,416 count= 173 avg_used= 70,592 block_cnt= 173 chunk_cnt= 30 mod=[T]ObSessionDIB [MEMORY] hold= 11,223,040 used= 11,209,664 count= 7 avg_used= 1,601,380 block_cnt= 7 chunk_cnt= 7 mod=MysqlRequesReco [MEMORY] hold= 9,125,888 used= 9,118,976 count= 5 avg_used= 1,823,795 block_cnt= 5 chunk_cnt= 5 mod=SqlFltSpanRec [MEMORY] hold= 8,777,728 used= 8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream [MEMORY] hold= 8,626,176 used= 7,356,416 count= 155 avg_used= 47,460 block_cnt= 155 chunk_cnt= 31 mod=[T]char [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv [MEMORY] hold= 5,971,968 used= 5,308,416 count= 81 avg_used= 65,536 block_cnt= 81 chunk_cnt= 23 mod=MajorMergeMgr [MEMORY] hold= 4,235,264 used= 4,194,304 count= 2 avg_used= 2,097,152 block_cnt= 2 chunk_cnt= 2 mod=SHARE_BLOCK [MEMORY] hold= 4,214,784 used= 4,194,336 count= 1 avg_used= 4,194,336 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMemCU [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMem [MEMORY] hold= 3,809,280 used= 3,770,880 count= 5 avg_used= 754,176 block_cnt= 5 chunk_cnt= 4 mod=ResourceGroup [MEMORY] hold= 3,772,416 used= 3,751,936 count= 1 avg_used= 3,751,936 block_cnt= 1 chunk_cnt= 1 mod=DiskIteratorSto [MEMORY] hold= 3,723,264 used= 3,698,688 count= 2 avg_used= 1,849,344 block_cnt= 2 chunk_cnt= 2 mod=LogDIOAligned [MEMORY] hold= 3,162,112 used= 3,145,808 count= 2 avg_used= 1,572,904 block_cnt= 2 chunk_cnt= 2 mod=TenantDLMgr [MEMORY] hold= 3,145,728 used= 2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 5 mod=SqlDtlQueue [MEMORY] hold= 2,777,088 used= 2,760,192 count= 1 avg_used= 2,760,192 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=FastFrezCkr [MEMORY] hold= 2,375,680 used= 2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta [MEMORY] hold= 2,248,704 used= 2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=LogReplayTask [MEMORY] hold= 1,843,200 used= 1,770,696 count= 9 avg_used= 196,744 block_cnt= 9 chunk_cnt= 2 mod=DagMap [MEMORY] hold= 1,777,664 used= 1,770,696 count= 1 avg_used= 1,770,696 block_cnt= 1 chunk_cnt= 1 mod=TableProc [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=TransTimeWheel [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=XATimeWheel [MEMORY] hold= 1,256,480 used= 1,237,448 count= 58 avg_used= 21,335 block_cnt= 58 chunk_cnt= 4 mod=TabletMap [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=RewriteRuleMap [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPlanCac [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap [MEMORY] hold= 1,072,704 used= 1,028,304 count= 222 avg_used= 4,632 block_cnt= 222 chunk_cnt= 34 mod=WorkerMap [MEMORY] hold= 1,064,960 used= 575,640 count= 65 avg_used= 8,856 block_cnt= 65 chunk_cnt= 22 mod=[T]ObDSActionAr [MEMORY] hold= 1,024,000 used= 1,015,808 count= 1 avg_used= 1,015,808 block_cnt= 1 chunk_cnt= 1 mod=CommitVersions [MEMORY] hold= 958,464 used= 950,272 count= 1 avg_used= 950,272 block_cnt= 1 chunk_cnt= 1 mod=IOWorkerLQ [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=ArcSenderQueue [MEMORY] hold= 909,312 used= 884,808 count= 3 avg_used= 294,936 block_cnt= 3 chunk_cnt= 2 mod=TmpBlockMap [MEMORY] hold= 798,112 used= 789,624 count= 3 avg_used= 263,208 block_cnt= 2 chunk_cnt= 2 mod=PartitTableTask [MEMORY] hold= 794,624 used= 788,480 count= 2 avg_used= 394,240 block_cnt= 2 chunk_cnt= 2 mod=ServerObjecPool [MEMORY] hold= 786,688 used= 524,352 count= 33 avg_used= 15,889 block_cnt= 33 chunk_cnt= 2 mod=LogAlloc [MEMORY] hold= 663,552 used= 659,200 count= 1 avg_used= 659,200 block_cnt= 1 chunk_cnt= 1 mod=MulLevelQueue [MEMORY] hold= 663,552 used= 655,360 count= 1 avg_used= 655,360 block_cnt= 1 chunk_cnt= 1 mod=FetchLog [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MdsT [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DupTbLease [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DetectorTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=OBJLockGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=ElectTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MultiVersionGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTF [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTR [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=FrzTrigger [MEMORY] hold= 598,016 used= 590,232 count= 1 avg_used= 590,232 block_cnt= 1 chunk_cnt= 1 mod=DagNetIdMap [MEMORY] hold= 584,640 used= 563,878 count= 22 avg_used= 25,630 block_cnt= 22 chunk_cnt= 14 mod=SqlMemMgr [MEMORY] hold= 544,576 used= 531,712 count= 67 avg_used= 7,936 block_cnt= 67 chunk_cnt= 6 mod=TltMemtablMgr [MEMORY] hold= 532,480 used= 531,072 count= 1 avg_used= 531,072 block_cnt= 1 chunk_cnt= 1 mod=TransService [MEMORY] hold= 516,096 used= 463,953 count= 7 avg_used= 66,279 block_cnt= 7 chunk_cnt= 3 mod=SqlDtlBuf [MEMORY] hold= 495,008 used= 483,104 count= 62 avg_used= 7,792 block_cnt= 62 chunk_cnt= 10 mod=CkptDgnMemNode [MEMORY] hold= 458,752 used= 440,928 count= 3 avg_used= 146,976 block_cnt= 3 chunk_cnt= 2 mod=bf_queue [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ApplySrv [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ReplaySrv [MEMORY] hold= 385,024 used= 377,120 count= 1 avg_used= 377,120 block_cnt= 1 chunk_cnt= 1 mod=ClogGe [MEMORY] hold= 338,112 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 49 chunk_cnt= 5 mod=LSMap [MEMORY] hold= 316,160 used= 301,568 count= 76 avg_used= 3,968 block_cnt= 76 chunk_cnt= 15 mod=CkpDgn [MEMORY] hold= 311,296 used= 262,144 count= 256 avg_used= 1,024 block_cnt= 43 chunk_cnt= 2 mod=TabletToLS [MEMORY] hold= 311,104 used= 303,904 count= 6 avg_used= 50,650 block_cnt= 6 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=DmlStatHashMap [MEMORY] hold= 303,104 used= 298,256 count= 1 avg_used= 298,256 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMon [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompactMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompStMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=TmpMap [MEMORY] hold= 243,840 used= 238,080 count= 30 avg_used= 7,936 block_cnt= 30 chunk_cnt= 21 mod=LinearHashMapNo [MEMORY] hold= 229,376 used= 112,672 count= 14 avg_used= 8,048 block_cnt= 14 chunk_cnt= 9 mod=[T]ObPerfEventR [MEMORY] hold= 212,992 used= 212,160 count= 1 avg_used= 212,160 block_cnt= 1 chunk_cnt= 1 mod=MetaMemMgr [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=TenDLBucket [MEMORY] hold= 204,800 used= 196,744 count= 1 avg_used= 196,744 block_cnt= 1 chunk_cnt= 1 mod=DagNetMap [MEMORY] hold= 172,032 used= 165,344 count= 1 avg_used= 165,344 block_cnt= 1 chunk_cnt= 1 mod=SQLSessionInfo [MEMORY] hold= 170,688 used= 166,656 count= 21 avg_used= 7,936 block_cnt= 21 chunk_cnt= 10 mod=CtxBlockPl [MEMORY] hold= 147,456 used= 139,264 count= 1 avg_used= 139,264 block_cnt= 1 chunk_cnt= 1 mod=RFLTaskQueue [MEMORY] hold= 147,456 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 2 mod=MemTblObj [MEMORY] hold= 147,456 used= 131,072 count= 2 avg_used= 65,536 block_cnt= 2 chunk_cnt= 2 mod=SuspectInfoMgr [MEMORY] hold= 146,304 used= 142,848 count= 18 avg_used= 7,936 block_cnt= 18 chunk_cnt= 14 mod=QueryAllocator [MEMORY] hold= 122,880 used= 110,608 count= 2 avg_used= 55,304 block_cnt= 2 chunk_cnt= 2 mod=TabletIDSetBkt [MEMORY] hold= 114,688 used= 106,496 count= 1 avg_used= 106,496 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlMgr [MEMORY] hold= 114,688 used= 111,096 count= 1 avg_used= 111,096 block_cnt= 1 chunk_cnt= 1 mod=AsyncFreezeTabl [MEMORY] hold= 108,928 used= 104,512 count= 23 avg_used= 4,544 block_cnt= 23 chunk_cnt= 14 mod=[T]ObTraceEvent [MEMORY] hold= 106,496 used= 98,320 count= 1 avg_used= 98,320 block_cnt= 1 chunk_cnt= 1 mod=T3MBucket [MEMORY] hold= 106,496 used= 98,432 count= 1 avg_used= 98,432 block_cnt= 1 chunk_cnt= 1 mod=tenant_mf_mgr [MEMORY] hold= 106,496 used= 98,320 count= 1 avg_used= 98,320 block_cnt= 1 chunk_cnt= 1 mod=ResourMapLock [MEMORY] hold= 106,496 used= 98,528 count= 1 avg_used= 98,528 block_cnt= 1 chunk_cnt= 1 mod=LSSvr [MEMORY] hold= 92,480 used= 22,416 count= 347 avg_used= 64 block_cnt= 147 chunk_cnt= 27 mod=Coro [MEMORY] hold= 90,112 used= 77,752 count= 2 avg_used= 38,876 block_cnt= 2 chunk_cnt= 2 mod=backupTaskSched [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 6 mod=RawPl [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=RlePl [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 9 mod=ConstPl [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 10 mod=DictPl [MEMORY] hold= 77,824 used= 65,536 count= 64 avg_used= 1,024 block_cnt= 11 chunk_cnt= 2 mod=TxCtxMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TmpBlockManager [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushMeta [MEMORY] hold= 73,728 used= 72,736 count= 1 avg_used= 72,736 block_cnt= 1 chunk_cnt= 1 mod=LogSharedQueueT [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=ResultSet [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=StorMetaCacheIO [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxCtxMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushLog [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxDataMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LockMemObj [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 6 mod=ParamCharsType [MEMORY] hold= 72,000 used= 70,272 count= 9 avg_used= 7,808 block_cnt= 9 chunk_cnt= 2 mod=TabletIDSetNode [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 7 mod=IntDiffPl [MEMORY] hold= 57,344 used= 50,496 count= 1 avg_used= 50,496 block_cnt= 1 chunk_cnt= 1 mod=palf_env [MEMORY] hold= 55,888 used= 54,544 count= 7 avg_used= 7,792 block_cnt= 7 chunk_cnt= 5 mod=CkptDgnCUNode [MEMORY] hold= 48,960 used= 44,840 count= 2 avg_used= 22,420 block_cnt= 2 chunk_cnt= 2 mod=DynamicFactor [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 5 mod=ColEqualPl [MEMORY] hold= 43,776 used= 41,472 count= 12 avg_used= 3,456 block_cnt= 8 chunk_cnt= 6 mod=CACHE_INST [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DagWarnHisBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=Autoincrement [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=HexStrPl [MEMORY] hold= 38,800 used= 4,152 count= 173 avg_used= 24 block_cnt= 119 chunk_cnt= 23 mod=[T]MemoryContex [MEMORY] hold= 32,768 used= 17,408 count= 2 avg_used= 8,704 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTTP [MEMORY] hold= 32,768 used= 25,664 count= 1 avg_used= 25,664 block_cnt= 1 chunk_cnt= 1 mod=TSQLSessionMgr [MEMORY] hold= 32,704 used= 26,079 count= 2 avg_used= 13,039 block_cnt= 2 chunk_cnt= 2 mod=CGReadInfoMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 3 mod=MdsTableMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 4 mod=StrDiffPl [MEMORY] hold= 32,000 used= 31,232 count= 4 avg_used= 7,808 block_cnt= 4 chunk_cnt= 4 mod=HashNodeLCSta [MEMORY] hold= 25,552 used= 19,232 count= 2 avg_used= 9,616 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe [MEMORY] hold= 24,576 used= 22,592 count= 1 avg_used= 22,592 block_cnt= 1 chunk_cnt= 1 mod=IO_MGR [MEMORY] hold= 24,576 used= 16,384 count= 1 avg_used= 16,384 block_cnt= 1 chunk_cnt= 1 mod=SlogWriteBuffer [MEMORY] hold= 24,368 used= 20,136 count= 2 avg_used= 10,068 block_cnt= 2 chunk_cnt= 2 mod=TabletStats [MEMORY] hold= 21,168 used= 16,896 count= 22 avg_used= 768 block_cnt= 15 chunk_cnt= 13 mod=TGTimer [MEMORY] hold= 17,024 used= 8,448 count= 3 avg_used= 2,816 block_cnt= 2 chunk_cnt= 1 mod=BaseLogWriter [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MedCheckSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_AssignedMgr [MEMORY] hold= 16,384 used= 8,448 count= 1 avg_used= 8,448 block_cnt= 1 chunk_cnt= 1 mod=PalfEnv [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_TableCtxMgr [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TabletSvrBucket [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=SlogNopLog [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_ClientTask [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=DiaTabletMap [MEMORY] hold= 16,384 used= 11,584 count= 1 avg_used= 11,584 block_cnt= 1 chunk_cnt= 1 mod=LogApplyStatus [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TmpMemBlkMgr [MEMORY] hold= 16,384 used= 13,824 count= 1 avg_used= 13,824 block_cnt= 1 chunk_cnt= 1 mod=LogReplayStatus [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=StrPrefixPl [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=ColSubStrPl [MEMORY] hold= 16,000 used= 15,616 count= 2 avg_used= 7,808 block_cnt= 2 chunk_cnt= 2 mod=HashNodePlanCac [MEMORY] hold= 15,968 used= 15,584 count= 2 avg_used= 7,792 block_cnt= 2 chunk_cnt= 2 mod=ColUsagNode [MEMORY] hold= 15,648 used= 15,264 count= 2 avg_used= 7,632 block_cnt= 2 chunk_cnt= 1 mod=CaLsInfoNode [MEMORY] hold= 12,960 used= 10,080 count= 15 avg_used= 672 block_cnt= 14 chunk_cnt= 13 mod=[T]ObWarningBuf [MEMORY] hold= 11,984 used= 11,592 count= 2 avg_used= 5,796 block_cnt= 2 chunk_cnt= 1 mod=StopWordSet [MEMORY] hold= 11,552 used= 7,904 count= 19 avg_used= 416 block_cnt= 10 chunk_cnt= 7 mod=OMT_Worker [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=Scheduler [MEMORY] hold= 9,376 used= 8,984 count= 2 avg_used= 4,492 block_cnt= 2 chunk_cnt= 1 mod=FTPluginMap [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapCn [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapDi [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MdsDebugMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=HandleSubmit [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairs [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FetchLogTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ShareBlksMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=APPLY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LockWaitMgr [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompCkmItems [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=REPLAY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FTPluginHandle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=IORunners [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LsElectRefInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=DASIDCache [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ls_id [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MatchLsnMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LCLSender [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=PurgeThrottle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ObDMMDL [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSIter [MEMORY] hold= 8,000 used= 7,808 count= 1 avg_used= 7,808 block_cnt= 1 chunk_cnt= 1 mod=DirToBlk [MEMORY] hold= 7,904 used= 7,712 count= 1 avg_used= 7,712 block_cnt= 1 chunk_cnt= 1 mod=DmlStatNode [MEMORY] hold= 7,824 used= 7,632 count= 1 avg_used= 7,632 block_cnt= 1 chunk_cnt= 1 mod=CheckMapNode [MEMORY] hold= 6,800 used= 6,216 count= 3 avg_used= 2,072 block_cnt= 2 chunk_cnt= 2 mod=DetectManager [MEMORY] hold= 5,056 used= 4,656 count= 2 avg_used= 2,328 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResGrpIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResRuleIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=PxPoolBkt [MEMORY] hold= 4,288 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 4,160 used= 3,968 count= 1 avg_used= 3,968 block_cnt= 1 chunk_cnt= 1 mod=TinyAllocator [MEMORY] hold= 4,032 used= 2,688 count= 7 avg_used= 384 block_cnt= 6 chunk_cnt= 5 mod=TableLockNode [MEMORY] hold= 3,328 used= 1,792 count= 8 avg_used= 224 block_cnt= 1 chunk_cnt= 1 mod=LogIOTask [MEMORY] hold= 3,200 used= 3,008 count= 1 avg_used= 3,008 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlDfc [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=MedCheckMap [MEMORY] hold= 2,048 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=LogIOWS [MEMORY] hold= 2,000 used= 1,800 count= 1 avg_used= 1,800 block_cnt= 1 chunk_cnt= 1 mod=PxResMgr [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=MSTXCTX [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TXBFRetryId [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TabStatMgrLock [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GCMemtableMap [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=DUP_LS_SET [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=IRMMemHashBuck [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GROUP_INDEX_MAP [MEMORY] hold= 1,280 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=ModuleInitCtx [MEMORY] hold= 1,248 used= 12 count= 6 avg_used= 2 block_cnt= 6 chunk_cnt= 4 mod=[T]TSIUseWeak [MEMORY] hold= 1,248 used= 1,056 count= 1 avg_used= 1,056 block_cnt= 1 chunk_cnt= 1 mod=LOG_HASH_MAP [MEMORY] hold= 1,248 used= 96 count= 6 avg_used= 16 block_cnt= 6 chunk_cnt= 6 mod=[T]TSILastOper [MEMORY] hold= 1,216 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=LockMemtableMgr [MEMORY] hold= 1,152 used= 960 count= 1 avg_used= 960 block_cnt= 1 chunk_cnt= 1 mod=FTParserMgr [MEMORY] hold= 1,120 used= 120 count= 5 avg_used= 24 block_cnt= 2 chunk_cnt= 1 mod=FreezeTask [MEMORY] hold= 1,024 used= 640 count= 2 avg_used= 320 block_cnt= 1 chunk_cnt= 1 mod=PoolArenaArray [MEMORY] hold= 976 used= 776 count= 1 avg_used= 776 block_cnt= 1 chunk_cnt= 1 mod=DepTabletSet [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TLD_TIMER [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TabShellTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=DiskUsageTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=FlushTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=CheckPointTimer [MEMORY] hold= 848 used= 648 count= 1 avg_used= 648 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfo [MEMORY] hold= 752 used= 552 count= 1 avg_used= 552 block_cnt= 1 chunk_cnt= 1 mod=LSFreeze [MEMORY] hold= 688 used= 288 count= 2 avg_used= 144 block_cnt= 2 chunk_cnt= 2 mod=ObTxLSLogCb [MEMORY] hold= 672 used= 472 count= 1 avg_used= 472 block_cnt= 1 chunk_cnt= 1 mod=Election [MEMORY] hold= 576 used= 384 count= 1 avg_used= 384 block_cnt= 1 chunk_cnt= 1 mod=HAScheduler [MEMORY] hold= 544 used= 144 count= 2 avg_used= 72 block_cnt= 1 chunk_cnt= 1 mod=TntSrvObjPool [MEMORY] hold= 448 used= 64 count= 2 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfFSCbNode [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTHandler [MEMORY] hold= 400 used= 208 count= 1 avg_used= 208 block_cnt= 1 chunk_cnt= 1 mod=[T]ObMacroBlock [MEMORY] hold= 320 used= 112 count= 1 avg_used= 112 block_cnt= 1 chunk_cnt= 1 mod=Coordinator [MEMORY] hold= 320 used= 102 count= 1 avg_used= 102 block_cnt= 1 chunk_cnt= 1 mod=TxLSLogBuf [MEMORY] hold= 304 used= 104 count= 1 avg_used= 104 block_cnt= 1 chunk_cnt= 1 mod=RedoLogBuckLock [MEMORY] hold= 256 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfoMap [MEMORY] hold= 240 used= 41 count= 1 avg_used= 41 block_cnt= 1 chunk_cnt= 1 mod=KeepAliveBuf [MEMORY] hold= 240 used= 48 count= 1 avg_used= 48 block_cnt= 1 chunk_cnt= 1 mod=[T]TSITabletTab [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfRCCbNode [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=RebuildCbNode [MEMORY] hold= 208 used= 16 count= 1 avg_used= 16 block_cnt= 1 chunk_cnt= 1 mod=logservice [MEMORY] hold= 301,135,824 used= 292,870,450 count= 4,754 avg_used= 61,605 mod=SUMMARY [2024-10-16 14:57:16.308285] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=48] [MEMORY] tenant_id= 1003 ctx_id= MEMSTORE_CTX_ID hold= 60,817,408 used= 60,342,272 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 60,342,272 used= 60,314,664 count= 29 avg_used= 2,079,816 block_cnt= 29 chunk_cnt= 29 mod=Memstore [MEMORY] hold= 60,342,272 used= 60,314,664 count= 29 avg_used= 2,079,816 mod=SUMMARY [2024-10-16 14:57:16.308295] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1003 ctx_id= TRANS_CTX_MGR_ID hold= 2,097,152 used= 729,088 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 block_cnt= 1 chunk_cnt= 1 mod=PartTranCtxMgr [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 mod=SUMMARY [2024-10-16 14:57:16.308305] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1003 ctx_id= PLAN_CACHE_CTX_ID hold= 48,234,496 used= 28,013,696 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 26,356,096 used= 23,843,126 count= 1,113 avg_used= 21,422 block_cnt= 1,113 chunk_cnt= 23 mod=SqlPhyPlan [MEMORY] hold= 1,172,544 used= 1,092,048 count= 124 avg_used= 8,806 block_cnt= 124 chunk_cnt= 22 mod=SqlPlanCache [MEMORY] hold= 483,072 used= 440,448 count= 222 avg_used= 1,984 block_cnt= 74 chunk_cnt= 20 mod=CreateContext [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 28,013,696 used= 25,377,222 count= 1,461 avg_used= 17,369 mod=SUMMARY [2024-10-16 14:57:16.308317] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1003 ctx_id= CO_STACK hold= 94,371,840 used= 90,316,800 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 90,316,800 used= 90,148,800 count= 175 avg_used= 515,136 block_cnt= 175 chunk_cnt= 45 mod=CoStack [MEMORY] hold= 90,316,800 used= 90,148,800 count= 175 avg_used= 515,136 mod=SUMMARY [2024-10-16 14:57:16.308329] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1003 ctx_id= KVSTORE_CACHE_ID hold= 278,921,216 used= 278,921,216 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 278,921,216 used= 0 count= 133 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 278,921,216 used= 0 count= 133 avg_used= 0 mod=SUMMARY [2024-10-16 14:57:16.308338] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1003 ctx_id= META_OBJ_CTX_ID hold= 20,971,520 used= 19,259,328 limit= 214,748,360 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 15,040,512 used= 13,380,564 count= 204 avg_used= 65,591 block_cnt= 204 chunk_cnt= 10 mod=L_TabletPool [MEMORY] hold= 4,128,768 used= 3,662,848 count= 56 avg_used= 65,408 block_cnt= 56 chunk_cnt= 6 mod=N_TabletPool [MEMORY] hold= 90,048 used= 84,768 count= 2 avg_used= 42,384 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 19,259,328 used= 17,128,180 count= 262 avg_used= 65,374 mod=SUMMARY [2024-10-16 14:57:16.308345] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1003 ctx_id= TX_CALLBACK_CTX_ID hold= 6,291,456 used= 4,080,256 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 4,080,256 used= 3,983,872 count= 502 avg_used= 7,936 block_cnt= 502 chunk_cnt= 3 mod=MemtableCallbac [MEMORY] hold= 4,080,256 used= 3,983,872 count= 502 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:16.308355] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 1003 ctx_id= LOB_CTX_ID hold= 2,097,152 used= 81,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LobAllocator [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 81,856 used= 73,344 count= 2 avg_used= 36,672 mod=SUMMARY [2024-10-16 14:57:16.308371] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1003 ctx_id= TX_DATA_TABLE hold= 27,283,456 used= 14,992,512 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 11,801,856 used= 11,523,072 count= 1,452 avg_used= 7,936 block_cnt= 1,452 chunk_cnt= 11 mod=TX_DATA_SLICE [MEMORY] hold= 3,182,528 used= 3,153,728 count= 3 avg_used= 1,051,242 block_cnt= 3 chunk_cnt= 3 mod=MEMTABLE_ARENA [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 14,992,512 used= 14,684,736 count= 1,456 avg_used= 10,085 mod=SUMMARY [2024-10-16 14:57:16.308381] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1003 ctx_id= MDS_DATA_ID hold= 2,097,152 used= 316,992 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 260,096 used= 253,952 count= 32 avg_used= 7,936 block_cnt= 32 chunk_cnt= 1 mod=MdsTable [MEMORY] hold= 56,896 used= 55,552 count= 7 avg_used= 7,936 block_cnt= 7 chunk_cnt= 1 mod=TX_OP [MEMORY] hold= 316,992 used= 309,504 count= 39 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:16.308403] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant: 1004, limit: 1,073,741,824 hold: 952,647,680 rpc_hold: 0 cache_hold: 79,691,776 cache_used: 79,691,776 cache_item_count: 38 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 606,367,744 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MEMSTORE_CTX_ID hold_bytes= 39,845,888 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TRANS_CTX_MGR_ID hold_bytes= 2,949,120 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PLAN_CACHE_CTX_ID hold_bytes= 43,933,696 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 100,974,592 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= META_OBJ_CTX_ID hold_bytes= 37,781,504 limit= 214,748,360 [MEMORY] ctx_id= TX_CALLBACK_CTX_ID hold_bytes= 5,013,504 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOB_CTX_ID hold_bytes= 114,688 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PS_CACHE_CTX_ID hold_bytes= 3,293,184 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TX_DATA_TABLE hold_bytes= 25,899,008 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MDS_DATA_ID hold_bytes= 6,782,976 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:16.308610] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 1004 ctx_id= DEFAULT_CTX_ID hold= 606,367,744 used= 508,385,920 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 159 washed_blocks= 884 washed_size= 231,309,312 [MEMORY] request_cached_chunk_cnt= 22 [MEMORY] hold= 167,854,080 used= 167,772,160 count= 4 avg_used= 41,943,040 block_cnt= 4 chunk_cnt= 4 mod=LogGroupBuffer [MEMORY] hold= 27,049,984 used= 27,036,672 count= 13 avg_used= 2,079,744 block_cnt= 13 chunk_cnt= 13 mod=CACHE_MAP_NODE [MEMORY] hold= 23,511,040 used= 23,433,792 count= 8 avg_used= 2,929,224 block_cnt= 8 chunk_cnt= 6 mod=IoControl [MEMORY] hold= 18,137,088 used= 16,121,856 count= 246 avg_used= 65,536 block_cnt= 246 chunk_cnt= 50 mod=MinorMergeMgr [MEMORY] hold= 17,473,536 used= 15,708,123 count= 237 avg_used= 66,279 block_cnt= 237 chunk_cnt= 57 mod=SqlDtlBuf [MEMORY] hold= 16,797,696 used= 16,777,336 count= 1 avg_used= 16,777,336 block_cnt= 1 chunk_cnt= 1 mod=MrgMemPoolChk [MEMORY] hold= 15,368,192 used= 15,358,208 count= 8 avg_used= 1,919,776 block_cnt= 8 chunk_cnt= 8 mod=SqlFltSpanRec [MEMORY] hold= 15,089,664 used= 15,007,744 count= 4 avg_used= 3,751,936 block_cnt= 4 chunk_cnt= 4 mod=DiskIteratorSto [MEMORY] hold= 15,034,800 used= 14,886,881 count= 98 avg_used= 151,906 block_cnt= 35 chunk_cnt= 6 mod=OmtTenant [MEMORY] hold= 14,893,056 used= 14,794,752 count= 8 avg_used= 1,849,344 block_cnt= 8 chunk_cnt= 6 mod=LogDIOAligned [MEMORY] hold= 14,082,048 used= 13,483,072 count= 191 avg_used= 70,592 block_cnt= 191 chunk_cnt= 35 mod=[T]ObSessionDIB [MEMORY] hold= 11,501,568 used= 10,223,616 count= 156 avg_used= 65,536 block_cnt= 156 chunk_cnt= 14 mod=DagWarnHisMgr [MEMORY] hold= 9,232,384 used= 7,880,704 count= 165 avg_used= 47,761 block_cnt= 165 chunk_cnt= 85 mod=[T]char [MEMORY] hold= 9,142,272 used= 9,129,920 count= 6 avg_used= 1,521,653 block_cnt= 6 chunk_cnt= 6 mod=MysqlRequesReco [MEMORY] hold= 8,777,728 used= 8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv [MEMORY] hold= 6,094,848 used= 6,033,408 count= 8 avg_used= 754,176 block_cnt= 8 chunk_cnt= 7 mod=ResourceGroup [MEMORY] hold= 6,045,696 used= 5,373,952 count= 82 avg_used= 65,536 block_cnt= 82 chunk_cnt= 6 mod=SuspectInfoMgr [MEMORY] hold= 6,045,696 used= 5,373,952 count= 82 avg_used= 65,536 block_cnt= 82 chunk_cnt= 35 mod=MajorMergeMgr [MEMORY] hold= 4,235,264 used= 4,194,304 count= 2 avg_used= 2,097,152 block_cnt= 2 chunk_cnt= 2 mod=SHARE_BLOCK [MEMORY] hold= 4,214,784 used= 4,194,336 count= 1 avg_used= 4,194,336 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 11 mod=CkptDgnMemCU [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 11 mod=CkptDgnMem [MEMORY] hold= 3,162,112 used= 3,145,808 count= 2 avg_used= 1,572,904 block_cnt= 2 chunk_cnt= 2 mod=TenantDLMgr [MEMORY] hold= 3,145,728 used= 2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 5 mod=SqlDtlQueue [MEMORY] hold= 2,777,088 used= 2,760,192 count= 1 avg_used= 2,760,192 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo [MEMORY] hold= 2,486,272 used= 2,457,920 count= 2 avg_used= 1,228,960 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=FastFrezCkr [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap [MEMORY] hold= 2,375,680 used= 2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta [MEMORY] hold= 2,248,704 used= 2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=LogReplayTask [MEMORY] hold= 1,843,200 used= 1,770,696 count= 9 avg_used= 196,744 block_cnt= 9 chunk_cnt= 2 mod=DagMap [MEMORY] hold= 1,799,472 used= 1,767,576 count= 125 avg_used= 14,140 block_cnt= 125 chunk_cnt= 5 mod=TabletMap [MEMORY] hold= 1,777,664 used= 1,770,696 count= 1 avg_used= 1,770,696 block_cnt= 1 chunk_cnt= 1 mod=TableProc [MEMORY] hold= 1,540,096 used= 1,508,480 count= 4 avg_used= 377,120 block_cnt= 4 chunk_cnt= 1 mod=ClogGe [MEMORY] hold= 1,376,256 used= 1,343,488 count= 4 avg_used= 335,872 block_cnt= 4 chunk_cnt= 2 mod=IOWorkerLQ [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=XATimeWheel [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=TransTimeWheel [MEMORY] hold= 1,212,416 used= 655,344 count= 74 avg_used= 8,856 block_cnt= 74 chunk_cnt= 24 mod=[T]ObDSActionAr [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsInfo [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPlanCac [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsCache [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=RewriteRuleMap [MEMORY] hold= 1,178,560 used= 1,150,720 count= 145 avg_used= 7,936 block_cnt= 145 chunk_cnt= 16 mod=TltMemtablMgr [MEMORY] hold= 1,001,920 used= 959,952 count= 210 avg_used= 4,571 block_cnt= 206 chunk_cnt= 50 mod=WorkerMap [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=ArcSenderQueue [MEMORY] hold= 909,312 used= 884,808 count= 3 avg_used= 294,936 block_cnt= 3 chunk_cnt= 1 mod=TmpBlockMap [MEMORY] hold= 798,112 used= 789,624 count= 3 avg_used= 263,208 block_cnt= 2 chunk_cnt= 2 mod=PartitTableTask [MEMORY] hold= 794,624 used= 788,480 count= 2 avg_used= 394,240 block_cnt= 2 chunk_cnt= 2 mod=ServerObjecPool [MEMORY] hold= 787,264 used= 524,352 count= 36 avg_used= 14,565 block_cnt= 34 chunk_cnt= 3 mod=LogAlloc [MEMORY] hold= 663,552 used= 659,200 count= 1 avg_used= 659,200 block_cnt= 1 chunk_cnt= 1 mod=MulLevelQueue [MEMORY] hold= 663,552 used= 655,360 count= 1 avg_used= 655,360 block_cnt= 1 chunk_cnt= 1 mod=FetchLog [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=FrzTrigger [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MultiVersionGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=OBJLockGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DupTbLease [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTF [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTR [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=ElectTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MdsT [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DetectorTimer [MEMORY] hold= 598,016 used= 590,232 count= 1 avg_used= 590,232 block_cnt= 1 chunk_cnt= 1 mod=DagNetIdMap [MEMORY] hold= 584,640 used= 563,878 count= 22 avg_used= 25,630 block_cnt= 22 chunk_cnt= 14 mod=SqlMemMgr [MEMORY] hold= 532,480 used= 531,072 count= 1 avg_used= 531,072 block_cnt= 1 chunk_cnt= 1 mod=TransService [MEMORY] hold= 491,520 used= 442,432 count= 8 avg_used= 55,304 block_cnt= 8 chunk_cnt= 3 mod=TabletIDSetBkt [MEMORY] hold= 458,752 used= 444,384 count= 4 avg_used= 111,096 block_cnt= 4 chunk_cnt= 2 mod=AsyncFreezeTabl [MEMORY] hold= 425,984 used= 394,112 count= 4 avg_used= 98,528 block_cnt= 4 chunk_cnt= 2 mod=LSSvr [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ApplySrv [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ReplaySrv [MEMORY] hold= 385,024 used= 375,520 count= 2 avg_used= 187,760 block_cnt= 2 chunk_cnt= 2 mod=bf_queue [MEMORY] hold= 338,112 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 49 chunk_cnt= 6 mod=LSMap [MEMORY] hold= 311,296 used= 262,144 count= 256 avg_used= 1,024 block_cnt= 43 chunk_cnt= 2 mod=TabletToLS [MEMORY] hold= 311,104 used= 303,904 count= 6 avg_used= 50,650 block_cnt= 6 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=TmpMap [MEMORY] hold= 303,104 used= 298,256 count= 1 avg_used= 298,256 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMon [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompStMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=DmlStatHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompactMap [MEMORY] hold= 284,480 used= 277,760 count= 35 avg_used= 7,936 block_cnt= 35 chunk_cnt= 23 mod=CtxBlockPl [MEMORY] hold= 276,352 used= 269,824 count= 34 avg_used= 7,936 block_cnt= 34 chunk_cnt= 24 mod=LinearHashMapNo [MEMORY] hold= 262,144 used= 253,952 count= 1 avg_used= 253,952 block_cnt= 1 chunk_cnt= 1 mod=CommitVersions [MEMORY] hold= 237,504 used= 229,624 count= 2 avg_used= 114,812 block_cnt= 2 chunk_cnt= 2 mod=IndexUsageMgr [MEMORY] hold= 229,376 used= 221,688 count= 1 avg_used= 221,688 block_cnt= 1 chunk_cnt= 1 mod=IndexUsageRepor [MEMORY] hold= 229,376 used= 201,984 count= 4 avg_used= 50,496 block_cnt= 4 chunk_cnt= 2 mod=palf_env [MEMORY] hold= 212,992 used= 104,624 count= 13 avg_used= 8,048 block_cnt= 13 chunk_cnt= 12 mod=[T]ObPerfEventR [MEMORY] hold= 212,992 used= 212,160 count= 1 avg_used= 212,160 block_cnt= 1 chunk_cnt= 1 mod=MetaMemMgr [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=TenDLBucket [MEMORY] hold= 204,800 used= 196,744 count= 1 avg_used= 196,744 block_cnt= 1 chunk_cnt= 1 mod=DagNetMap [MEMORY] hold= 191,616 used= 187,008 count= 24 avg_used= 7,792 block_cnt= 24 chunk_cnt= 4 mod=CkptDgnMemNode [MEMORY] hold= 183,632 used= 179,216 count= 23 avg_used= 7,792 block_cnt= 23 chunk_cnt= 17 mod=CkptDgnCUNode [MEMORY] hold= 172,032 used= 165,344 count= 1 avg_used= 165,344 block_cnt= 1 chunk_cnt= 1 mod=SQLSessionInfo [MEMORY] hold= 168,000 used= 163,968 count= 21 avg_used= 7,808 block_cnt= 21 chunk_cnt= 2 mod=TabletIDSetNode [MEMORY] hold= 147,456 used= 139,264 count= 1 avg_used= 139,264 block_cnt= 1 chunk_cnt= 1 mod=RFLTaskQueue [MEMORY] hold= 147,456 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 2 mod=MemTblObj [MEMORY] hold= 147,456 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 1 mod=LockMemObj [MEMORY] hold= 130,048 used= 126,976 count= 16 avg_used= 7,936 block_cnt= 16 chunk_cnt= 10 mod=QueryAllocator [MEMORY] hold= 118,432 used= 113,600 count= 25 avg_used= 4,544 block_cnt= 25 chunk_cnt= 15 mod=[T]ObTraceEvent [MEMORY] hold= 114,688 used= 106,496 count= 1 avg_used= 106,496 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlMgr [MEMORY] hold= 106,496 used= 98,320 count= 1 avg_used= 98,320 block_cnt= 1 chunk_cnt= 1 mod=ResourMapLock [MEMORY] hold= 106,496 used= 98,320 count= 1 avg_used= 98,320 block_cnt= 1 chunk_cnt= 1 mod=T3MBucket [MEMORY] hold= 106,496 used= 98,432 count= 1 avg_used= 98,432 block_cnt= 1 chunk_cnt= 1 mod=tenant_mf_mgr [MEMORY] hold= 100,016 used= 24,456 count= 375 avg_used= 65 block_cnt= 168 chunk_cnt= 27 mod=Coro [MEMORY] hold= 99,840 used= 95,232 count= 24 avg_used= 3,968 block_cnt= 24 chunk_cnt= 18 mod=CkpDgn [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=ConstPl [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=RawPl [MEMORY] hold= 81,920 used= 73,896 count= 1 avg_used= 73,896 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairCache [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 8 mod=DictPl [MEMORY] hold= 79,840 used= 77,920 count= 10 avg_used= 7,792 block_cnt= 10 chunk_cnt= 3 mod=DiaTabletNode [MEMORY] hold= 77,824 used= 65,536 count= 64 avg_used= 1,024 block_cnt= 11 chunk_cnt= 2 mod=TxCtxMgr [MEMORY] hold= 73,728 used= 65,536 count= 1 avg_used= 65,536 block_cnt= 1 chunk_cnt= 1 mod=SHADiagMgr [MEMORY] hold= 73,728 used= 72,736 count= 1 avg_used= 72,736 block_cnt= 1 chunk_cnt= 1 mod=LogSharedQueueT [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushLog [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TmpBlockManager [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=StorMetaCacheIO [MEMORY] hold= 73,728 used= 66,176 count= 1 avg_used= 66,176 block_cnt= 1 chunk_cnt= 1 mod=Rpc [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxDataMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxCtxMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushMeta [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 9 mod=IntDiffPl [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 5 mod=MdsTableMgr [MEMORY] hold= 65,536 used= 46,336 count= 4 avg_used= 11,584 block_cnt= 4 chunk_cnt= 2 mod=LogApplyStatus [MEMORY] hold= 65,536 used= 55,296 count= 4 avg_used= 13,824 block_cnt= 4 chunk_cnt= 2 mod=LogReplayStatus [MEMORY] hold= 65,536 used= 40,992 count= 4 avg_used= 10,248 block_cnt= 4 chunk_cnt= 2 mod=TabletSvrBucket [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 7 mod=RlePl [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 8 mod=PartCtx [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 5 mod=ParamCharsType [MEMORY] hold= 56,896 used= 55,552 count= 7 avg_used= 7,936 block_cnt= 7 chunk_cnt= 7 mod=ColEqualPl [MEMORY] hold= 56,896 used= 55,552 count= 7 avg_used= 7,936 block_cnt= 7 chunk_cnt= 7 mod=ShareBlksMap [MEMORY] hold= 48,960 used= 44,840 count= 2 avg_used= 22,420 block_cnt= 2 chunk_cnt= 2 mod=DynamicFactor [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 6 mod=HexStrPl [MEMORY] hold= 47,424 used= 44,928 count= 13 avg_used= 3,456 block_cnt= 10 chunk_cnt= 9 mod=CACHE_INST [MEMORY] hold= 42,880 used= 4,584 count= 191 avg_used= 24 block_cnt= 129 chunk_cnt= 25 mod=[T]MemoryContex [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=Autoincrement [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DagWarnHisBkt [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=StrPrefixPl [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 1 mod=ObDMMDL [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=StrDiffPl [MEMORY] hold= 32,768 used= 17,408 count= 2 avg_used= 8,704 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTTP [MEMORY] hold= 32,768 used= 25,664 count= 1 avg_used= 25,664 block_cnt= 1 chunk_cnt= 1 mod=TSQLSessionMgr [MEMORY] hold= 32,768 used= 24,688 count= 2 avg_used= 12,344 block_cnt= 2 chunk_cnt= 2 mod=TLD_AssignedMgr [MEMORY] hold= 32,704 used= 26,079 count= 2 avg_used= 13,039 block_cnt= 2 chunk_cnt= 2 mod=CGReadInfoMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 4 mod=LSReadableSCN [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 2 mod=MatchLsnMap [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 4 mod=ColSubStrPl [MEMORY] hold= 32,000 used= 31,232 count= 4 avg_used= 7,808 block_cnt= 4 chunk_cnt= 3 mod=HashNodeLCSta [MEMORY] hold= 26,928 used= 21,504 count= 28 avg_used= 768 block_cnt= 21 chunk_cnt= 16 mod=TGTimer [MEMORY] hold= 24,576 used= 22,592 count= 1 avg_used= 22,592 block_cnt= 1 chunk_cnt= 1 mod=IO_MGR [MEMORY] hold= 24,576 used= 16,384 count= 1 avg_used= 16,384 block_cnt= 1 chunk_cnt= 1 mod=SlogWriteBuffer [MEMORY] hold= 24,368 used= 20,136 count= 2 avg_used= 10,068 block_cnt= 2 chunk_cnt= 2 mod=TabletStats [MEMORY] hold= 24,336 used= 20,104 count= 2 avg_used= 10,052 block_cnt= 2 chunk_cnt= 2 mod=TLD_ResourceMgr [MEMORY] hold= 21,440 used= 20,480 count= 5 avg_used= 4,096 block_cnt= 5 chunk_cnt= 2 mod=LinearHashMapDi [MEMORY] hold= 21,440 used= 20,480 count= 5 avg_used= 4,096 block_cnt= 5 chunk_cnt= 3 mod=LinearHashMapCn [MEMORY] hold= 17,024 used= 8,448 count= 3 avg_used= 2,816 block_cnt= 2 chunk_cnt= 1 mod=BaseLogWriter [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TmpMemBlkMgr [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=DiaTabletMap [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=SlogNopLog [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MedCheckSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_ClientTask [MEMORY] hold= 16,384 used= 8,448 count= 1 avg_used= 8,448 block_cnt= 1 chunk_cnt= 1 mod=PalfEnv [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_TableCtxMgr [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=ArcSendQueue [MEMORY] hold= 16,000 used= 15,616 count= 2 avg_used= 7,808 block_cnt= 2 chunk_cnt= 1 mod=HashNodePsInfo [MEMORY] hold= 16,000 used= 15,616 count= 2 avg_used= 7,808 block_cnt= 2 chunk_cnt= 2 mod=HashNodePlanCac [MEMORY] hold= 15,840 used= 15,456 count= 2 avg_used= 7,728 block_cnt= 2 chunk_cnt= 2 mod=HashNodePsCache [MEMORY] hold= 15,648 used= 15,264 count= 2 avg_used= 7,632 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoNode [MEMORY] hold= 14,496 used= 13,896 count= 3 avg_used= 4,632 block_cnt= 3 chunk_cnt= 2 mod=TTLTaskBucket [MEMORY] hold= 12,832 used= 12,440 count= 2 avg_used= 6,220 block_cnt= 2 chunk_cnt= 1 mod=PerfDiagItem [MEMORY] hold= 12,112 used= 9,408 count= 14 avg_used= 672 block_cnt= 13 chunk_cnt= 11 mod=[T]ObWarningBuf [MEMORY] hold= 11,984 used= 11,592 count= 2 avg_used= 5,796 block_cnt= 2 chunk_cnt= 1 mod=StopWordSet [MEMORY] hold= 10,944 used= 7,488 count= 18 avg_used= 416 block_cnt= 9 chunk_cnt= 6 mod=OMT_Worker [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=HAScheduler [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=Scheduler [MEMORY] hold= 9,376 used= 8,984 count= 2 avg_used= 4,492 block_cnt= 2 chunk_cnt= 1 mod=FTPluginMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=REPLAY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=HandleSubmit [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=PurgeThrottle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairs [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FetchLogTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MdsDebugMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LockWaitMgr [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LCLSender [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSIter [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ObDataDictSvc [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=IORunners [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=APPLY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompCkmItems [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=DASIDCache [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LsElectRefInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ls_id [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FTPluginHandle [MEMORY] hold= 8,000 used= 7,808 count= 1 avg_used= 7,808 block_cnt= 1 chunk_cnt= 1 mod=DirToBlk [MEMORY] hold= 7,984 used= 7,792 count= 1 avg_used= 7,792 block_cnt= 1 chunk_cnt= 1 mod=ColUsagNode [MEMORY] hold= 7,904 used= 7,712 count= 1 avg_used= 7,712 block_cnt= 1 chunk_cnt= 1 mod=DmlStatNode [MEMORY] hold= 7,824 used= 7,632 count= 1 avg_used= 7,632 block_cnt= 1 chunk_cnt= 1 mod=CheckMapNode [MEMORY] hold= 7,616 used= 7,424 count= 1 avg_used= 7,424 block_cnt= 1 chunk_cnt= 1 mod=LogIOWS [MEMORY] hold= 6,976 used= 6,176 count= 4 avg_used= 1,544 block_cnt= 4 chunk_cnt= 2 mod=TXBFRetryId [MEMORY] hold= 6,800 used= 6,216 count= 3 avg_used= 2,072 block_cnt= 2 chunk_cnt= 2 mod=DetectManager [MEMORY] hold= 5,056 used= 4,656 count= 2 avg_used= 2,328 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoMap [MEMORY] hold= 4,864 used= 4,096 count= 4 avg_used= 1,024 block_cnt= 3 chunk_cnt= 3 mod=LockMemtableMgr [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResGrpIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResRuleIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=PxPoolBkt [MEMORY] hold= 4,288 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 4,160 used= 3,968 count= 1 avg_used= 3,968 block_cnt= 1 chunk_cnt= 1 mod=TinyAllocator [MEMORY] hold= 3,904 used= 3,104 count= 4 avg_used= 776 block_cnt= 4 chunk_cnt= 2 mod=DepTabletSet [MEMORY] hold= 3,360 used= 32 count= 16 avg_used= 2 block_cnt= 16 chunk_cnt= 14 mod=[T]TSIUseWeak [MEMORY] hold= 3,328 used= 1,792 count= 8 avg_used= 224 block_cnt= 1 chunk_cnt= 1 mod=LogIOTask [MEMORY] hold= 3,200 used= 3,008 count= 1 avg_used= 3,008 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlDfc [MEMORY] hold= 2,688 used= 1,888 count= 4 avg_used= 472 block_cnt= 4 chunk_cnt= 1 mod=Election [MEMORY] hold= 2,688 used= 1,152 count= 8 avg_used= 144 block_cnt= 7 chunk_cnt= 6 mod=ObTxLSLogCb [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=MedCheckMap [MEMORY] hold= 2,000 used= 1,800 count= 1 avg_used= 1,800 block_cnt= 1 chunk_cnt= 1 mod=PxResMgr [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=MSTXCTX [MEMORY] hold= 1,792 used= 256 count= 8 avg_used= 32 block_cnt= 4 chunk_cnt= 2 mod=PalfFSCbNode [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TabStatMgrLock [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=IRMMemHashBuck [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=DUP_LS_SET [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GCMemtableMap [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GROUP_INDEX_MAP [MEMORY] hold= 1,280 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=ModuleInitCtx [MEMORY] hold= 1,264 used= 408 count= 4 avg_used= 102 block_cnt= 4 chunk_cnt= 3 mod=TxLSLogBuf [MEMORY] hold= 1,248 used= 1,056 count= 1 avg_used= 1,056 block_cnt= 1 chunk_cnt= 1 mod=LOG_HASH_MAP [MEMORY] hold= 1,216 used= 416 count= 4 avg_used= 104 block_cnt= 4 chunk_cnt= 2 mod=RedoLogBuckLock [MEMORY] hold= 1,152 used= 960 count= 1 avg_used= 960 block_cnt= 1 chunk_cnt= 1 mod=FTParserMgr [MEMORY] hold= 1,120 used= 120 count= 5 avg_used= 24 block_cnt= 2 chunk_cnt= 1 mod=FreezeTask [MEMORY] hold= 1,088 used= 896 count= 1 avg_used= 896 block_cnt= 1 chunk_cnt= 1 mod=tenant_ttl_mgr [MEMORY] hold= 1,040 used= 80 count= 5 avg_used= 16 block_cnt= 5 chunk_cnt= 5 mod=[T]TSILastOper [MEMORY] hold= 1,024 used= 640 count= 2 avg_used= 320 block_cnt= 1 chunk_cnt= 1 mod=PoolArenaArray [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=CheckPointTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=FlushTimer [MEMORY] hold= 960 used= 164 count= 4 avg_used= 41 block_cnt= 4 chunk_cnt= 2 mod=KeepAliveBuf [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=DiskUsageTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TabShellTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TLD_TIMER [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=tenant_rm_mgr [MEMORY] hold= 928 used= 128 count= 4 avg_used= 32 block_cnt= 4 chunk_cnt= 3 mod=PalfRCCbNode [MEMORY] hold= 896 used= 128 count= 4 avg_used= 32 block_cnt= 4 chunk_cnt= 3 mod=RebuildCbNode [MEMORY] hold= 848 used= 648 count= 1 avg_used= 648 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfo [MEMORY] hold= 752 used= 552 count= 1 avg_used= 552 block_cnt= 1 chunk_cnt= 1 mod=LSFreeze [MEMORY] hold= 544 used= 144 count= 2 avg_used= 72 block_cnt= 1 chunk_cnt= 1 mod=TntSrvObjPool [MEMORY] hold= 512 used= 120 count= 2 avg_used= 60 block_cnt= 1 chunk_cnt= 1 mod=UserResourceMgr [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 2 mod=ObLogEXTHandler [MEMORY] hold= 400 used= 208 count= 1 avg_used= 208 block_cnt= 1 chunk_cnt= 1 mod=[T]ObMacroBlock [MEMORY] hold= 304 used= 112 count= 1 avg_used= 112 block_cnt= 1 chunk_cnt= 1 mod=Coordinator [MEMORY] hold= 256 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfoMap [MEMORY] hold= 240 used= 48 count= 1 avg_used= 48 block_cnt= 1 chunk_cnt= 1 mod=[T]TSITabletTab [MEMORY] hold= 208 used= 16 count= 1 avg_used= 16 block_cnt= 1 chunk_cnt= 1 mod=logservice [MEMORY] hold= 508,385,920 used= 495,590,845 count= 5,584 avg_used= 88,751 mod=SUMMARY [2024-10-16 14:57:16.308670] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=54] [MEMORY] tenant_id= 1004 ctx_id= MEMSTORE_CTX_ID hold= 39,845,888 used= 39,534,592 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 39,534,592 used= 39,516,504 count= 19 avg_used= 2,079,816 block_cnt= 19 chunk_cnt= 19 mod=Memstore [MEMORY] hold= 39,534,592 used= 39,516,504 count= 19 avg_used= 2,079,816 mod=SUMMARY [2024-10-16 14:57:16.308680] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= TRANS_CTX_MGR_ID hold= 2,949,120 used= 2,916,352 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 2 washed_blocks= 2 washed_size= 1,245,184 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 2,916,352 used= 2,890,240 count= 4 avg_used= 722,560 block_cnt= 4 chunk_cnt= 2 mod=PartTranCtxMgr [MEMORY] hold= 2,916,352 used= 2,890,240 count= 4 avg_used= 722,560 mod=SUMMARY [2024-10-16 14:57:16.308690] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= PLAN_CACHE_CTX_ID hold= 43,933,696 used= 25,806,656 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 20 washed_blocks= 29 washed_size= 27,369,472 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 24,191,488 used= 21,916,545 count= 979 avg_used= 22,386 block_cnt= 979 chunk_cnt= 33 mod=SqlPhyPlan [MEMORY] hold= 1,164,928 used= 1,085,592 count= 114 avg_used= 9,522 block_cnt= 114 chunk_cnt= 28 mod=SqlPlanCache [MEMORY] hold= 448,256 used= 408,704 count= 206 avg_used= 1,984 block_cnt= 76 chunk_cnt= 25 mod=CreateContext [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 25,806,656 used= 23,412,441 count= 1,301 avg_used= 17,995 mod=SUMMARY [2024-10-16 14:57:16.308702] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= CO_STACK hold= 100,974,592 used= 98,574,336 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 50 washed_blocks= 53 washed_size= 5,980,160 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 98,574,336 used= 98,390,976 count= 191 avg_used= 515,136 block_cnt= 191 chunk_cnt= 51 mod=CoStack [MEMORY] hold= 98,574,336 used= 98,390,976 count= 191 avg_used= 515,136 mod=SUMMARY [2024-10-16 14:57:16.308714] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= KVSTORE_CACHE_ID hold= 79,691,776 used= 79,691,776 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 79,691,776 used= 0 count= 38 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 79,691,776 used= 0 count= 38 avg_used= 0 mod=SUMMARY [2024-10-16 14:57:16.308723] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= META_OBJ_CTX_ID hold= 37,781,504 used= 37,470,144 limit= 214,748,360 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 18 washed_blocks= 19 washed_size= 2,064,384 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 29,491,200 used= 26,236,400 count= 400 avg_used= 65,591 block_cnt= 400 chunk_cnt= 17 mod=L_TabletPool [MEMORY] hold= 7,888,896 used= 6,998,656 count= 107 avg_used= 65,408 block_cnt= 107 chunk_cnt= 18 mod=N_TabletPool [MEMORY] hold= 90,048 used= 84,768 count= 2 avg_used= 42,384 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 37,470,144 used= 33,319,824 count= 509 avg_used= 65,461 mod=SUMMARY [2024-10-16 14:57:16.308731] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= TX_CALLBACK_CTX_ID hold= 5,013,504 used= 3,267,456 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 9 washed_blocks= 59 washed_size= 15,958,016 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 3,267,456 used= 3,190,272 count= 402 avg_used= 7,936 block_cnt= 402 chunk_cnt= 10 mod=MemtableCallbac [MEMORY] hold= 3,267,456 used= 3,190,272 count= 402 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:16.308739] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1004 ctx_id= LOB_CTX_ID hold= 114,688 used= 81,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 2 washed_blocks= 2 washed_size= 4,079,616 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LobAllocator [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 81,856 used= 73,344 count= 2 avg_used= 36,672 mod=SUMMARY [2024-10-16 14:57:16.308746] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1004 ctx_id= PS_CACHE_CTX_ID hold= 3,293,184 used= 3,205,056 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 2 washed_blocks= 2 washed_size= 901,120 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 3,205,056 used= 3,097,170 count= 266 avg_used= 11,643 block_cnt= 59 chunk_cnt= 2 mod=SqlPsCache [MEMORY] hold= 3,205,056 used= 3,097,170 count= 266 avg_used= 11,643 mod=SUMMARY [2024-10-16 14:57:16.308759] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1004 ctx_id= TX_DATA_TABLE hold= 25,899,008 used= 17,334,272 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 17 washed_blocks= 52 washed_size= 28,647,424 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 11,644,672 used= 11,566,336 count= 12 avg_used= 963,861 block_cnt= 12 chunk_cnt= 10 mod=MEMTABLE_ARENA [MEMORY] hold= 5,657,088 used= 5,523,456 count= 696 avg_used= 7,936 block_cnt= 696 chunk_cnt= 16 mod=TX_DATA_SLICE [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 17,334,272 used= 17,121,536 count= 712 avg_used= 24,047 mod=SUMMARY [2024-10-16 14:57:16.308770] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= MDS_DATA_ID hold= 6,782,976 used= 4,072,128 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 8 washed_blocks= 53 washed_size= 16,285,696 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 3,958,336 used= 3,864,832 count= 487 avg_used= 7,936 block_cnt= 487 chunk_cnt= 8 mod=MdsTable [MEMORY] hold= 113,792 used= 111,104 count= 14 avg_used= 7,936 block_cnt= 14 chunk_cnt= 6 mod=TX_OP [MEMORY] hold= 4,072,128 used= 3,975,936 count= 501 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:16.308856] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5119, mem_used=10398720) [2024-10-16 14:57:16.309044] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:16.310615] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:57:16.310632] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:16.310650] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373479874, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:16.310850] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:16.310930] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA5-0-0] [lt=11] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061849029444000, latest_log_ts:{val:1729061832415320001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:16.310987] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA5-0-0] [lt=53] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:16.312817] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=12] rpc statistics(total_trans_msg_count=116, total_batch_msg_count=2) [2024-10-16 14:57:16.313888] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:16.313927] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:16.314574] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=62, avg_log_batch_cnt=0, total_group_log_size=143793, avg_group_log_size=2319) [2024-10-16 14:57:16.316342] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2138}, to_seq_no={branch:0, seq:1069}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285066} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.320397] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:16.323655] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3181}, to_seq_no={branch:0, seq:2120}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285069} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.326192] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:57:16.326205] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:57:16.326208] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:16.326841] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061849029444000, latest_log_ts:{val:1729061832415320001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:16.327848] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1729061836327552000, start_id=1729061836327552000, end_id=1729061836327552001) [2024-10-16 14:57:16.328717] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get_max_applied_scn(scn={val:1729061836217613000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204021875401}, palf_committed_end_scn_:{val:1729061836324368000, v:0}, last_check_scn_:{val:1729061836324368000, v:0}, max_applied_cb_scn_:{val:1729061836217613000, v:0}}) [2024-10-16 14:57:16.328737] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:16.328795] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3182}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285070} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.329741] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=9] worker thread began to shrink(tenant_->id()=1003, group_id_=1, token=0) [2024-10-16 14:57:16.331150] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061836330730000, start_id=1729061836330730000, end_id=1729061836330730001) [2024-10-16 14:57:16.331683] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106956][T1003_L0_G1][T1003][Y0-0000000000000000-0-0] [lt=6] worker thread exit(tenant_->id()=1003, workers_.get_size()=2) [2024-10-16 14:57:16.331697] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106956][T1003_L0_G1][T0][Y0-0000000000000000-0-0] [lt=9] tenant thread end_run(id_=1003, ret=0, thread_count_=172) [2024-10-16 14:57:16.332759] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061836331795000, start_id=1729061836331795000, end_id=1729061836331795001) [2024-10-16 14:57:16.332962] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=9] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:133533009, max_lsn:66390022573, max_scn:{val:1729061836275385000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:33216076378}, reuse_lsn:{lsn:66390022573}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:66348079533}}, last_submit_lsn:{lsn:66390021986}, last_submit_end_lsn:{lsn:66390022573}, last_submit_log_id:133533009, last_submit_log_pid:48, max_flushed_lsn:{lsn:66390021986}, max_flushed_end_lsn:{lsn:66390022573}, max_flushed_log_pid:48, committed_end_lsn:{lsn:66390022573}, last_slide_log_id:133533009, last_slide_scn:{val:1729061836275385000, v:0}, last_slide_lsn:{lsn:66390021986}, last_slide_end_lsn:{lsn:66390022573}, last_slide_log_pid:48, last_slide_log_accum_checksum:3843225904, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0de0acc530}, [StateMgr]={this:0x2b0de0ad0830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:133533010, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261740374, palf_role_change_cb:0x2b0de0ad7230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, reconfig_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, persistent_config_version:{proposal_id:48, config_seq:11}, ms_ack_list:0, resend_config_version:{proposal_id:48, config_seq:11}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494260838312, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494260838312, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0de0acdc70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261941355, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728741813401825001, v:0}, min_block_id:965, min_block_min_scn:{val:1728727703672931001, v:0}, base_lsn_for_block_gc:{lsn:66098196480}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, log_config_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, log_snapshot_meta:{version:1, base_lsn:{lsn:66098196480}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1634304}, readable_log_tail:{lsn:1634304}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66433720320}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/meta", dir_fd:128, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65470464, block_header_serialize_buf_:0x2b0de0ad29f8, flashback_version:0}, log_storage:{log_tail:{lsn:66390022573}, readable_log_tail:{lsn:66390022573}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66366615552}, min_scn:{val:1729057321384354001, v:0}, curr_block_id:989, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/log", dir_fd:134, min_block_id:965, max_block_id:990, curr_writable_block_id:989}, logical_block_size_:67104768, curr_block_writable_size_:43697747, block_header_serialize_buf_:0x2b0de0ad46b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261739251, this:0x2b0de0ad0f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:48, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:47, majority_max_lsn:{lsn:33216819320}, saved_end_lsn:{lsn:33216819320}, last_submit_prepare_req_time_us:1722494261740552, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0de0ad09b0}) [2024-10-16 14:57:16.333100] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=136] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061836294446, last_flushed_end_lsn:{lsn:66390022573}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061836293521, last_flushed_end_lsn:{lsn:66390022573}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061836293510, last_flushed_end_lsn:{lsn:66390022573}}]) [2024-10-16 14:57:16.333115] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=14] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=133533010, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061836294446, last_flushed_end_lsn:{lsn:66390022573}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061836293521, last_flushed_end_lsn:{lsn:66390022573}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061836293510, last_flushed_end_lsn:{lsn:66390022573}}]) [2024-10-16 14:57:16.334854] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C07-0-0] [lt=56][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304203248, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893595, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.335137] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2162}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285071} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.336584] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C37-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:16.336614] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C37-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061836320121001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061836320121001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:16.338640] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E00-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061836338196000, start_id=1729061836338196000, end_id=1729061836338196001) [2024-10-16 14:57:16.341036] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C09-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304209430, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893596, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.344749] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3186}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285073} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.345169] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C0B-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304213563, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893597, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.345273] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C37-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061836345257000, start_id=1729061836345257000, end_id=1729061836345257001) [2024-10-16 14:57:16.349303] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C37-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061836348624000, start_id=1729061836348624000, end_id=1729061836348624001) [2024-10-16 14:57:16.350772] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF225B-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304219164, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893598, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.351145] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4247}, to_seq_no={branch:0, seq:2128}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285074} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.352529] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.354015] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1049}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285075} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.355150] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C0D-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304223544, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893599, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.356100] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=30] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=62, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:16.356113] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:16.359736] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3142}, to_seq_no={branch:0, seq:2082}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285076} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.360803] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C12-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304229197, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893600, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.361456] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.361977] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061836361683000, start_id=1729061836361683000, end_id=1729061836361683001) [2024-10-16 14:57:16.363657] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1038}, to_seq_no={branch:0, seq:1033}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285077} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.367416] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2140}, to_seq_no={branch:0, seq:1081}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285078} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.372513] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3184}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285079} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.372801] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BDA-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:16.375971] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061836375467000, start_id=1729061836375467000, end_id=1729061836375467001) [2024-10-16 14:57:16.382744] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C14-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304251138, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893601, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.382763] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:16.382778] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=28439740, tenant_id=1002) [2024-10-16 14:57:16.382793] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:16.382814] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] TxData Memory Statistic(logstream info): (ls_id={id:1}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=1) [2024-10-16 14:57:16.382821] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=6] TxData Memory Statistic(logstream info): (ls_id={id:1001}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=1) [2024-10-16 14:57:16.382824] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=3] TxData Memory Statistic(logstream info): (ls_id={id:1002}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=1) [2024-10-16 14:57:16.382827] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=3] TxData Memory Statistic(logstream info): (ls_id={id:1003}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=1) [2024-10-16 14:57:16.382831] INFO [STORAGE] check_and_freeze_tx_data_ (ob_tenant_freezer.cpp:566) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=3] TxData Memory Statistic : (Tenant Total Memory(MB)=1024, Tenant Frozen TxData Memory(MB)=0, Tenant Active TxData Memory(MB)=4, Freeze TxData Trigger Memory(MB)=20, Total TxDataTable Hold Memory(MB)=20, Total TxDataTable Memory Limit(MB)=204) [2024-10-16 14:57:16.383340] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=15] gc stale ls task succ [2024-10-16 14:57:16.383491] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=59, avg_log_batch_cnt=0, total_group_log_size=93006, avg_group_log_size=1576) [2024-10-16 14:57:16.388306] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:16.389484] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C16-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304257878, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893602, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.395198] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061836394553000, start_id=1729061836394553000, end_id=1729061836394553001) [2024-10-16 14:57:16.398681] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:16.398702] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=16] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53593280, tenant_id=1003) [2024-10-16 14:57:16.398715] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:16.399782] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1044}, to_seq_no={branch:0, seq:1039}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285086} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.403871] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C18-0-0] [lt=46][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304272259, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893603, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.406029] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1044}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285087} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.408334] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF225D-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304276724, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893604, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.410124] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2154}, to_seq_no={branch:0, seq:1091}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285088} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.412940] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF225E-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304281333, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893605, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.414233] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF1C-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=489, avg_log_submit_to_slide_time=786) [2024-10-16 14:57:16.414522] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1071}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285089} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.428047] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2245}, to_seq_no={branch:0, seq:1193}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285093} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.430782] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C1B-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304299176, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893606, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.431806] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285094} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.434566] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C1C-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304302959, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893607, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.443104] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A66-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:16.443118] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A66-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:16.447136] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3218}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285096} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.452592] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.453001] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4C32-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=65, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:16.453014] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4C32-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=65, stat_interval=1000000, avg (count/cost)=3108, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:16.453019] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=25] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=105542, hit_cnt=65, read_cnt=65, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:16.454855] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C1E-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304323248, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893608, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.456139] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:16.456660] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:57:16.456675] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=9] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:16.456681] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:16.456686] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:16.458982] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:57:16.459130] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=145] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:16.461193] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C20-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304329586, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893609, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.461522] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.461709] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3175}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285100} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.464906] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061836464505000, start_id=1729061836464505000, end_id=1729061836464505001) [2024-10-16 14:57:16.465003] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C21-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304333397, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893610, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.465760] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=77, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=393, avg_log_submit_to_slide_time=988) [2024-10-16 14:57:16.467325] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=866, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:16.467342] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=148, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:16.470117] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3183}, to_seq_no={branch:0, seq:1069}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285102} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.471447] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061836470872000, start_id=1729061836470872000, end_id=1729061836470872001) [2024-10-16 14:57:16.473309] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F26-0-0] [lt=38] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:16.473322] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F26-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=26484, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:16.473321] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:16.473326] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F26-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=407, avg_log_submit_to_slide_time=26479) [2024-10-16 14:57:16.480627] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C26-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304349021, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893611, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.483834] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061836482500000, start_id=1729061836482500000, end_id=1729061836482500001) [2024-10-16 14:57:16.484822] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3624}, to_seq_no={branch:0, seq:2141}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285105} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.485780] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=40] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:16.488355] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C27-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304356740, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893612, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.489851] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285107} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.494405] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285109} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.496485] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061836495686000, start_id=1729061836495686000, end_id=1729061836495686001) [2024-10-16 14:57:16.502520] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1179615232, global_cache_size=1589706752, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:16.504245] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3179}, to_seq_no={branch:0, seq:1049}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285111} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.505284] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C29-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304373676, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893613, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.507130] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C2A-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304375524, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893614, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.511963] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061836511707000, start_id=1729061836511707000, end_id=1729061836511707001) [2024-10-16 14:57:16.512137] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=18] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061836512131], gts:1729061836511707000, latest_srr:[mts=1729061836512131]}) [2024-10-16 14:57:16.513953] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1072}, to_seq_no={branch:0, seq:1067}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285114} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.518652] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C2C-0-0] [lt=36][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304387046, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893615, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.522241] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C2E-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304390635, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893616, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.523776] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C30-0-0] [lt=31][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304392171, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893617, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.526523] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2041}, to_seq_no={branch:0, seq:1027}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285116} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.528031] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=41] get number(ret=0, service_type_=0, range=1, base_id=1729061836527643000, start_id=1729061836527643000, end_id=1729061836527643001) [2024-10-16 14:57:16.531319] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061836529795000, start_id=1729061836529795000, end_id=1729061836529795001) [2024-10-16 14:57:16.531433] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C31-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304399828, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893618, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.531462] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:16.531474] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4382) [2024-10-16 14:57:16.531488] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1440780364, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:16.532810] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061836532579000, start_id=1729061836532579000, end_id=1729061836532579001) [2024-10-16 14:57:16.535265] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C33-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304403660, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893619, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.539096] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C35-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304407490, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893620, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.546737] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C37-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304415131, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893621, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.553588] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D54-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:16.553619] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D54-0-0] [lt=22] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061836121545003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061836121545003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:16.553684] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.554775] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D54-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061836554143000, start_id=1729061836554143000, end_id=1729061836554143001) [2024-10-16 14:57:16.554920] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5024, dropped:138, tid:12153}]) [2024-10-16 14:57:16.555697] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C39-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304424091, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893622, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.555787] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3AEC-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=680, avg_log_submit_to_slide_time=852) [2024-10-16 14:57:16.559762] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D54-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061836558563000, start_id=1729061836558563000, end_id=1729061836558563001) [2024-10-16 14:57:16.561586] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.561854] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C3B-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304430249, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893623, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.564260] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:16.564276] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:16.567875] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22697][T1004_TntTransf][T1004][YB4A0A000407-00061E9ABD5C8934-0-0] [lt=11] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:57:16.568544] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.568620] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.568624] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.569082] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.573441] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C3D-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304441835, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893624, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.573709] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:16.576065] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.576079] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.576099] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061836575303000, start_id=1729061836575303000, end_id=1729061836575303001) [2024-10-16 14:57:16.576085] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046220505088},{"BIGINT":1729058400},{"VARCHAR":"?Ô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.576139] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.576643] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2139}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285128} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.578389] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1003}, {id:1002}]) [2024-10-16 14:57:16.578874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061836578478000, start_id=1729061836578478000, end_id=1729061836578478001) [2024-10-16 14:57:16.580196] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.580208] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.580214] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046245670912},{"BIGINT":1729058400},{"VARCHAR":"B$", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.580273] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.580462] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1076}, to_seq_no={branch:0, seq:1071}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285129} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.586280] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C3F-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304454674, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893625, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.588384] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:16.589942] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C41-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304458336, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893626, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.590968] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.590984] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.590990] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046472163328},{"BIGINT":1729058400},{"VARCHAR":"?à", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.591047] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.592347] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3209}, to_seq_no={branch:0, seq:1080}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285131} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.594900] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.594916] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.594922] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046522494976},{"BIGINT":1729058400},{"VARCHAR":"B", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.594979] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.595139] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285132} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.595632] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4459, mem_used=8318976) [2024-10-16 14:57:16.598235] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061836597243000, start_id=1729061836597243000, end_id=1729061836597243001) [2024-10-16 14:57:16.599776] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C43-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304468169, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893627, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.605801] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106946][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4C4A-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=103922) [2024-10-16 14:57:16.605850] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=102298, hit_cnt=63, read_cnt=63, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:16.613992] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=64, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:16.614238] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=64, stat_interval=1000000, avg (count/cost)=267, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:16.614256] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=64, stat_interval=1000000, avg (count/cost)=160, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:16.614279] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=64, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:16.618402] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C45-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304486795, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893628, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.622107] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B5B-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061836621458000, start_id=1729061836621458000, end_id=1729061836621458001) [2024-10-16 14:57:16.622126] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B5F-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061836621458000, start_id=1729061836621458000, end_id=1729061836621458001) [2024-10-16 14:57:16.622165] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B5C-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061836621458000, start_id=1729061836621458000, end_id=1729061836621458001) [2024-10-16 14:57:16.624042] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C47-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304492435, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893629, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.635775] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C49-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304504169, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893630, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.638550] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.638571] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.638578] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046824484864},{"BIGINT":1729058400},{"VARCHAR":"BÖ*Œ)", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.638649] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.639663] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3169}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285138} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.649022] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:16.652937] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C4C-0-0] [lt=28][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304521329, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893631, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.653743] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.655515] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.655528] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.655535] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047088726016},{"BIGINT":1729058400},{"VARCHAR":"Ac×€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.655588] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.655908] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2180}, to_seq_no={branch:0, seq:1116}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285142} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.656033] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C4E-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304524428, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893632, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.656158] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:57:16.656170] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:57:16.656174] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:16.656877] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833508923001, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850098662000, latest_log_ts:{val:1729061833508923001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:16.657642] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=46] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833508923001, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:16.658358] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=46] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833508923001, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:16.659104] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=78] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061833508923001, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:16.659277] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A5F-0-0] [lt=8][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178206, log_task={header:{begin_lsn:{lsn:14538457673}, end_lsn:{lsn:14538457795}, log_id:115178206, min_scn:{val:1729061836121545004, v:0}, max_scn:{val:1729061836121545004, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538457551}, prev_proposal_id:23, committed_end_lsn:{lsn:14538457673}, data_checksum:1715074492, accum_checksum:521962205, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061836557399, freeze_ts:1729061836557399, submit_ts:1729061836557402, flushed_ts:1729061836557688, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:286}, fs_cb_begin_ts=1729061836659271, log_life_time=101872) [2024-10-16 14:57:16.659711] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=227, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:16.659729] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=48, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:16.659770] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:16.661654] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.662948] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C50-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304531342, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893633, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.667249] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:16.667267] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3658) [2024-10-16 14:57:16.667276] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:16.667633] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3658) [2024-10-16 14:57:16.667793] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C55-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304536187, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893634, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.667938] INFO [LIB] stat (utility.h:1165) [106635][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F1B-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:16.667947] INFO [LIB] stat (utility.h:1165) [106635][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F1B-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=2201, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:16.669439] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C57-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304537834, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893635, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.672332] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=14] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061832653917, last_first_register_time_us=1722494150998907, register_time_us=1729061832653917, curr_time_us=1729061836672331) [2024-10-16 14:57:16.672362] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061836672348, last_submit_register_req_time_us=1729061836672348, register_parent_reason=3) [2024-10-16 14:57:16.672370] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:16.673035] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C59-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304541430, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893636, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.673188] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BE0-0-0] [lt=22][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172899, log_task={header:{begin_lsn:{lsn:14352343918}, end_lsn:{lsn:14352344040}, log_id:115172899, min_scn:{val:1729061836121545005, v:0}, max_scn:{val:1729061836121545005, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352343796}, prev_proposal_id:17, committed_end_lsn:{lsn:14352343918}, data_checksum:940009244, accum_checksum:344976390, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061836572989, freeze_ts:1729061836572989, submit_ts:1729061836572992, flushed_ts:1729061836573734, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:742}, fs_cb_begin_ts=1729061836673181, log_life_time=100192) [2024-10-16 14:57:16.673688] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=248, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:16.673708] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=237, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:16.673748] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:16.676476] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=41] worker thread began to shrink(tenant_->id()=1, group_id_=1, token=0) [2024-10-16 14:57:16.677169] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2264-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304545563, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893637, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.678035] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106635][T1_L0_G1][T1][Y0-0000000000000000-0-0] [lt=3] worker thread exit(tenant_->id()=1, workers_.get_size()=2) [2024-10-16 14:57:16.678051] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106635][T1_L0_G1][T0][Y0-0000000000000000-0-0] [lt=12] tenant thread end_run(id_=1, ret=0, thread_count_=240) [2024-10-16 14:57:16.684017] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061836683347000, start_id=1729061836683347000, end_id=1729061836683347001) [2024-10-16 14:57:16.684504] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C5C-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304552897, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893638, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.684506] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4132) [2024-10-16 14:57:16.687755] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C5E-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304556150, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893639, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.689150] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C60-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304557546, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893640, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.690555] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C62-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304558952, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893641, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.694111] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C64-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304562506, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893642, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.695154] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=21] gc stale ls task succ [2024-10-16 14:57:16.696841] INFO [STORAGE] runTimerTask (ob_locality_manager.cpp:708) [21260][LocalityReload][T0][Y0-0000000000000000-0-0] [lt=10] runTimer to refresh locality_info(ret=0) [2024-10-16 14:57:16.697739] INFO [SHARE] set_server_locality_array (ob_server_locality_cache.cpp:346) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=11] set_server_locality_array success(server_locality_array=[{inited:true, addr:"10.0.4.7:2890", zone:"myoceanbase1", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494256558000, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.8:2890", zone:"myoceanbase2", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494430723273, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.9:2890", zone:"myoceanbase3", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494593087951, server_stop_time:0, server_status:1}], has_readonly_zone=false) [2024-10-16 14:57:16.697777] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=32] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1, primary_zone=RANDOM) [2024-10-16 14:57:16.697783] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=6] add_locality_zone success(locality_zone={tenant_id:1, region_priority:18446744073709551615}) [2024-10-16 14:57:16.697792] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=5] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1001, primary_zone=RANDOM) [2024-10-16 14:57:16.697799] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=6] add_locality_zone success(locality_zone={tenant_id:1001, region_priority:18446744073709551615}) [2024-10-16 14:57:16.697805] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1002, primary_zone=RANDOM) [2024-10-16 14:57:16.697811] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=5] add_locality_zone success(locality_zone={tenant_id:1002, region_priority:18446744073709551615}) [2024-10-16 14:57:16.697816] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1003, primary_zone=RANDOM) [2024-10-16 14:57:16.697821] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=5] add_locality_zone success(locality_zone={tenant_id:1003, region_priority:18446744073709551615}) [2024-10-16 14:57:16.697825] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1004, primary_zone=RANDOM) [2024-10-16 14:57:16.697831] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=5] add_locality_zone success(locality_zone={tenant_id:1004, region_priority:18446744073709551615}) [2024-10-16 14:57:16.697835] INFO [SHARE] load_region (ob_locality_table_operator.cpp:373) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] load region(ret=0, locality_info={version:0, local_region:"sys_region", local_zone:"myoceanbase1", local_idc:"", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["myoceanbase1", "myoceanbase2", "myoceanbase3"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:18446744073709551615}, {tenant_id:1002, region_priority:18446744073709551615}, {tenant_id:1003, region_priority:18446744073709551615}, {tenant_id:1004, region_priority:18446744073709551615}]}) [2024-10-16 14:57:16.700797] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061836699740000, start_id=1729061836699740000, end_id=1729061836699740001) [2024-10-16 14:57:16.702868] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1175420928, global_cache_size=1593868288, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:16.703848] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.703865] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.703873] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047566876672},{"BIGINT":1729058400},{"VARCHAR":"AÆ­", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.703942] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.704202] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1300}, to_seq_no={branch:0, seq:1295}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285152} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.705543] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C66-0-0] [lt=9][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304573929, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893643, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.705990] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061836705383000, start_id=1729061836705383000, end_id=1729061836705383001) [2024-10-16 14:57:16.706101] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:142) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16] transaction rpc statistics(total_rt=300821, total_process=3566, avg_rt=84) [2024-10-16 14:57:16.706490] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=78, stat_interval=1000000, avg (count/cost)=271, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:16.706508] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=78, stat_interval=1000000, avg (count/cost)=108, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:16.706532] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=30] [PALF STAT FLUSH CB COST TIME](cur_stat_count=78, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:16.707624] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C68-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304576017, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893644, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.710030] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.710047] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.710053] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047638179841},{"BIGINT":1729058400},{"VARCHAR":"B1ÕÅ75", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.710123] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.710475] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285154} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.710565] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:16.713029] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061836712878000, start_id=1729061836712878000, end_id=1729061836712878001) [2024-10-16 14:57:16.713939] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:16.713957] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:16.714870] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:16.715045] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=17][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:16.715594] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.715620] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.715627] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047667539968},{"BIGINT":1729058400},{"VARCHAR":"B&ÍNj", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.715699] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.718177] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C6A-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304586557, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893645, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.718493] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4271}, to_seq_no={branch:0, seq:1082}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285155} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.719373] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15CA-0-0] [lt=7] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:16.720242] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:16.726966] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.726982] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.726989] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047839506433},{"BIGINT":1729058400},{"VARCHAR":"A„»×€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.727039] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.728048] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2265-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304596442, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893646, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.728052] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061836726742000, start_id=1729061836726742000, end_id=1729061836726742001) [2024-10-16 14:57:16.728253] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3505}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285157} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.731450] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061836731287000, start_id=1729061836731287000, end_id=1729061836731287001) [2024-10-16 14:57:16.732827] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061836732376000, start_id=1729061836732376000, end_id=1729061836732376001) [2024-10-16 14:57:16.733974] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C6D-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304602368, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893647, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.734950] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.734966] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.734972] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047982112768},{"BIGINT":1729058400},{"VARCHAR":"AWþòÀ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.735022] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.735340] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2126}, to_seq_no={branch:0, seq:2121}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285158} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.744284] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.744304] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.744311] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048183439360},{"BIGINT":1729058400},{"VARCHAR":"B,IbÛ,", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.744373] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.745419] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3175}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285161} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.747056] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061832740715, last_first_register_time_us=1722494157071428, register_time_us=1729061832740715, curr_time_us=1729061836747055) [2024-10-16 14:57:16.747098] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=23] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061836747080, last_submit_register_req_time_us=1729061836747080, register_parent_reason=3) [2024-10-16 14:57:16.747106] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:16.747641] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257654, offset=39002548}, end_cursor:ObLogCursor{file_id=197, log_id=49257655, offset=39002831}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b54050, len:3660, log_data_len:283, seq:49257654, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:16.748531] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:16.748553] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=10] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:16.748559] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=5] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:16.750028] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB356-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061836749326000, start_id=1729061836749326000, end_id=1729061836749326001) [2024-10-16 14:57:16.750968] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.750980] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.750987] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048221188096},{"BIGINT":1729058400},{"VARCHAR":"@x@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.751045] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=35][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.751095] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257655, offset=39002831}, end_cursor:ObLogCursor{file_id=197, log_id=49257656, offset=39003113}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2afc050, len:3377, log_data_len:282, seq:49257655, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:16.751379] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3221}, to_seq_no={branch:0, seq:2126}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285162} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.753827] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.754062] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061836753565000, start_id=1729061836753565000, end_id=1729061836753565001) [2024-10-16 14:57:16.755109] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA867-0-0] [lt=9] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:16.755132] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA867-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061836738668000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061836738668000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:16.755162] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C6F-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304623555, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893648, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.755793] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257656, offset=39003113}, end_cursor:ObLogCursor{file_id=197, log_id=49257657, offset=39003395}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29b8050, len:3095, log_data_len:282, seq:49257656, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:16.756205] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A61-0-0] [lt=22] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:16.756476] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA867-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061836755703000, start_id=1729061836755703000, end_id=1729061836755703001) [2024-10-16 14:57:16.757185] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C71-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304625580, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893649, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.757379] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62649967, miss_cnt:32, cache_read_size:8050387002, hit_ratio:"9.999994892258498025e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:16.757623] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257657, offset=39003395}, end_cursor:ObLogCursor{file_id=197, log_id=49257658, offset=39003677}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29cc050, len:2813, log_data_len:282, seq:49257657, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:16.758284] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656702, miss_cnt:17, cache_read_size:8050134383, hit_ratio:"9.999997286803351537e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:16.760795] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=12] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:16.760918] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA867-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061836759933000, start_id=1729061836759933000, end_id=1729061836759933001) [2024-10-16 14:57:16.760975] INFO [LIB] stat (utility.h:1165) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA867-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=21, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:16.760987] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA867-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=10912, avg_group_log_size=545) [2024-10-16 14:57:16.760996] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA867-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10912) [2024-10-16 14:57:16.761003] INFO [LIB] stat (utility.h:1165) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA867-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:16.761216] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=246, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:16.761225] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10912) [2024-10-16 14:57:16.761234] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=47, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:16.761253] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10912) [2024-10-16 14:57:16.761274] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:16.761552] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5677-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:16.761562] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5677-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=733, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:16.761566] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5677-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=433, avg_log_submit_to_slide_time=704) [2024-10-16 14:57:16.761621] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=5] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:16.761630] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=41, this=0x2b0e07718280, extra_info="") [2024-10-16 14:57:16.761633] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=1771, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:57:16.761637] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:57:16.761640] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=2] [APPLY STAT CB EXECUTE TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=-487, this=0x2b0e07718910, extra_info="") [2024-10-16 14:57:16.761643] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=2] [APPLY STAT CB TOTAL TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=1325, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:57:16.761698] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=42][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.763370] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601288, offset=9597524}, end_cursor:ObLogCursor{file_id=192, log_id=47601289, offset=9597808}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b80050, len:3500, log_data_len:284, seq:47601288, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:16.763846] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBCB-0-0] [lt=18] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061836731655, keepalive_ts:-1}) [2024-10-16 14:57:16.763861] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBCB-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061836731655, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:16.766046] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48B-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061836765212000, start_id=1729061836765212000, end_id=1729061836765212001) [2024-10-16 14:57:16.766179] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137597999, miss_cnt:9, cache_read_size:149776693813, hit_ratio:"9.999999345920763671e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:16.766995] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601289, offset=9597808}, end_cursor:ObLogCursor{file_id=192, log_id=47601290, offset=9598082}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd298c050, len:3216, log_data_len:274, seq:47601289, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:16.767631] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=69, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:16.768045] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560402, miss_cnt:1, cache_read_size:152589431490, hit_ratio:"9.999999928346438027e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:16.768088] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:16.770001] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601290, offset=9598082}, end_cursor:ObLogCursor{file_id=192, log_id=47601291, offset=9598356}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b64050, len:2942, log_data_len:274, seq:47601290, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:16.770205] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601291, offset=9598356}, end_cursor:ObLogCursor{file_id=192, log_id=47601292, offset=9598630}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29f8050, len:2668, log_data_len:274, seq:47601291, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:16.776101] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061836774868000, start_id=1729061836774868000, end_id=1729061836774868001) [2024-10-16 14:57:16.779146] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=14] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:16.780619] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C73-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304649013, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893650, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.782338] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C75-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304650733, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893651, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.788377] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.788396] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.788465] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:16.788615] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.788899] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.789374] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C77-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304657768, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893652, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.789428] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.789451] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.789459] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.789471] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.789485] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.789500] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.789658] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:16.797243] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C79-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304665635, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893653, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.800632] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061836800553000, start_id=1729061836800553000, end_id=1729061836800553001) [2024-10-16 14:57:16.801836] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC583-0-0] [lt=26] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:57:16.802553] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.802565] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.802571] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048707727361},{"BIGINT":1729058400},{"VARCHAR":"AG%", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.802645] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=44][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.802646] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC583-0-0] [lt=15] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1002) [2024-10-16 14:57:16.802656] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC583-0-0] [lt=8] no transfer partition job [2024-10-16 14:57:16.802662] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC583-0-0] [lt=4] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:57:16.803917] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC583-0-0] [lt=7] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1002, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:57:16.803939] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC583-0-0] [lt=15] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:57:16.804204] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3385}, to_seq_no={branch:0, seq:1083}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285173} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.805260] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:500) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC583-0-0] [lt=13][errcode=-4152] get obj error(ret=-4152) [2024-10-16 14:57:16.805301] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC583-0-0] [lt=10] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1001, status:0}], ls_array_=[{tenant_id:1002, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729057174083520, last_statistic_bg_stat_time=1729008957751717, last_statistic_schema_version=1729008953019688, last_statistic_max_transfer_task_id={id:-1}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:57:16.805561] INFO [LIB] stat (utility.h:1165) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=80, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:16.805577] INFO [LIB] stat (utility.h:1165) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=80, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:16.805956] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=128392) [2024-10-16 14:57:16.805982] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=128392) [2024-10-16 14:57:16.806421] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=10] [APPLY STAT CB APPEND COST TIME](cur_stat_count=71, stat_interval=5000000, avg (count/cost)=37, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:57:16.806432] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=71, stat_interval=5000000, avg (count/cost)=1278, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:57:16.806436] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=71, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:57:16.806439] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=71, stat_interval=5000000, avg (count/cost)=-529, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:57:16.806442] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=71, stat_interval=5000000, avg (count/cost)=787, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:57:16.808453] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C7B-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304676848, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893654, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.808578] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.808594] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.808607] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048749670400},{"BIGINT":1729058400},{"VARCHAR":"AÄ@ýW€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.808659] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.808889] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2187}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285174} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.811594] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.811616] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.811622] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048787419136},{"BIGINT":1729058400},{"VARCHAR":"@Õk@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.811669] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.811849] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285175} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.814293] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=189, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:16.814314] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=96, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:16.814665] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061836814645000, start_id=1729061836814645000, end_id=1729061836814645001) [2024-10-16 14:57:16.815307] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1001}, {id:1003}]) [2024-10-16 14:57:16.820216] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C7D-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304688609, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893655, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.820999] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4C6B-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=66, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:16.821011] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4C6B-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=66, stat_interval=1000000, avg (count/cost)=1826, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:16.825822] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C7F-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304694216, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893656, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.827297] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C81-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304695692, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893657, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.829124] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C83-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304697520, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893658, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.832414] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C85-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304700810, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893659, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.833921] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C87-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304702317, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893660, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.835449] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C89-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304703845, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893661, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.836873] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C8B-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304705269, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893662, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.847962] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C8D-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304716355, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893663, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.849210] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2881, mem_used=6239232) [2024-10-16 14:57:16.851498] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.851515] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.851526] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048913248256},{"BIGINT":1729058400},{"VARCHAR":"Ad]m ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.851594] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.851824] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2118}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285180} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.853898] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.855801] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E95-0-0] [lt=17] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061836823198, keepalive_ts:-1}) [2024-10-16 14:57:16.855823] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E95-0-0] [lt=21] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061836823198, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:16.855846] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E96-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061836823241, keepalive_ts:-1}) [2024-10-16 14:57:16.855851] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E96-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061836823241, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:16.856304] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A63-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:16.856325] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A63-0-0] [lt=13][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178208, log_task={header:{begin_lsn:{lsn:14538457917}, end_lsn:{lsn:14538458039}, log_id:115178208, min_scn:{val:1729061836621458001, v:0}, max_scn:{val:1729061836621458001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538457795}, prev_proposal_id:23, committed_end_lsn:{lsn:14538457917}, data_checksum:1715074492, accum_checksum:3686197855, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061836756196, freeze_ts:1729061836756196, submit_ts:1729061836756200, flushed_ts:1729061836758459, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:2259}, fs_cb_begin_ts=1729061836856319, log_life_time=100123) [2024-10-16 14:57:16.856726] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=65, stat_interval=1000000, avg (count/cost)=276, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:16.856743] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=66, stat_interval=1000000, avg (count/cost)=156, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:16.856770] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=66, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:16.857662] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.857675] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.857681] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048963579904},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.857741] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.859090] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22703][T1004_TntShared][T1004][Y0-0000000000000000-0-0] [lt=12] clear dtl interm result cost(us)(clear_cost=907, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:16.859249] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2160}, to_seq_no={branch:0, seq:1104}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285182} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.860176] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=4] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:16.860783] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:16.860843] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=78, avg_log_batch_cnt=1, total_group_log_size=125144, avg_group_log_size=1604) [2024-10-16 14:57:16.860855] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=125144) [2024-10-16 14:57:16.860829] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061827226807000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:16.861749] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.862218] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.862232] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.862238] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049001328640},{"BIGINT":1729058400},{"VARCHAR":"AoÝB ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.862289] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.862471] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2127}, to_seq_no={branch:0, seq:1071}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285183} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.865303] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=11][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:16.868884] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C95-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304737278, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893664, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.874959] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C97-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304743353, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893665, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.876723] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C99-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304745118, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893666, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.878391] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C9B-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304746788, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893667, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.882827] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:16.883151] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061836883150, bytes: 0, bw: 0.000000 MB/s, add_ts: 1006831, add_bytes: 0 [2024-10-16 14:57:16.884124] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061836883819000, start_id=1729061836883819000, end_id=1729061836883819001) [2024-10-16 14:57:16.885787] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=21] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061836511707000, v:0}, new_max_version={val:1729061836511707000, v:0}, last_min_version={val:1729061836275385000, v:0}, last_max_version={val:1729061836275385000, v:0}, total_time_us=2510, delta_us=374079) [2024-10-16 14:57:16.885803] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061836511707000, v:0}, min_version={val:1729061836511707000, v:0}, max_version={val:1729061836511707000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:16.885831] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C9D-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304754225, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893668, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.889416] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=25] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 151.00; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 12498.05, iops: 38.80, rt: 450.44; ) [2024-10-16 14:57:16.889449] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=30] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:16.889494] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 481.25; ) [2024-10-16 14:57:16.889500] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:16.889508] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:16.893616] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0C9F-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304762010, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893669, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.894894] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CA1-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304763290, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893670, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.899045] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.899058] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.899065] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049223626753},{"BIGINT":1729058400},{"VARCHAR":"A", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.899127] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=35][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.899212] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CA3-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304767608, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893671, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.899466] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285190} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.901373] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2266-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304769766, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893672, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.901820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061836901628000, start_id=1729061836901628000, end_id=1729061836901628001) [2024-10-16 14:57:16.903154] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1177518080, global_cache_size=1593868288, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:16.907576] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CA6-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304775970, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893673, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.911310] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CA8-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304779704, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893674, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.919304] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:16.919914] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:16.919933] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=208) [2024-10-16 14:57:16.920802] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DD4-0-0] [lt=12] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:57:16.921566] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CAA-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304789960, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893675, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.922760] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=137, stat_interval=1000000, avg (count/cost)=33, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:16.922776] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=137, stat_interval=1000000, avg (count/cost)=98, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:16.923173] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.923188] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.923202] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049429147648},{"BIGINT":1729058400},{"VARCHAR":"B*+ÁÀ&", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.923265] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.923474] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1058}, to_seq_no={branch:0, seq:1053}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285197} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.924735] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061836923820000, start_id=1729061836923820000, end_id=1729061836923820001) [2024-10-16 14:57:16.925817] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.925828] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.925834] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049466896384},{"BIGINT":1729058400},{"VARCHAR":"Bb¨k", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.925888] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.926183] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285198} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.927726] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CAC-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304796120, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893676, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.928770] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061836928348000, start_id=1729061836928348000, end_id=1729061836928348001) [2024-10-16 14:57:16.931507] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061836930461000, start_id=1729061836930461000, end_id=1729061836930461001) [2024-10-16 14:57:16.931522] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CAE-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304799917, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893677, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.933374] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061836932687000, start_id=1729061836932687000, end_id=1729061836932687001) [2024-10-16 14:57:16.935927] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CB0-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304804322, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893678, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.936839] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1026, mem_used=2079744) [2024-10-16 14:57:16.937455] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.937466] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.937473] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049567559680},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.937524] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.937827] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285201} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.940797] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:16.940811] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=28047760, tenant_id=1004) [2024-10-16 14:57:16.940820] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:16.941752] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.941765] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.941771] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049592725504},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.941830] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.942369] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2155}, to_seq_no={branch:0, seq:1096}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285202} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.945819] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CB2-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304814213, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893679, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.948336] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] [COMPACTION DAG STATUS] (start_time=1729061826946937, end_time=1729061836948334, dag_status={}) [2024-10-16 14:57:16.948354] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=17] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.948362] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.948366] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.948370] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.948373] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.948376] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.948380] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.948383] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.948386] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:16.948391] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948398] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948402] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948406] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948410] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948414] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948418] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948421] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948425] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948429] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948433] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948437] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948441] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948445] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948449] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948452] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948456] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948460] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948464] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948467] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948471] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948475] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948479] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948482] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948486] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948490] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948494] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948497] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948501] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948505] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948509] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948513] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948516] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948520] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948524] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948528] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948531] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948535] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948539] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948543] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948546] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948550] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948554] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948557] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948561] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948565] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948569] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948573] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948576] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948580] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948583] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948587] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948596] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948605] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948609] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948613] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948617] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948620] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:16.948625] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:16.948628] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:16.948631] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:16.948634] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:16.948637] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:16.948639] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:16.948641] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:16.948644] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:16.948646] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:16.948648] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:16.948651] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:16.949016] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=11] ====== tenant freeze timer task ====== [2024-10-16 14:57:16.951305] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061836951303, bytes: 4446789761302, bw: 1.885806 MB/s, add_ts: 1001395, add_bytes: 1980169 [2024-10-16 14:57:16.953941] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:16.953963] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=21] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:16.953970] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.955128] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CB4-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304823522, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893680, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.955318] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFB9-0-0] [lt=5] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:16.955339] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFB9-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:16.956893] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CB6-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304825231, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893681, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.956906] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A65-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:16.961815] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:16.965565] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=39][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.965583] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.965590] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049731137536},{"BIGINT":1729058400},{"VARCHAR":"BòBN", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.965653] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.965927] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2209}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285206} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.966449] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CB8-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304834842, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893682, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:16.976398] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061836974679000, start_id=1729061836974679000, end_id=1729061836974679001) [2024-10-16 14:57:16.977249] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.977262] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.977269] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049794052096},{"BIGINT":1729058400},{"VARCHAR":"Byƒ ÇÆð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.977317] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.978353] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4600}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285208} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.982188] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.982205] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.982211] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049819217920},{"BIGINT":1729058400},{"VARCHAR":"Aæ>œ ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.982271] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.982470] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285209} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.988519] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:16.994371] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=93][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.994388] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.994395] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049919881216},{"BIGINT":1729058400},{"VARCHAR":"B¨¥),J", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.994458] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.994768] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285212} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:16.998814] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:16.998828] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:16.998834] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049957629952},{"BIGINT":1729058400},{"VARCHAR":"AÂ/", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:16.998891] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:16.999843] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2267-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304868237, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893683, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.000204] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3209}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285213} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.003398] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CBB-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304871792, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893684, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.003735] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061837002701000, start_id=1729061837002701000, end_id=1729061837002701001) [2024-10-16 14:57:17.004220] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.004236] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.004242] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830050007961600},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.004299] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.004514] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2163}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285214} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.008939] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=8] Garbage Collector is running(seq_=656696, gc_interval=10000000) [2024-10-16 14:57:17.008968] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=12] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:17.008979] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=29) [2024-10-16 14:57:17.008985] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:17.009651] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:17.009664] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:17.009675] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=9] execute_gc cost time(ret=0, time_us=10) [2024-10-16 14:57:17.009679] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:17.012354] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061837012046000, start_id=1729061837012046000, end_id=1729061837012046001) [2024-10-16 14:57:17.012376] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061837012350], gts:1729061837012046000, latest_srr:[mts=1729061837012350]}) [2024-10-16 14:57:17.012396] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:17.014111] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:17.014142] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:17.014532] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:17.014553] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:17.014564] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:17.019785] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CC0-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304888179, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893685, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.023142] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CC2-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304891536, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893686, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.024692] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CC4-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304893088, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893687, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.029662] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CC6-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304898057, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893688, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.036285] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:17.036302] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:17.036317] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=291828220, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:17.044750] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.044769] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.044776] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830050381254657},{"BIGINT":1729058400},{"VARCHAR":"AÌÖ8", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.044844] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.045047] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2357}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285223} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.047801] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=5] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:17.048395] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:17.048443] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061830183137000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:17.050878] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061837050169000, start_id=1729061837050169000, end_id=1729061837050169001) [2024-10-16 14:57:17.054003] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.054490] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CC8-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304922872, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893689, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.057104] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=37] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:17.057134] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT APPEND COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:17.057335] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=290, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:17.057352] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=845) [2024-10-16 14:57:17.057362] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=426, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:17.057373] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=845) [2024-10-16 14:57:17.057385] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:17.057684] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3935-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:17.057691] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3935-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=963, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:17.060261] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CCA-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304928654, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893690, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.061862] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.069476] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.069493] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.069499] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830050532249601},{"BIGINT":1729058400},{"VARCHAR":"B?ÌÕ-º", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.069566] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.069885] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3202}, to_seq_no={branch:0, seq:2141}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285228} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.073926] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.073945] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.073952] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830050561609728},{"BIGINT":1729058400},{"VARCHAR":"B.eòµ*", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.074013] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=36][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.074269] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1153}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285229} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.075293] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CCC-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304943686, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893691, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.077150] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.077166] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.077172] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830050586775552},{"BIGINT":1729058400},{"VARCHAR":"A”/T", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.077223] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.077476] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1031}, to_seq_no={branch:0, seq:1026}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285230} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.080030] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061837079794000, start_id=1729061837079794000, end_id=1729061837079794001) [2024-10-16 14:57:17.081100] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CCD-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304949494, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893692, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.082788] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CCF-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304951182, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893693, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.083322] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=11] ====== tenant freeze timer task ====== [2024-10-16 14:57:17.083336] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=149477360, tenant_id=1) [2024-10-16 14:57:17.083340] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.083348] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:17.083349] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.083355] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830050611941376},{"BIGINT":1729058400},{"VARCHAR":"A”/d", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.083407] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.083920] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3050}, to_seq_no={branch:0, seq:3044}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285231} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.084402] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061837082838000, start_id=1729061837082838000, end_id=1729061837082838001) [2024-10-16 14:57:17.085667] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CD1-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304954063, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893694, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.093581] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CD3-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304961974, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893695, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.100970] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CD5-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304969364, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893696, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.102616] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CD7-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304970996, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893697, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.102847] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061837102841000, start_id=1729061837102841000, end_id=1729061837102841001) [2024-10-16 14:57:17.103210] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.103224] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.103231] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830050863599616},{"BIGINT":1729058400},{"VARCHAR":"@` ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.103296] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.103447] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1173323776, global_cache_size=1595949056, tenant_max_wash_size=7525990, tenant_min_wash_size=7525990, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:17.103736] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285236} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.109839] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:17.109856] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:17.113062] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21675][T1_L0_G0][T1][YB4A0A000407-00061E99798D0075-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304981453, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410505, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.113121] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:17.114310] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:17.114344] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=32] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:17.114374] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:17.114379] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:17.114543] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=268, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:17.114560] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=102, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:17.114578] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:17.114798] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF31-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:17.114821] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF31-0-0] [lt=23] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=668, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:17.115296] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF32-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:17.115304] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF32-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=710, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:17.115321] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF32-0-0] [lt=18] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=384, avg_log_submit_to_slide_time=695) [2024-10-16 14:57:17.116506] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.116519] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.116526] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830050955874306},{"BIGINT":1729058400},{"VARCHAR":"A…V¶˜", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.116587] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.116825] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1187}, to_seq_no={branch:0, seq:1182}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285239} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.117511] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CD9-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304985905, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893698, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.118323] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AB-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:17.118348] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AB-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061836622485004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061836622485004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:17.119256] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CDB-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304987650, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893699, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.120400] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.120413] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.120420] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830050985234433},{"BIGINT":1729058400},{"VARCHAR":"@ÿ%€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.120472] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.120578] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9AB-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061837119764000, start_id=1729061837119764000, end_id=1729061837119764001) [2024-10-16 14:57:17.121806] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2118}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285240} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.122074] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B61-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061837121884000, start_id=1729061837121884000, end_id=1729061837121884001) [2024-10-16 14:57:17.122454] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B64-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061837121884000, start_id=1729061837121884000, end_id=1729061837121884001) [2024-10-16 14:57:17.122888] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B60-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061837121884000, start_id=1729061837121884000, end_id=1729061837121884001) [2024-10-16 14:57:17.122917] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:17.123381] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AB-0-0] [lt=35] get number(ret=0, service_type_=0, range=1, base_id=1729061837123002000, start_id=1729061837123002002, end_id=1729061837123002003) [2024-10-16 14:57:17.123447] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AB-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10327, avg_group_log_size=573) [2024-10-16 14:57:17.123458] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AB-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10327) [2024-10-16 14:57:17.123738] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=246, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:17.123755] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10327) [2024-10-16 14:57:17.123768] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=206, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:17.123789] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10327) [2024-10-16 14:57:17.123799] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:17.123959] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797A9C-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:17.123966] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797A9C-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=894, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:17.123970] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797A9C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=528, avg_log_submit_to_slide_time=866) [2024-10-16 14:57:17.124042] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:17.124451] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=12] current monitor number(seq_=-1) [2024-10-16 14:57:17.124776] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=17381, mem_used=31196160) [2024-10-16 14:57:17.125329] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.125346] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.125353] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051014594560},{"BIGINT":1729058400},{"VARCHAR":"A¨X3¾", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.125405] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.125618] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285241} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.128077] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF226A-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510304996471, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893700, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.129055] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061837128186000, start_id=1729061837128186000, end_id=1729061837128186001) [2024-10-16 14:57:17.129109] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.129119] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.129124] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051039760384},{"BIGINT":1729058400},{"VARCHAR":"B@ 4bd€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.129212] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.129497] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1074}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285242} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.131672] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061837131432000, start_id=1729061837131432000, end_id=1729061837131432001) [2024-10-16 14:57:17.133422] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061837132513000, start_id=1729061837132513000, end_id=1729061837132513001) [2024-10-16 14:57:17.135464] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CDC-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305003858, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893701, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.137623] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.137638] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.137645] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051098480641},{"BIGINT":1729058400},{"VARCHAR":"A⥕ó", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.137702] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.137874] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285244} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.139645] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CDE-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305008039, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893702, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.144795] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=118537) [2024-10-16 14:57:17.148395] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CE0-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305016788, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893703, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.151233] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=97426) [2024-10-16 14:57:17.151942] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CE2-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305020335, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893704, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.152166] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=97426) [2024-10-16 14:57:17.153589] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CE4-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305021976, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893705, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.154041] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.156997] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FB91-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:17.157011] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FB91-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=41759, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:17.157008] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1464, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:17.157015] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FB91-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=498, avg_log_submit_to_slide_time=41752) [2024-10-16 14:57:17.161799] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5013-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:17.161819] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5013-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:17.161941] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.165765] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CE6-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305034159, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893706, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.168304] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.168320] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.168327] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051354333184},{"BIGINT":1729058400},{"VARCHAR":"A’È", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.168388] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.168645] INFO [STORAGE.TRANS] get_tx_ctx_ (ob_trans_ctx_mgr_v4.cpp:495) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] transaction statistics(ls_id={id:1001}, total_tx_ctx_count=1) [2024-10-16 14:57:17.168671] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2147}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285254} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.170035] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061837169524000, start_id=1729061837169524000, end_id=1729061837169524001) [2024-10-16 14:57:17.170550] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3B4F-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:17.170568] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3B4F-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=726, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:17.173313] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CE8-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305041706, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893707, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.173484] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BEA-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:17.173498] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BEA-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:17.173959] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.173973] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.173980] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051392081920},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.174036] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.174335] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2137}, to_seq_no={branch:0, seq:1050}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285256} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.174370] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:145) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20] transaction rpc statistics(total_rt=29497, total_process=617, avg_rt=47) [2024-10-16 14:57:17.175704] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CEA-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305044099, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893708, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.176577] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061837175809000, start_id=1729061837175809000, end_id=1729061837175809001) [2024-10-16 14:57:17.177314] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.177328] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.177334] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051417247744},{"BIGINT":1729058400},{"VARCHAR":"A&̬", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.177383] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.177624] WDIAG [OCCAM] dump_statistics (ob_vtable_event_recycle_buffer.h:144) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15C1-0-0] [lt=14][errcode=-4006] not init(ret=-4006) [2024-10-16 14:57:17.177640] INFO [MDS] dump_map_holding_item (mds_tenant_service.cpp:473) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15C1-0-0] [lt=10] finish scan map holding items(scan_cnt=0) [2024-10-16 14:57:17.177726] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:1055}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285257} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.182367] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15C2-0-0] [lt=4] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:17.186345] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CEC-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305054738, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893709, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.187865] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.187880] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.187887] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051631157248},{"BIGINT":1729058400},{"VARCHAR":"A@ÿ €", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.187940] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.188127] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285259} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.188584] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:17.194104] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CEE-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305062498, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893710, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.194196] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15C6-0-0] [lt=19] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:17.196058] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:17.196148] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15C8-0-0] [lt=11] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:17.196171] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D80-0-0] [lt=9] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061847008855000, latest_log_ts:{val:1729061830358352001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:17.196229] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D80-0-0] [lt=54] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:17.196346] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=43] success to update min reserved snapshot(reserved_snapshot=1729060034390737000, duration=1800, snapshot_gc_ts=1729061834390737000) [2024-10-16 14:57:17.197120] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=12] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060034390737000) [2024-10-16 14:57:17.197134] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=12] update reserved snapshot finished(cost_ts=790, reserved_snapshot=1729060034390737000) [2024-10-16 14:57:17.202344] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CF0-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305070738, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893711, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.207899] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CF2-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305076293, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893712, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.209306] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CF4-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305077701, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893713, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.209361] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061837208317000, start_id=1729061837208317000, end_id=1729061837208317001) [2024-10-16 14:57:17.215230] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CF6-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305083624, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893714, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.220888] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CF8-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305089271, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893715, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.222275] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CFA-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305090669, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893716, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.222919] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.222937] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.222943] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051958312960},{"BIGINT":1729058400},{"VARCHAR":"@E", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.222999] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.223226] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3207}, to_seq_no={branch:0, seq:2122}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285266} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.224634] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061837223850000, start_id=1729061837223850000, end_id=1729061837223850001) [2024-10-16 14:57:17.225370] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CFC-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305093766, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893717, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.227032] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0CFE-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305095428, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893718, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.227695] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.227710] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.227717] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830052021227520},{"BIGINT":1729058400},{"VARCHAR":"B,–óϼ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.227768] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.228015] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1191}, to_seq_no={branch:0, seq:1186}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285268} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.233001] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.233018] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.233024] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830052058976256},{"BIGINT":1729058400},{"VARCHAR":"AAn@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.233125] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.233319] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285270} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.242284] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D00-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305110677, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893719, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.243226] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.243239] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.243251] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830052121890816},{"BIGINT":1729058400},{"VARCHAR":"BeÄ ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.243303] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.243980] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D02-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305112376, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893720, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.245179] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5753}, to_seq_no={branch:0, seq:2194}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285272} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.245213] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD102A-0-0] [lt=15] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:17.247537] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D04-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305115933, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893721, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.254192] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.255200] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.255213] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.255220] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830052172222464},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.255296] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.257035] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21531][T1_TntSharedTim][T1][Y0-0000000000000000-0-0] [lt=12] clear dtl interm result cost(us)(clear_cost=1938, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:17.257220] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=7, avg_log_batch_cnt=1, total_group_log_size=845, avg_group_log_size=120) [2024-10-16 14:57:17.257239] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=845) [2024-10-16 14:57:17.257627] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4218}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285275} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.262002] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.267370] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D06-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305135763, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893722, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.267690] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3657, avg_group_log_size=332) [2024-10-16 14:57:17.267980] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:17.269767] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D08-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305138162, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893723, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.284559] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3ACEC7-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.709790515899658203e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=854, Throttle Trigger(MB)=512, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.330853799468864196e+01) [2024-10-16 14:57:17.284761] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061837284560000, start_id=1729061837284560000, end_id=1729061837284560001) [2024-10-16 14:57:17.285721] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45AEC-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=1231, avg_log_submit_to_slide_time=1778) [2024-10-16 14:57:17.286178] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:17.286575] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2139}, to_seq_no={branch:0, seq:1081}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285282} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.288015] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061837287789000, start_id=1729061837287789000, end_id=1729061837287789001) [2024-10-16 14:57:17.291040] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2120}, to_seq_no={branch:0, seq:1054}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285284} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.293683] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D09-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305162076, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893724, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.298391] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:17.298691] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1211}, to_seq_no={branch:0, seq:1206}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285287} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.299220] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D0B-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305167614, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893725, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.303039] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D0D-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305171433, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893726, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.303742] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1171226624, global_cache_size=1595949056, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:17.304042] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash memory, (tenant_id=1002, cache_size=101957632, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:57:17.304190] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash time detail, (compute_wash_size_time=101, refresh_score_time=77, wash_time=209) [2024-10-16 14:57:17.307011] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=8] Cache wash clean map node details(ret=0, clean_node_count=43, clean_time=0, clean_start_pos=188742, clean_num=62914) [2024-10-16 14:57:17.308925] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:171) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=8] Eliminate task evict sql audit(request_manager_->get_tenant_id()=1003, queue_size=100000, config_mem_limit_=32212254, request_manager_->get_size_used()=5189, evict_high_size_level=90000, evict_low_size_level=80000, allocator->allocated()=10398720, evict_high_mem_level=16106127, evict_low_mem_level=0) [2024-10-16 14:57:17.308940] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5189, mem_used=10398720) [2024-10-16 14:57:17.309331] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061837309307000, start_id=1729061837309307000, end_id=1729061837309307001) [2024-10-16 14:57:17.310009] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2127}, to_seq_no={branch:0, seq:1053}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285289} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.313681] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D0F-0-0] [lt=28][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305182074, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893727, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.314777] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=71, avg_log_batch_cnt=0, total_group_log_size=115292, avg_group_log_size=1623) [2024-10-16 14:57:17.315482] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:17.315519] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:17.317486] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D11-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305185880, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893728, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.319465] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D13-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305187860, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893729, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.320440] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=16] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:17.323862] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2125}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285293} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.323903] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23] rpc statistics(total_trans_msg_count=161, total_batch_msg_count=0) [2024-10-16 14:57:17.324861] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D14-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305193255, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893730, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.325218] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=38] success to update min reserved snapshot(reserved_snapshot=1729060028711668000, duration=1800, snapshot_gc_ts=1729061828711668000) [2024-10-16 14:57:17.326102] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=28] update reserved snapshot finished(cost_ts=888, reserved_snapshot=1729060028711668000) [2024-10-16 14:57:17.328360] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D16-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305196755, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893731, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.329293] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061837328811000, start_id=1729061837328811000, end_id=1729061837328811001) [2024-10-16 14:57:17.331820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061837330929000, start_id=1729061837330929000, end_id=1729061837330929001) [2024-10-16 14:57:17.333634] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061837331959000, start_id=1729061837331959000, end_id=1729061837331959001) [2024-10-16 14:57:17.333976] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D18-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305202371, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893732, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.344610] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA1-0-0] [lt=19] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:17.344638] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA1-0-0] [lt=23] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=30) [2024-10-16 14:57:17.346838] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D1A-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305215232, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893733, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.347562] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3180}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285297} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.351002] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=9] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:17.351022] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:17.351030] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:17.351034] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:17.351037] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:17.351040] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17110) [2024-10-16 14:57:17.352628] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D1C-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305221022, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893734, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.354234] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.355645] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C38-0-0] [lt=6] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:17.355667] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C38-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061837335835000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061837335835000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:17.356985] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D1E-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305225379, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893735, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.357143] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A6D-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178213, log_task={header:{begin_lsn:{lsn:14538458527}, end_lsn:{lsn:14538458649}, log_id:115178213, min_scn:{val:1729061837121884001, v:0}, max_scn:{val:1729061837121884001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538458405}, prev_proposal_id:23, committed_end_lsn:{lsn:14538458527}, data_checksum:1715074492, accum_checksum:694990192, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061837256715, freeze_ts:1729061837256715, submit_ts:1729061837256719, flushed_ts:1729061837256912, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:193}, fs_cb_begin_ts=1729061837357136, log_life_time=100421) [2024-10-16 14:57:17.357207] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C38-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061837357012000, start_id=1729061837357012000, end_id=1729061837357012001) [2024-10-16 14:57:17.357416] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=401, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:17.357425] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:17.358000] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2120}, to_seq_no={branch:0, seq:1055}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285300} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.360694] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C38-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061837360524000, start_id=1729061837360524000, end_id=1729061837360524001) [2024-10-16 14:57:17.362079] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.367146] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=15] blacklist_loop exec finished(cost_time=16, is_enabled=true, send_cnt=0) [2024-10-16 14:57:17.369056] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D23-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305237449, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893736, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.373655] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BEE-0-0] [lt=25] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:17.374097] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D25-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305242491, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893737, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.376511] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285304} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.376667] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061837375355000, start_id=1729061837375355000, end_id=1729061837375355001) [2024-10-16 14:57:17.383405] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:17.384165] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2131}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285305} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.388287] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2118}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285306} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.390260] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] Cache replace map node details(ret=0, replace_node_count=57, replace_time=0, replace_start_pos=503312, replace_num=31457) [2024-10-16 14:57:17.390287] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] replace map num details(ret=0, replace_node_count=57, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:17.392476] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061837392253000, start_id=1729061837392253000, end_id=1729061837392253001) [2024-10-16 14:57:17.395682] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D27-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305264076, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893738, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.400818] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=78, avg_log_batch_cnt=0, total_group_log_size=126655, avg_group_log_size=1623) [2024-10-16 14:57:17.402624] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D29-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305271017, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893739, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.404317] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D2B-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305272712, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893740, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.410753] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061837410083000, start_id=1729061837410083000, end_id=1729061837410083001) [2024-10-16 14:57:17.413996] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D2D-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305282391, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893741, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.415747] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D2F-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305284142, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893742, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.416113] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=463, avg_log_submit_to_slide_time=618) [2024-10-16 14:57:17.419835] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D31-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305288229, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893743, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.424214] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2080}, to_seq_no={branch:0, seq:1022}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285315} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.425528] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061837425085000, start_id=1729061837425085000, end_id=1729061837425085001) [2024-10-16 14:57:17.425732] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D33-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305294079, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893744, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.430138] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3139}, to_seq_no={branch:0, seq:2073}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285317} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.434611] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D35-0-0] [lt=30][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305302997, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893745, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.437472] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2070}, to_seq_no={branch:0, seq:2065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285319} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.444358] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A67-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:17.444373] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A67-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:17.454300] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.455830] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D37-0-0] [lt=32][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305324224, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893746, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.457026] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=43832979, this={dio_aligned_buf:{buf_write_offset:1805, buf_padding_size:2291, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35368971, truncate_used_ts:14640998}, log_block_size:67108864, dir_fd:201, io_fd:197}, aligned_buf_len=4096, aligned_block_offset=43831296, buf_len=122, total_write_size_=12294, total_write_size_after_dio_=425984, ob_pwrite_used_ts=26986, count=101, each_pwrite_cost=267) [2024-10-16 14:57:17.457053] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:17.459450] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285325} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.462141] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.462784] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D39-0-0] [lt=34][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305331177, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893747, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.465135] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79CF-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061837456341, keepalive_ts:-1}) [2024-10-16 14:57:17.465154] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79CF-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061837456341, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:17.468275] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=218, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:17.468292] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=51, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:17.472777] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=17) [2024-10-16 14:57:17.473949] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F3A-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:17.473965] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F3A-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10702, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:17.473965] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:17.473970] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F3A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=306, avg_log_submit_to_slide_time=10696) [2024-10-16 14:57:17.475757] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:17.475769] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=12] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:17.475781] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:17.475776] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A21-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:17.475786] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:17.475790] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:17.475789] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A21-0-0] [lt=12] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:17.475793] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:17.475796] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:17.475798] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:17.475800] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:17.475803] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:17.475805] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:17.478385] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3166}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285328} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.481901] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3983-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=73, avg_log_gen_to_freeze_time=5, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=424, avg_log_submit_to_slide_time=638) [2024-10-16 14:57:17.484831] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D3D-0-0] [lt=31][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305353223, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893748, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.485746] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061837485676000, start_id=1729061837485676000, end_id=1729061837485676001) [2024-10-16 14:57:17.487913] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:17.487924] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:17.487937] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=70432388, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:17.488952] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2818}, to_seq_no={branch:0, seq:1758}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285330} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.491185] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D3E-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305359578, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893749, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.494196] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3229}, to_seq_no={branch:0, seq:1112}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285331} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.495374] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF226E-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305363767, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893750, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.496789] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061837496520000, start_id=1729061837496520000, end_id=1729061837496520001) [2024-10-16 14:57:17.498506] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1041}, to_seq_no={branch:0, seq:1036}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285332} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.501472] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D41-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305369866, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893751, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.501897] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] get number(ret=0, service_type_=0, range=1, base_id=1729061837500745000, start_id=1729061837500745000, end_id=1729061837500745001) [2024-10-16 14:57:17.504091] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D42-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305372484, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893752, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.507349] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1175420928, global_cache_size=1591787520, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:17.508635] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D0-0-0] [lt=16] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061837476371, keepalive_ts:-1}) [2024-10-16 14:57:17.508653] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D0-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061837476371, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:17.511130] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061837510481000, start_id=1729061837510481000, end_id=1729061837510481001) [2024-10-16 14:57:17.512753] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061837512602000, start_id=1729061837512602000, end_id=1729061837512602001) [2024-10-16 14:57:17.512772] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=16] ts response task statistics(alloc_count=261363466, free_count=130681732) [2024-10-16 14:57:17.517274] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:17.517587] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=30] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a65d90, tx_id:{txid:199663859}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061837512602, active_ts:1729061837512602, commit_ts:-1, finish_ts:1729061837512602, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1729061867512595, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061837512629, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:17.518480] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=52] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a4a2d0, tx_id:{txid:199663860}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061837512602, active_ts:1729061837512602, commit_ts:-1, finish_ts:1729061837512602, timeout_us:29999996, lock_timeout_us:-1, expire_ts:1729061867512598, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061837512631, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:17.524331] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D44-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305392723, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893753, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.524558] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:17.525350] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2132}, to_seq_no={branch:0, seq:1075}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285338} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.526377] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D46-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305394772, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893754, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.527134] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=47] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:17.527151] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:17.529591] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061837528689000, start_id=1729061837528689000, end_id=1729061837528689001) [2024-10-16 14:57:17.532028] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061837530854000, start_id=1729061837530854000, end_id=1729061837530854001) [2024-10-16 14:57:17.533793] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061837533426000, start_id=1729061837533426000, end_id=1729061837533426001) [2024-10-16 14:57:17.534427] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33576, start_service_tstamp_=1722494263708085, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:17.535063] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=79, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:17.535076] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=79, stat_interval=1000000, avg (count/cost)=3135, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:17.535099] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=127376, hit_cnt=78, read_cnt=78, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:17.536743] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2118}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285342} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.539527] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1071}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285343} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.540913] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D48-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305409306, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893755, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.542646] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D4A-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305411041, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893756, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.546396] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D4C-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305414790, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893757, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.548618] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D4E-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305417013, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893758, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.548911] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285346} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.550219] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061837549331000, start_id=1729061837549331000, end_id=1729061837549331001) [2024-10-16 14:57:17.550308] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D50-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305418704, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893759, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.554330] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.558780] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F398F-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=7, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=626, avg_log_submit_to_slide_time=997) [2024-10-16 14:57:17.561711] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D52-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305430104, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893760, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.562207] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.564624] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [55867][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D55-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:17.564650] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [55867][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D55-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061837121884003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061837121884003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:17.565212] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:17.565229] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:17.566116] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D55-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061837565554000, start_id=1729061837565554000, end_id=1729061837565554001) [2024-10-16 14:57:17.568683] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D55-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061837567666000, start_id=1729061837567666000, end_id=1729061837567666001) [2024-10-16 14:57:17.568964] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.569492] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.569613] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.569634] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.570107] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2350}, to_seq_no={branch:0, seq:1147}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285349} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.571951] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061837571046000, start_id=1729061837571046000, end_id=1729061837571046001) [2024-10-16 14:57:17.572839] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D54-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305441233, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893761, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.576805] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061837576345000, start_id=1729061837576345000, end_id=1729061837576345001) [2024-10-16 14:57:17.578374] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D56-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305446767, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893762, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.578503] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1002}, {id:1001}]) [2024-10-16 14:57:17.579630] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [22451][T1003_ObXAHbWor][T1003][Y0-0000000000000000-0-0] [lt=15] XA scheduler heartbeat task statistics(avg_time=1) [2024-10-16 14:57:17.591976] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=54, replace_time=0, replace_start_pos=534769, replace_num=31457) [2024-10-16 14:57:17.591996] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] replace map num details(ret=0, replace_node_count=54, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:17.595707] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4475, mem_used=8318976) [2024-10-16 14:57:17.600396] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D58-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305468789, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893763, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.602177] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D5A-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305470571, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893764, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.602195] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2236}, to_seq_no={branch:0, seq:1175}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285358} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.604783] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061837604137000, start_id=1729061837604137000, end_id=1729061837604137001) [2024-10-16 14:57:17.610570] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=18] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=128282) [2024-10-16 14:57:17.611092] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=129906, hit_cnt=80, read_cnt=80, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:17.613589] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D5C-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305481983, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893765, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.613929] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061837613724000, start_id=1729061837613724000, end_id=1729061837613724001) [2024-10-16 14:57:17.614619] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4256}, to_seq_no={branch:0, seq:4251}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285360} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.617924] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285361} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.621454] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2271-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305489846, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893766, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.622515] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B65-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061837621624000, start_id=1729061837621624000, end_id=1729061837621624001) [2024-10-16 14:57:17.622635] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B66-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061837621624000, start_id=1729061837621624000, end_id=1729061837621624001) [2024-10-16 14:57:17.623313] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B8DE-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061837621624000, start_id=1729061837621624000, end_id=1729061837621624001) [2024-10-16 14:57:17.623639] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D5E-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305492033, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893767, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.624616] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285362} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.628054] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D60-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305496449, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893768, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.629850] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285364} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.633205] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285365} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.633956] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D62-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305502349, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893769, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.634825] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=21] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:57:17.636760] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2272-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305505154, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893770, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.642065] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=80, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:17.642338] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=80, stat_interval=1000000, avg (count/cost)=342, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:17.642356] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=80, stat_interval=1000000, avg (count/cost)=56, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:17.642395] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=80, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:17.645698] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D65-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305514092, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893771, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.647940] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:57:17.647955] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:17.647962] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:17.647968] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:17.649071] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:17.650164] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=111, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:57:17.650320] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=144] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:17.652914] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D68-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305521308, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893772, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.654668] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.655820] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=12] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:57:17.656592] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061837655118000, start_id=1729061837655118000, end_id=1729061837655118001) [2024-10-16 14:57:17.657020] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=28][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5024, dropped:116, tid:12153}]) [2024-10-16 14:57:17.658512] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.658527] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.658533] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545342459990016},{"BIGINT":1729058400},{"VARCHAR":"AÙ™™", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.658594] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.658808] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1036}, to_seq_no={branch:0, seq:1031}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285370} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.662270] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.667982] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:17.668016] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=24] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3657) [2024-10-16 14:57:17.668024] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:17.668557] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F26-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3779) [2024-10-16 14:57:17.668598] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F26-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:17.668612] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F26-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=932, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:17.669323] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.669336] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.669343] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545342598402048},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.669465] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.669712] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1072}, to_seq_no={branch:0, seq:1067}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285374} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.670302] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D6D-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305538694, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893773, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.673936] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=176, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:17.673953] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:17.673964] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:17.674003] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:17.678569] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D6F-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305546961, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893774, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:17.680590] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=14] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:17.685988] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061837684214000, start_id=1729061837684214000, end_id=1729061837684214001) [2024-10-16 14:57:17.687239] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3657) [2024-10-16 14:57:17.689442] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.689459] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.689469] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545342875226112},{"BIGINT":1729058400},{"VARCHAR":"@U", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.689538] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.689742] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2128}, to_seq_no={branch:0, seq:1045}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285379} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.692071] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.692085] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.692091] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545342912974848},{"BIGINT":1729058400},{"VARCHAR":"@U€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.692138] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.693232] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2115}, to_seq_no={branch:0, seq:1057}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285380} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.695276] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:17.707655] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1177518080, global_cache_size=1591787520, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:17.715556] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:17.715576] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:17.716548] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061837716054000, start_id=1729061837716054000, end_id=1729061837716054001) [2024-10-16 14:57:17.720335] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:17.722243] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15CC-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:17.729611] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=45] get number(ret=0, service_type_=0, range=1, base_id=1729061837728931000, start_id=1729061837728931000, end_id=1729061837728931001) [2024-10-16 14:57:17.730194] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:207588446290}, scn={val:1729061837716054001, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061837726521, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207588446290}, next_to_submit_scn_:{val:1729061837716054001, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:1624, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1624, log_storage_:{IteratorStorage:{start_lsn:{lsn:207588444666}, end_lsn:{lsn:207588446290}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061837716054000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:1536, scn_:{val:1729061837716054000, v:0}, data_checksum:2307591500, flag:1}}, init_mode_version:0, accumulate_checksum:15390573, curr_entry_is_padding:0, padding_entry_size:1568, padding_entry_scn:{val:1729061837716054000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598071, miss_cnt:9, cache_read_size:149776810731, hit_ratio:"9.999999345921105620e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:57:17.732103] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061837731067000, start_id=1729061837731067000, end_id=1729061837731067001) [2024-10-16 14:57:17.733923] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061837733289000, start_id=1729061837733289000, end_id=1729061837733289001) [2024-10-16 14:57:17.748282] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=3] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061833741264, last_first_register_time_us=1722494157071428, register_time_us=1729061833741264, curr_time_us=1729061837748279) [2024-10-16 14:57:17.748334] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=25] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061837748308, last_submit_register_req_time_us=1729061837748308, register_parent_reason=3) [2024-10-16 14:57:17.748358] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=23] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:17.748617] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:17.748643] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:17.748651] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:17.750566] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=22] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):113, sequence:1}) [2024-10-16 14:57:17.754735] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.756697] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A75-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:17.756875] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=328, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:17.756891] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=114, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:17.756917] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:17.757015] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62649978, miss_cnt:32, cache_read_size:8050388344, hit_ratio:"9.999994892259395085e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:17.758037] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=48, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:57:17.758050] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=13] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=735, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:57:17.758054] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:57:17.758058] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=-599, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:57:17.758061] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=184, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:57:17.759467] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257658, offset=39003677}, end_cursor:ObLogCursor{file_id=197, log_id=49257659, offset=39003960}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a14050, len:2531, log_data_len:283, seq:49257658, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:17.759611] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.759626] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.759633] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343277879296},{"BIGINT":1729058400},{"VARCHAR":"@U", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.759704] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.759885] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1058}, to_seq_no={branch:0, seq:1053}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285388} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.761086] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061837760001000, start_id=1729061837760001000, end_id=1729061837760001001) [2024-10-16 14:57:17.762157] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB357-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061837760001000, start_id=1729061837760001000, end_id=1729061837760001001) [2024-10-16 14:57:17.762339] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.762455] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=304, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:17.762473] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=73, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:17.762532] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:17.762965] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257659, offset=39003960}, end_cursor:ObLogCursor{file_id=197, log_id=49257660, offset=39004242}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b30050, len:2248, log_data_len:282, seq:49257659, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:17.764630] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=12] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:17.765429] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257660, offset=39004242}, end_cursor:ObLogCursor{file_id=197, log_id=49257661, offset=39004524}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29a8050, len:1966, log_data_len:282, seq:49257660, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:17.765570] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257661, offset=39004524}, end_cursor:ObLogCursor{file_id=197, log_id=49257662, offset=39004806}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a24050, len:1684, log_data_len:282, seq:49257661, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:17.765642] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656712, miss_cnt:17, cache_read_size:8050135603, hit_ratio:"9.999997286803784524e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:17.768088] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.768104] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.768114] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:17.768111] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343340793856},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.768170] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.768410] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2105}, to_seq_no={branch:0, seq:1027}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285390} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.770517] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598075, miss_cnt:9, cache_read_size:149776817224, hit_ratio:"9.999999345921124494e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:17.771084] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601292, offset=9598630}, end_cursor:ObLogCursor{file_id=192, log_id=47601293, offset=9598914}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bec050, len:2394, log_data_len:284, seq:47601292, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:17.773121] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48C-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061837773063000, start_id=1729061837773063000, end_id=1729061837773063001) [2024-10-16 14:57:17.773188] INFO [LIB] stat (utility.h:1165) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48C-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:17.773200] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48C-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=10435, avg_group_log_size=521) [2024-10-16 14:57:17.773209] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48C-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10048) [2024-10-16 14:57:17.773215] INFO [LIB] stat (utility.h:1165) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48C-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:17.773379] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=80, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:17.773420] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=291, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:17.773430] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10435) [2024-10-16 14:57:17.773438] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=56, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:17.773449] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10048) [2024-10-16 14:57:17.773477] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=25] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:17.773697] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F932F-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:17.773706] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F932F-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=756, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:17.773710] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F932F-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=11, avg_log_gen_to_submit_time=38, avg_log_submit_to_flush_time=515, avg_log_submit_to_slide_time=717) [2024-10-16 14:57:17.774079] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=24] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601293, offset=9598914}, end_cursor:ObLogCursor{file_id=192, log_id=47601294, offset=9599188}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b60050, len:2110, log_data_len:274, seq:47601293, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:17.774399] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560481, miss_cnt:1, cache_read_size:152589560490, hit_ratio:"9.999999928346477995e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:17.774528] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA868-0-0] [lt=26] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:17.774546] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA868-0-0] [lt=9] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061837767698000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061837767698000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:17.776713] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA868-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061837775157000, start_id=1729061837775157001, end_id=1729061837775157002) [2024-10-16 14:57:17.776975] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061837775157000, start_id=1729061837775157002, end_id=1729061837775157003) [2024-10-16 14:57:17.779274] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=22] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:17.779359] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601294, offset=9599188}, end_cursor:ObLogCursor{file_id=192, log_id=47601295, offset=9599462}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29a4050, len:1836, log_data_len:274, seq:47601294, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:17.779459] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:17.779472] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:17.779480] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=46) [2024-10-16 14:57:17.779594] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601295, offset=9599462}, end_cursor:ObLogCursor{file_id=192, log_id=47601296, offset=9599736}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2874050, len:1562, log_data_len:274, seq:47601295, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:17.780779] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA868-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061837780327000, start_id=1729061837780327000, end_id=1729061837780327001) [2024-10-16 14:57:17.781637] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=2] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:17.782149] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.782165] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.782171] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343466622976},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.782241] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.782590] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2106}, to_seq_no={branch:0, seq:1088}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285393} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.786460] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=30][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.786479] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.786486] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343512760321},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.786556] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=42][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.788339] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.788362] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.788371] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.788379] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.789949] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.790021] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.790639] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.790660] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.790677] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.790696] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.790708] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:17.791307] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6665}, to_seq_no={branch:0, seq:1074}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285394} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.793723] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] Cache replace map node details(ret=0, replace_node_count=43, replace_time=0, replace_start_pos=566226, replace_num=31457) [2024-10-16 14:57:17.793740] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=43, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:17.794173] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.794190] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.794198] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256244707328},{"BIGINT":1729058400},{"VARCHAR":"@ûÄ@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.794264] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.794774] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3160}, to_seq_no={branch:0, seq:1050}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202134} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.795817] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.795834] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.795873] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=36][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343554703360},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.795933] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.796241] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1070}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285395} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.806032] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=4] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:17.806671] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=21] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:17.806723] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061834390737000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:17.812958] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.812974] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.812982] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343667949568},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.813053] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.813270] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1070}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285398} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.814792] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061837814327000, start_id=1729061837814327001, end_id=1729061837814327002) [2024-10-16 14:57:17.814836] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.814848] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.814858] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=76, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:17.814871] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=76, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:17.814854] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256433451008},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.814908] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.815068] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=123412) [2024-10-16 14:57:17.815091] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=123412) [2024-10-16 14:57:17.815411] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1001}, {id:1003}]) [2024-10-16 14:57:17.815783] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2114}, to_seq_no={branch:0, seq:1052}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202138} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.815835] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=221, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:17.815848] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=824, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:17.816573] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.816588] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.816594] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343701504001},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.816671] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=36][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.816932] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285399} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.819205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061837818575000, start_id=1729061837818575000, end_id=1729061837818575001) [2024-10-16 14:57:17.819572] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.819620] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=47][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.819626] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256458616832},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.819676] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.819884] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.819899] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.819906] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343726669825},{"BIGINT":1729058400},{"VARCHAR":"@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.819960] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.820052] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1178}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202139} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.820165] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1177}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285400} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.822484] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.822499] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.822505] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343781195776},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.822552] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.822744] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285401} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.825680] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.825694] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.825699] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343831527424},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.825746] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.826016] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285402} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.827068] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:17.827086] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:17.827099] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52168296, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:17.828869] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.828886] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.828892] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343881859072},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.828943] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.829203] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:1055}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285403} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.831991] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.832006] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.832013] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343919607808},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.832057] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.832065] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.832068] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.832074] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256534114304},{"BIGINT":1729058400},{"VARCHAR":"Aå", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.832130] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.832358] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1072}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285404} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.832467] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1072}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202142} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.849293] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2936, mem_used=6239232) [2024-10-16 14:57:17.854752] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.856153] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=40] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:17.856637] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22136][T1002_TntTransf][T1002][YB4A0A000407-00061E9AB24CC6BA-0-0] [lt=17] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:57:17.856765] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=13] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:17.856819] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=17] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061831822855000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:17.857149] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A77-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:17.862400] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.882059] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4D10-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=78, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:17.882077] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4D10-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=78, stat_interval=1000000, avg (count/cost)=3640, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:17.882903] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=15] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:17.886048] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061837885398000, start_id=1729061837885398000, end_id=1729061837885398001) [2024-10-16 14:57:17.887181] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061837512602000, v:0}, new_max_version={val:1729061837512602000, v:0}, last_min_version={val:1729061837284560001, v:0}, last_max_version={val:1729061837284560001, v:0}, total_time_us=1995, delta_us=374578) [2024-10-16 14:57:17.887201] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061837512602000, v:0}, min_version={val:1729061837512602000, v:0}, max_version={val:1729061837512602000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:17.890768] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061837890767, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007617, add_bytes: 0 [2024-10-16 14:57:17.891886] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 369.00; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 13824.00, iops: 23.94, rt: 109.50; ) [2024-10-16 14:57:17.891909] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:17.891962] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 123.00; ) [2024-10-16 14:57:17.891971] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:17.891980] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:17.898008] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.898024] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.898031] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256571863040},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.898098] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.899631] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:207588464207}, scn={val:1729061837860160001, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:17.899669] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:16, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061837899584, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207588464263}, next_to_submit_scn_:{val:1729061837860160002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207588464151}, end_lsn:{lsn:207588464263}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061837860160001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061837860160001, v:0}, data_checksum:2980535072, flag:0}}, init_mode_version:0, accumulate_checksum:2697261818, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061837860160001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598083, miss_cnt:9, cache_read_size:149776828704, hit_ratio:"9.999999345921163352e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207588464207}, scn_:{val:1729061837860160001, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061837860532, first_handle_ts_:1729061837899668, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f4ae208c0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:17.899831] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=70] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060030183137000) [2024-10-16 14:57:17.899843] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207588464207}, scn_:{val:1729061837860160001, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061837860532, first_handle_ts_:1729061837899818, replay_cost_:23, retry_cost_:-1, read_log_buf_:0x2b0f4ae208d0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061837899584, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207588464263}, next_to_submit_scn_:{val:1729061837860160002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207588464151}, end_lsn:{lsn:207588464263}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061837860160001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061837860160001, v:0}, data_checksum:2980535072, flag:0}}, init_mode_version:0, accumulate_checksum:2697261818, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061837860160001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598083, miss_cnt:9, cache_read_size:149776828704, hit_ratio:"9.999999345921163352e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_queue_idx=20, ret=0) [2024-10-16 14:57:17.907977] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1173323776, global_cache_size=1593868288, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:17.908433] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:16009}, to_seq_no={branch:0, seq:4189}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202144} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.910210] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.910226] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.910234] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343969939456},{"BIGINT":1729058400},{"VARCHAR":"@Ÿ0", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.910314] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=43][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.910516] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3156}, to_seq_no={branch:0, seq:3151}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285406} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.911624] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061837910817000, start_id=1729061837910817000, end_id=1729061837910817001) [2024-10-16 14:57:17.911691] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=82, avg_log_batch_cnt=1, total_group_log_size=133155, avg_group_log_size=1623) [2024-10-16 14:57:17.911702] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=133155) [2024-10-16 14:57:17.915058] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22267][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061837913996000, start_id=1729061837913996000, end_id=1729061837913996001) [2024-10-16 14:57:17.918762] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=29] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:17.918960] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061837918868000, start_id=1729061837918868000, end_id=1729061837918868001) [2024-10-16 14:57:17.919405] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:17.919822] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:17.921026] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=83, stat_interval=1000000, avg (count/cost)=232, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:17.921043] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=83, stat_interval=1000000, avg (count/cost)=80, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:17.921071] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=83, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:17.925843] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=17] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:17.927872] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:57:17.927886] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=11] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:17.927892] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:17.927897] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:17.929610] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=54] get number(ret=0, service_type_=0, range=1, base_id=1729061837928469000, start_id=1729061837928469000, end_id=1729061837928469001) [2024-10-16 14:57:17.930130] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:57:17.930471] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=338] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:17.932207] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061837930929000, start_id=1729061837930929000, end_id=1729061837930929001) [2024-10-16 14:57:17.933989] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061837930929000, start_id=1729061837930929001, end_id=1729061837930929002) [2024-10-16 14:57:17.936952] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=1255, mem_used=4159488) [2024-10-16 14:57:17.943264] WDIAG [PALF] is_prev_log_pid_match (log_sliding_window.cpp:3884) [106946][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=4][errcode=-4018] get_prev_log_info_ failed(tmp_ret=-4018, log_id=218496611, palf_id=1003, self="10.0.4.7:2890", log_id=218496611, lsn={lsn:207588469135}, prev_lsn={lsn:207588467511}, prev_log_pid=81) [2024-10-16 14:57:17.948362] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=5] LogLoopThread round_cost_time(us)(round_cost_time=27) [2024-10-16 14:57:17.951242] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=20] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:17.951611] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061837951610, bytes: 4446791717235, bw: 1.864751 MB/s, add_ts: 1000307, add_bytes: 1955933 [2024-10-16 14:57:17.954813] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.957051] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:17.957069] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:17.957081] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=9] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=154) [2024-10-16 14:57:17.957685] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.957696] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.957703] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256622194688},{"BIGINT":1729058400},{"VARCHAR":"@jÀ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.957764] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.957777] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A79-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:17.957801] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A79-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178219, log_task={header:{begin_lsn:{lsn:14538459259}, end_lsn:{lsn:14538459381}, log_id:115178219, min_scn:{val:1729061837623684002, v:0}, max_scn:{val:1729061837623684002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538459137}, prev_proposal_id:23, committed_end_lsn:{lsn:14538459259}, data_checksum:1715074492, accum_checksum:4069104427, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061837857135, freeze_ts:1729061837857135, submit_ts:1729061837857141, flushed_ts:1729061837857750, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:609}, fs_cb_begin_ts=1729061837957794, log_life_time=100659) [2024-10-16 14:57:17.962242] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:15633}, to_seq_no={branch:0, seq:8470}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202147} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.962429] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:17.974491] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5] ====== checkpoint timer task ====== [2024-10-16 14:57:17.975205] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:17.975230] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=24] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:17.975253] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:17.975269] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=39] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:17.975327] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=11] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061849029444000, latest_log_ts:{val:1729061832415320001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:17.975377] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=47] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:17.975395] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:57:17.975402] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061832415320001, v:0}) [2024-10-16 14:57:17.975406] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:17.975410] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:17.975415] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:17.975425] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:57:17.975433] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:17.975438] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:17.976982] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061837976580000, start_id=1729061837976580000, end_id=1729061837976580001) [2024-10-16 14:57:17.981306] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:17.984888] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=182, stat_interval=1000000, avg (count/cost)=47, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:17.984902] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=182, stat_interval=1000000, avg (count/cost)=80, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:17.986691] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.986707] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.986713] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344171266048},{"BIGINT":1729058400},{"VARCHAR":"@Ó•€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.986770] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.986965] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:1057}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285410} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:17.995204] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=49, replace_time=0, replace_start_pos=597683, replace_num=31457) [2024-10-16 14:57:17.995219] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=49, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:17.995695] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=18] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:17.998354] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=36][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:17.998367] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:17.998374] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344209014784},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:17.998429] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:17.999235] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2123}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285411} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.013068] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061838012065000, start_id=1729061838012065000, end_id=1729061838012065001) [2024-10-16 14:57:18.013090] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=548, get_gts_with_stc_cnt=32, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=30, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:18.013099] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:18.013111] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=1204, get_gts_with_stc_cnt=195, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=190, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:18.013122] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=3] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061838013118], gts:1729061838012065000, latest_srr:[mts=1729061838013118]}) [2024-10-16 14:57:18.015707] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:18.015726] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:18.015927] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:18.015941] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:18.015967] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:18.023755] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.023771] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.023783] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256710275072},{"BIGINT":1729058400},{"VARCHAR":"B[Ýí˜h@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.023850] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.032319] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:25946}, to_seq_no={branch:0, seq:10086}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202151} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.043107] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061838042224000, start_id=1729061838042224000, end_id=1729061838042224001) [2024-10-16 14:57:18.048855] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061838048561000, start_id=1729061838048561000, end_id=1729061838048561001) [2024-10-16 14:57:18.050344] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.050362] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.050369] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256722857984},{"BIGINT":1729058400},{"VARCHAR":"AoÈ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.050428] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.050871] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2114}, to_seq_no={branch:0, seq:1068}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202152} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.054870] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.054876] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.054883] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.054889] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344397758464},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.054944] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.055189] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2115}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285414} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.057922] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:18.057938] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:18.058122] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:18.058130] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=732) [2024-10-16 14:57:18.058146] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=82, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:18.058173] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=732) [2024-10-16 14:57:18.058180] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:18.058368] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3BD2-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:18.058374] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3BD2-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=924, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:18.059552] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.059565] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.059570] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344460673024},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.059627] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.059785] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1786}, to_seq_no={branch:0, seq:1781}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285415} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.061904] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.061917] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.061923] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344599085056},{"BIGINT":1729058400},{"VARCHAR":"@µ$", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.061967] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.062262] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285416} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.062493] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.073172] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109202867, max_lsn:17959663138, max_scn:{val:1729061838056090000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7385580775}, reuse_lsn:{lsn:17959663138}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17917720098}}, last_submit_lsn:{lsn:17959663016}, last_submit_end_lsn:{lsn:17959663138}, last_submit_log_id:109202867, last_submit_log_pid:246, max_flushed_lsn:{lsn:17959663016}, max_flushed_end_lsn:{lsn:17959663138}, max_flushed_log_pid:246, committed_end_lsn:{lsn:17959663138}, last_slide_log_id:109202867, last_slide_scn:{val:1729061838056090000, v:0}, last_slide_lsn:{lsn:17959663016}, last_slide_end_lsn:{lsn:17959663138}, last_slide_log_pid:246, last_slide_log_accum_checksum:3080936528, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1d4530}, [StateMgr]={this:0x2b0e0f1d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109202868, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261839590, palf_role_change_cb:0x2b0e0f1df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, reconfig_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, persistent_config_version:{proposal_id:246, config_seq:121}, ms_ack_list:0, resend_config_version:{proposal_id:246, config_seq:121}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258131745, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258131745, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262039736, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728994126835081001, v:0}, min_block_id:265, min_block_min_scn:{val:1728952113212346001, v:0}, base_lsn_for_block_gc:{lsn:17849868288}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, log_config_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, log_snapshot_meta:{version:1, base_lsn:{lsn:17849868288}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1798144}, readable_log_tail:{lsn:1798144}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17984077824}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/meta", dir_fd:167, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65306624, block_header_serialize_buf_:0x2b0e0f1da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17959663138}, readable_log_tail:{lsn:17959663138}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17916973056}, min_scn:{val:1729035023717936004, v:0}, curr_block_id:267, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/log", dir_fd:169, min_block_id:265, max_block_id:268, curr_writable_block_id:267}, logical_block_size_:67104768, curr_block_writable_size_:24414686, block_header_serialize_buf_:0x2b0e0f1dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261838799, this:0x2b0e0f1d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:246, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:245, majority_max_lsn:{lsn:7385763025}, saved_end_lsn:{lsn:7385763025}, last_submit_prepare_req_time_us:1722494261839744, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1d89b0}) [2024-10-16 14:57:18.073321] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=146] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838058170, last_flushed_end_lsn:{lsn:17959663138}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838058352, last_flushed_end_lsn:{lsn:17959663138}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838059360, last_flushed_end_lsn:{lsn:17959663138}}]) [2024-10-16 14:57:18.073332] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=109202868, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838058170, last_flushed_end_lsn:{lsn:17959663138}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838058352, last_flushed_end_lsn:{lsn:17959663138}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838059360, last_flushed_end_lsn:{lsn:17959663138}}]) [2024-10-16 14:57:18.086143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061838085324000, start_id=1729061838085324000, end_id=1729061838085324001) [2024-10-16 14:57:18.108258] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1167032320, global_cache_size=1593868288, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:18.110006] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:18.110020] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:18.115744] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:18.115760] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:18.115786] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:18.115791] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:18.116529] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C8A-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510305984922, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410506, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:18.116913] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:18.116925] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1701, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:18.116929] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=1411, avg_log_submit_to_slide_time=1687) [2024-10-16 14:57:18.116957] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=25, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:57:18.116962] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=1265, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:57:18.116965] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:57:18.116968] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=-928, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:57:18.116972] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=362, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:57:18.116980] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=290, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:18.116986] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=508, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:18.116996] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:18.117000] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1715, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:18.117003] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:18.119622] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.119633] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.119639] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256810938368},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.119701] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.120020] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:15811}, to_seq_no={branch:0, seq:13694}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202155} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.121998] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=24] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97928097792) [2024-10-16 14:57:18.122985] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=7] gc stale ls task succ [2024-10-16 14:57:18.123106] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B6A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061838122276000, start_id=1729061838122276000, end_id=1729061838122276001) [2024-10-16 14:57:18.123168] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B6E-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061838122276000, start_id=1729061838122276000, end_id=1729061838122276001) [2024-10-16 14:57:18.123355] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B6B-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061838123335000, start_id=1729061838123335000, end_id=1729061838123335001) [2024-10-16 14:57:18.123402] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0B6D-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061838123335000, start_id=1729061838123335000, end_id=1729061838123335001) [2024-10-16 14:57:18.123887] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.123900] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.123906] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344745885697},{"BIGINT":1729058400},{"VARCHAR":"A`„", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.123960] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.124205] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285420} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.124544] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=11] current monitor number(seq_=-1) [2024-10-16 14:57:18.124857] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:185) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] sql audit evict mem start(request_manager_->get_tenant_id()=1, evict_low_mem_level=0, evict_high_mem_level=32212254, size_used=17477, mem_used=33275904) [2024-10-16 14:57:18.126516] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:191) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] release old cannot free more memory [2024-10-16 14:57:18.126525] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=8] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=5, elapse_time=1668, size_used=0, mem_used=2079744) [2024-10-16 14:57:18.127468] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10827, avg_group_log_size=569) [2024-10-16 14:57:18.127482] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11460) [2024-10-16 14:57:18.127799] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=249, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:18.127811] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10827) [2024-10-16 14:57:18.127819] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=50, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:18.127841] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11460) [2024-10-16 14:57:18.127857] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:18.127954] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCF9B-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:18.127961] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCF9B-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=799, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:18.127965] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCF9B-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=425, avg_log_submit_to_slide_time=769) [2024-10-16 14:57:18.128557] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.128572] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.128588] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256873852928},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.128645] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.128923] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2189}, to_seq_no={branch:0, seq:1102}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202158} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.129043] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.129051] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.129056] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344800411648},{"BIGINT":1729058400},{"VARCHAR":"A¡!", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.129096] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.129280] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1091}, to_seq_no={branch:0, seq:1085}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285421} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.129757] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061838128711000, start_id=1729061838128711000, end_id=1729061838128711001) [2024-10-16 14:57:18.132429] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061838131923000, start_id=1729061838131923000, end_id=1729061838131923001) [2024-10-16 14:57:18.134285] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061838134056000, start_id=1729061838134056000, end_id=1729061838134056001) [2024-10-16 14:57:18.136039] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.136053] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.136059] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344888492032},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.136108] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.136296] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1056}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285424} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.154938] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.158126] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F47-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:18.158140] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F47-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30401, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:18.158145] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F47-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=579, avg_log_submit_to_slide_time=30395) [2024-10-16 14:57:18.158172] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:18.161426] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061838160815000, start_id=1729061838160815000, end_id=1729061838160815001) [2024-10-16 14:57:18.161561] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061838160815000, start_id=1729061838160815001, end_id=1729061838160815002) [2024-10-16 14:57:18.161984] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=107286) [2024-10-16 14:57:18.162185] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFFA-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061838153359, keepalive_ts:-1}) [2024-10-16 14:57:18.162196] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFFA-0-0] [lt=10] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061838153359, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:18.162271] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFF9-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061838153323, keepalive_ts:-1}) [2024-10-16 14:57:18.162276] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFF9-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061838153323, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:18.162553] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.162579] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5014-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:18.162611] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5014-0-0] [lt=30] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:18.164153] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.164169] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.164175] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256886435840},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.164242] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.164627] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2131}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202159} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.166448] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AC-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:18.166476] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AC-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061837621624003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061837621624003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:18.177217] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061838176733000, start_id=1729061838176733000, end_id=1729061838176733001) [2024-10-16 14:57:18.181745] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=116007) [2024-10-16 14:57:18.181874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AC-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061838180973000, start_id=1729061838180973000, end_id=1729061838180973001) [2024-10-16 14:57:18.181940] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=116006) [2024-10-16 14:57:18.182639] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=5] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:18.186573] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BFE-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:18.186599] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53BFE-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:18.196762] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=58, replace_time=0, replace_start_pos=629140, replace_num=31457) [2024-10-16 14:57:18.196788] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] replace map num details(ret=0, replace_node_count=58, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:18.204236] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.204253] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.204260] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256936767488},{"BIGINT":1729058400},{"VARCHAR":"A+"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.204327] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.204575] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2388}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202162} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.207710] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061838207640000, start_id=1729061838207640000, end_id=1729061838207640001) [2024-10-16 14:57:18.208284] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3BE4-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=74, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:18.208296] INFO [LIB] stat (utility.h:1165) [106946][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3BE4-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=74, stat_interval=1000000, avg (count/cost)=644, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:18.209467] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.209483] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.209490] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545341872787458},{"BIGINT":1729058400},{"VARCHAR":"A·8", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.209549] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.209900] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2113}, to_seq_no={branch:0, seq:1053}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202163} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.214325] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.214345] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.214351] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618576442281984},{"BIGINT":1729058400},{"VARCHAR":"AU·€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.214407] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.214630] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1035}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285432} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.219737] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [106967][][T1002][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1002, ret=0, thread_count_=191) [2024-10-16 14:57:18.219895] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [106968][][T1002][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1002, ret=0, thread_count_=192) [2024-10-16 14:57:18.219909] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [106968][][T1002][Y0-0000000000000000-0-0] [lt=12] Init thread local success [2024-10-16 14:57:18.219933] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=7] init point thread id with(&point=0x55e3435c2480, idx_=472, point=[thread id=106968, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=106968) [2024-10-16 14:57:18.220389] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [106967][][T1002][Y0-0000000000000000-0-0] [lt=649] Init thread local success [2024-10-16 14:57:18.220396] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [106969][][T1002][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1002, ret=0, thread_count_=193) [2024-10-16 14:57:18.220409] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [106969][][T1002][Y0-0000000000000000-0-0] [lt=12] Init thread local success [2024-10-16 14:57:18.220430] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] worker thread created(tenant_->id()=1002, group_id_=1, token=1) [2024-10-16 14:57:18.220865] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.220876] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.220883] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618576656191488},{"BIGINT":1729058400},{"VARCHAR":"AØaÙ€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.220936] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.221516] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1289}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285433} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.222903] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061838222521000, start_id=1729061838222521001, end_id=1729061838222521002) [2024-10-16 14:57:18.224678] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.224694] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.224700] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618576782020608},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.224769] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.225015] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2094}, to_seq_no={branch:0, seq:1054}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285434} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.227566] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.227581] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.227587] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618576920432640},{"BIGINT":1729058400},{"VARCHAR":"B Ê% ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.227641] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.229522] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3215}, to_seq_no={branch:0, seq:1069}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285435} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.234710] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [106967][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7E3C9C6E-0-0] [lt=24] init point thread id with(&point=0x55e3435c2440, idx_=471, point=[thread id=106967, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=106967) [2024-10-16 14:57:18.235498] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.235509] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.235515] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618577096593408},{"BIGINT":1729058400},{"VARCHAR":"A6-", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.235577] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.235648] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:213057148165}, scn={val:1729061838226755002, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:18.235786] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060030183137000) [2024-10-16 14:57:18.235794] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=8] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:213057148165}, scn_:{val:1729061838226755002, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061838225789, first_handle_ts_:1729061838235772, replay_cost_:21, retry_cost_:-1, read_log_buf_:0x2b0f4ae2f920, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061838235594, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213057148221}, next_to_submit_scn_:{val:1729061838226755003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:213057148109}, end_lsn:{lsn:213057148221}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061838226755002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061838226755002, v:0}, data_checksum:324580586, flag:0}}, init_mode_version:0, accumulate_checksum:35063522, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061838226755002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560499, miss_cnt:1, cache_read_size:152589588203, hit_ratio:"9.999999928346487987e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}, replay_queue_idx=29, ret=0) [2024-10-16 14:57:18.236217] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1235}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285436} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.240887] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.240905] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.240911] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545342069919744},{"BIGINT":1729058400},{"VARCHAR":"AËèxß", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.240964] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.242044] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6024}, to_seq_no={branch:0, seq:3916}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202168} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.244619] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] succ to submit log task to replay service(to_submit_lsn={lsn:213057149900}, to_submit_scn={val:1729061838241668001, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061838244547, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213057151468}, next_to_submit_scn_:{val:1729061838241668002, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:1624, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1624, log_storage_:{IteratorStorage:{start_lsn:{lsn:213057149844}, end_lsn:{lsn:213057151468}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061838241668001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:1536, scn_:{val:1729061838241668001, v:0}, data_checksum:1450559795, flag:1}}, init_mode_version:0, accumulate_checksum:1226015509, curr_entry_is_padding:0, padding_entry_size:1568, padding_entry_scn:{val:1729061838241668001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560501, miss_cnt:1, cache_read_size:152589591450, hit_ratio:"9.999999928346489098e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:18.250214] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.250240] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.250250] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618577612492800},{"BIGINT":1729058400},{"VARCHAR":"AO¾", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.250321] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.251669] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3223}, to_seq_no={branch:0, seq:1022}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285439} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.254967] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.258032] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=6, avg_log_batch_cnt=1, total_group_log_size=732, avg_group_log_size=122) [2024-10-16 14:57:18.258047] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=732) [2024-10-16 14:57:18.259637] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.259654] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.259660] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545342271246336},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.259724] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.259936] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2121}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202172} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.261442] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061838260937000, start_id=1729061838260937000, end_id=1729061838260937001) [2024-10-16 14:57:18.262290] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061838261997000, start_id=1729061838261997000, end_id=1729061838261997001) [2024-10-16 14:57:18.262614] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.265642] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.265665] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.265672] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618578493296640},{"BIGINT":1729058400},{"VARCHAR":"AÙ™™", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.265751] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.266042] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2143}, to_seq_no={branch:0, seq:1081}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285444} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.268636] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3545, avg_group_log_size=354) [2024-10-16 14:57:18.268904] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:18.269707] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.269723] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.269729] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545342384492544},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.269782] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.274073] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5892}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202175} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.274646] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D2F-0-0] [lt=15] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061838265838, keepalive_ts:-1}) [2024-10-16 14:57:18.274660] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D2F-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061838265838, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:18.274756] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D30-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061838265883, keepalive_ts:-1}) [2024-10-16 14:57:18.274761] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D30-0-0] [lt=6] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061838265883, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:18.279475] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.279492] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.279499] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545342447407104},{"BIGINT":1729058400},{"VARCHAR":"AÙ™™", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.279558] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.279864] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2091}, to_seq_no={branch:0, seq:1031}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202177} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.286197] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3ACED6-0-0] [lt=22] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.743390598297119141e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=871, Throttle Trigger(MB)=523, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.082092942195812313e+01) [2024-10-16 14:57:18.286345] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061838283939000, start_id=1729061838283939000, end_id=1729061838283939001) [2024-10-16 14:57:18.287096] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F2D-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=397, avg_log_submit_to_slide_time=861) [2024-10-16 14:57:18.292624] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4887-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061838260384, keepalive_ts:-1}) [2024-10-16 14:57:18.292646] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4887-0-0] [lt=21] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061838260384, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:18.296713] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.296729] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.296742] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618579663507456},{"BIGINT":1729058400},{"VARCHAR":"AÉ"–8€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.296801] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.297101] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2125}, to_seq_no={branch:0, seq:1070}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285451} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.308539] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1194295296, global_cache_size=1595949056, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:18.308583] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22468][T1003_MediumLoo][T1003][YB4A0A000407-00061E9973DF38DF-0-0] [lt=13] MediumCheckTask(cost_ts=6) [2024-10-16 14:57:18.308848] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash memory, (tenant_id=1002, cache_size=101957632, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:57:18.308986] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Wash time detail, (compute_wash_size_time=92, refresh_score_time=92, wash_time=208) [2024-10-16 14:57:18.309039] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=6] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=5269, mem_used=10398720) [2024-10-16 14:57:18.310685] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:18.310698] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:18.310717] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373489869, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:18.311808] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=5] Cache wash clean map node details(ret=0, clean_node_count=39, clean_time=0, clean_start_pos=251656, clean_num=62914) [2024-10-16 14:57:18.316177] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:18.316200] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:18.316734] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:18.316980] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2133}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285456} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.319560] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285457} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.320516] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:18.321272] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:18.325488] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:19053}, to_seq_no={branch:0, seq:16890}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202182} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.325530] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26] rpc statistics(total_trans_msg_count=154, total_batch_msg_count=0) [2024-10-16 14:57:18.326216] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=71, avg_log_batch_cnt=0, total_group_log_size=112864, avg_group_log_size=1589) [2024-10-16 14:57:18.327585] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1228}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285460} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.329145] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061838328342000, start_id=1729061838328342001, end_id=1729061838328342002) [2024-10-16 14:57:18.329677] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1283}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202183} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.330036] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061838329624000, start_id=1729061838329624000, end_id=1729061838329624001) [2024-10-16 14:57:18.331569] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:18.332497] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061838331769000, start_id=1729061838331769000, end_id=1729061838331769001) [2024-10-16 14:57:18.334478] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061838333898000, start_id=1729061838333898000, end_id=1729061838333898001) [2024-10-16 14:57:18.335536] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=20] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:18.355031] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.359029] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=28, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:18.359048] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:18.361587] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061838361435000, start_id=1729061838361435000, end_id=1729061838361435001) [2024-10-16 14:57:18.362960] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.364022] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202187} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.365869] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C39-0-0] [lt=33] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:18.365909] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C39-0-0] [lt=27] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061838360377001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061838360377001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:18.367079] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C39-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061838366712000, start_id=1729061838366712000, end_id=1729061838366712001) [2024-10-16 14:57:18.368404] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061838367810000, start_id=1729061838367810000, end_id=1729061838367810001) [2024-10-16 14:57:18.371365] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C39-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061838371185000, start_id=1729061838371185000, end_id=1729061838371185001) [2024-10-16 14:57:18.377780] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061838377522000, start_id=1729061838377522000, end_id=1729061838377522001) [2024-10-16 14:57:18.381661] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1084}, to_seq_no={branch:0, seq:1079}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285468} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.383471] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:18.384077] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:18.384098] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=16] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=28020300, tenant_id=1002) [2024-10-16 14:57:18.384124] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=10] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:18.386591] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C02-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:18.394112] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4642}, to_seq_no={branch:0, seq:1458}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285471} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.395133] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=2][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:18.395248] WDIAG read_multi_row (ob_table_access_helper.h:238) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:18.395788] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=13] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:18.396137] INFO delete_row (ob_table_access_helper.h:312) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1002, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061838395258) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061838395258) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:18.396374] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=15] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:18.396578] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=13] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:18.396816] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=6] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:18.398280] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] Cache replace map node details(ret=0, replace_node_count=68, replace_time=0, replace_start_pos=660597, replace_num=31457) [2024-10-16 14:57:18.398297] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=68, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:18.399772] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:18.399788] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53593280, tenant_id=1003) [2024-10-16 14:57:18.399799] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:18.401016] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1117}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285472} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.403877] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1058}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285473} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.406476] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3642}, to_seq_no={branch:0, seq:2584}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202195} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.408369] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=82, avg_log_batch_cnt=0, total_group_log_size=131646, avg_group_log_size=1605) [2024-10-16 14:57:18.410154] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2129}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202196} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.411458] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1070}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285476} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.416377] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF58-0-0] [lt=27] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=1127, avg_log_submit_to_slide_time=2009) [2024-10-16 14:57:18.431642] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1091}, to_seq_no={branch:0, seq:1086}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285483} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.431715] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=15] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:18.431734] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info(lsn={lsn:14538459991}, scn={val:1729061838122276003, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061838358980, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538459991}, next_to_submit_scn_:{val:1729061838122276003, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538459869}, end_lsn:{lsn:14538459991}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061838122276002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061838122276002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:4028041387, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061838122276002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62649984, miss_cnt:32, cache_read_size:8050389076, hit_ratio:"9.999994892259883583e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:18.431791] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=54] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:18.433430] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3162}, to_seq_no={branch:0, seq:1087}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202205} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.441824] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=11] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:18.442467] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:18.442515] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061828711668000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:18.442823] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5801}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202206} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.445109] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A68-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:18.445128] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A68-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:18.448697] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115178224, max_lsn:14538459991, max_scn:{val:1729061838122276002, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6487749074}, reuse_lsn:{lsn:14538459991}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14496516951}}, last_submit_lsn:{lsn:14538459869}, last_submit_end_lsn:{lsn:14538459991}, last_submit_log_id:115178224, last_submit_log_pid:23, max_flushed_lsn:{lsn:14538459869}, max_flushed_end_lsn:{lsn:14538459991}, max_flushed_log_pid:23, committed_end_lsn:{lsn:14538459991}, last_slide_log_id:115178224, last_slide_scn:{val:1729061838122276002, v:0}, last_slide_lsn:{lsn:14538459869}, last_slide_end_lsn:{lsn:14538459991}, last_slide_log_pid:23, last_slide_log_accum_checksum:4028041387, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43cc0530}, [StateMgr]={this:0x2b0e43cc4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, leader:"10.0.4.8:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6488080380}, scan_disk_log_finished:true, last_check_start_id:115178225, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261653045, palf_role_change_cb:0x2b0e43ccb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, reconfig_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, persistent_config_version:{proposal_id:23, config_seq:34}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061837748308, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061837748308, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43cc1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, accepted_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1728259580444569001, v:0}, min_block_id:201, min_block_min_scn:{val:1728204893605525001, v:0}, base_lsn_for_block_gc:{lsn:14427525120}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, log_config_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, log_snapshot_meta:{version:1, base_lsn:{lsn:14427525120}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}}, log_meta_storage:{log_tail:{lsn:1134592}, readable_log_tail:{lsn:1134592}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14561734656}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/meta", dir_fd:166, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65970176, block_header_serialize_buf_:0x2b0e43cc69f8, flashback_version:0}, log_storage:{log_tail:{lsn:14538459991}, readable_log_tail:{lsn:14538459991}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14494629888}, min_scn:{val:1729026040007117003, v:0}, curr_block_id:216, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/log", dir_fd:201, min_block_id:201, max_block_id:217, curr_writable_block_id:216}, logical_block_size_:67104768, curr_block_writable_size_:23274665, block_header_serialize_buf_:0x2b0e43cc86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261652788, this:0x2b0e43cc4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:22, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:21, majority_max_lsn:{lsn:6487893593}, saved_end_lsn:{lsn:6487893593}, last_submit_prepare_req_time_us:1722494261653224, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43cc49b0}) [2024-10-16 14:57:18.448845] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=145] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:18.448857] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=115178225, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:18.456994] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.458330] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:18.459272] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4D76-0-0] [lt=8] init point thread id with(&point=0x55e3435c24c0, idx_=473, point=[thread id=106969, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=106969) [2024-10-16 14:57:18.460471] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2126}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202208} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.462263] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061838461872000, start_id=1729061838461872000, end_id=1729061838461872001) [2024-10-16 14:57:18.463026] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.464365] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202209} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.466725] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1053}, to_seq_no={branch:0, seq:9}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285491} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.468833] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22267][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2120}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202210} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.469163] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=342, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:18.469197] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=30] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=27, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:18.471091] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061838470521000, start_id=1729061838470521000, end_id=1729061838470521001) [2024-10-16 14:57:18.473371] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:220937495, max_lsn:213057208298, max_scn:{val:1729061838472664000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:60464363030}, reuse_lsn:{lsn:213057206674}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:213015265258}}, last_submit_lsn:{lsn:213057206674}, last_submit_end_lsn:{lsn:213057208298}, last_submit_log_id:220937495, last_submit_log_pid:88, max_flushed_lsn:{lsn:213057205050}, max_flushed_end_lsn:{lsn:213057206674}, max_flushed_log_pid:88, committed_end_lsn:{lsn:213057206674}, last_slide_log_id:220937494, last_slide_scn:{val:1729061838470521002, v:0}, last_slide_lsn:{lsn:213057205050}, last_slide_end_lsn:{lsn:213057206674}, last_slide_log_pid:88, last_slide_log_accum_checksum:1307845528, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0dac0530}, [StateMgr]={this:0x2b0e0dac4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, leader:"10.0.4.8:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:60468525403}, scan_disk_log_finished:true, last_check_start_id:220937495, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261639483, palf_role_change_cb:0x2b0e0dacb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, reconfig_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, persistent_config_version:{proposal_id:88, config_seq:21}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061835670652, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061835670652, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0dac1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, accepted_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1729009096591759001, v:0}, min_block_id:3154, min_block_min_scn:{val:1729006455168279002, v:0}, base_lsn_for_block_gc:{lsn:212587905024}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, log_config_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, log_snapshot_meta:{version:1, base_lsn:{lsn:212587905024}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}}, log_meta_storage:{log_tail:{lsn:5767168}, readable_log_tail:{lsn:5767168}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213057638400}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/meta", dir_fd:175, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61337600, block_header_serialize_buf_:0x2b0e0dac69f8, flashback_version:0}, log_storage:{log_tail:{lsn:213057206674}, readable_log_tail:{lsn:213057206674}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:212990533632}, min_scn:{val:1729059200214792001, v:0}, curr_block_id:3174, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/log", dir_fd:177, min_block_id:3154, max_block_id:3175, curr_writable_block_id:3174}, logical_block_size_:67104768, curr_block_writable_size_:431726, block_header_serialize_buf_:0x2b0e0dac86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261637707, this:0x2b0e0dac4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:87, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:86, majority_max_lsn:{lsn:60466038076}, saved_end_lsn:{lsn:60466038076}, last_submit_prepare_req_time_us:1722494261639634, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0dac49b0}) [2024-10-16 14:57:18.473524] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=149] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:18.473535] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=220937495, log_task={header:{begin_lsn:{lsn:213057206674}, end_lsn:{lsn:213057208298}, log_id:220937495, min_scn:{val:1729061838472664000, v:0}, max_scn:{val:1729061838472664000, v:0}, data_len:1568, proposal_id:88, prev_lsn:{lsn:213057205050}, prev_proposal_id:88, committed_end_lsn:{lsn:213057206674}, data_checksum:954828208, accum_checksum:1957194546, is_padding_log:false, is_raw_write:false}, state_map:{val:26}, ref_cnt:0, gen_ts:1729061838473213, freeze_ts:1729061838473213, submit_ts:1729061838473216, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:18.473558] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:211146576, max_lsn:204022156435, max_scn:{val:1729061838470521000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57922170954}, reuse_lsn:{lsn:204022156435}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:203980213395}}, last_submit_lsn:{lsn:204022154812}, last_submit_end_lsn:{lsn:204022156435}, last_submit_log_id:211146576, last_submit_log_pid:130, max_flushed_lsn:{lsn:204022154812}, max_flushed_end_lsn:{lsn:204022156435}, max_flushed_log_pid:130, committed_end_lsn:{lsn:204022156435}, last_slide_log_id:211146576, last_slide_scn:{val:1729061838470521000, v:0}, last_slide_lsn:{lsn:204022154812}, last_slide_end_lsn:{lsn:204022156435}, last_slide_log_pid:130, last_slide_log_accum_checksum:1632188561, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1ec530}, [StateMgr]={this:0x2b0e0f1f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:211146577, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261539368, palf_role_change_cb:0x2b0e0f1f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, reconfig_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, persistent_config_version:{proposal_id:130, config_seq:17}, ms_ack_list:0, resend_config_version:{proposal_id:130, config_seq:17}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258231773, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258231773, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, accepted_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261739861, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1729008439619722001, v:0}, min_block_id:3020, min_block_min_scn:{val:1729005678665693002, v:0}, base_lsn_for_block_gc:{lsn:203595866112}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, log_config_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, log_snapshot_meta:{version:1, base_lsn:{lsn:203595866112}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}}, log_meta_storage:{log_tail:{lsn:6397952}, readable_log_tail:{lsn:6397952}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:204065599488}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/meta", dir_fd:145, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:60706816, block_header_serialize_buf_:0x2b0e0f1f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:204022156435}, readable_log_tail:{lsn:204022156435}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:203998494720}, min_scn:{val:1729060878194301003, v:0}, curr_block_id:3040, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/log", dir_fd:173, min_block_id:3020, max_block_id:3041, curr_writable_block_id:3040}, logical_block_size_:67104768, curr_block_writable_size_:43443053, block_header_serialize_buf_:0x2b0e0f1f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261539352, this:0x2b0e0f1f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:130, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:129, majority_max_lsn:{lsn:57923794664}, saved_end_lsn:{lsn:57923794664}, last_submit_prepare_req_time_us:1722494261539574, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1f09b0}) [2024-10-16 14:57:18.473681] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=122] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838471340, last_flushed_end_lsn:{lsn:204022156435}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838471728, last_flushed_end_lsn:{lsn:204022156435}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838471480, last_flushed_end_lsn:{lsn:204022156435}}]) [2024-10-16 14:57:18.473690] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=211146577, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838471340, last_flushed_end_lsn:{lsn:204022156435}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838471728, last_flushed_end_lsn:{lsn:204022156435}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838471480, last_flushed_end_lsn:{lsn:204022156435}}]) [2024-10-16 14:57:18.473709] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:218496666, max_lsn:207588560071, max_scn:{val:1729061838464003000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57807896114}, reuse_lsn:{lsn:207588560071}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:207546617031}}, last_submit_lsn:{lsn:207588558447}, last_submit_end_lsn:{lsn:207588560071}, last_submit_log_id:218496666, last_submit_log_pid:81, max_flushed_lsn:{lsn:207588558447}, max_flushed_end_lsn:{lsn:207588560071}, max_flushed_log_pid:81, committed_end_lsn:{lsn:207588560071}, last_slide_log_id:218496666, last_slide_scn:{val:1729061838464003000, v:0}, last_slide_lsn:{lsn:207588558447}, last_slide_end_lsn:{lsn:207588560071}, last_slide_log_pid:81, last_slide_log_accum_checksum:1764483307, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0db32530}, [StateMgr]={this:0x2b0e0db36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, leader:"10.0.4.9:2890", leader_epoch:7, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:57814812323}, scan_disk_log_finished:true, last_check_start_id:218496667, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433991656, palf_role_change_cb:0x2b0e0db3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, reconfig_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, persistent_config_version:{proposal_id:81, config_seq:20}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061836672348, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061836672348, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0db33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, accepted_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1729009518284639001, v:0}, min_block_id:3073, min_block_min_scn:{val:1729006819670466001, v:0}, base_lsn_for_block_gc:{lsn:207152418816}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, log_config_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, log_snapshot_meta:{version:1, base_lsn:{lsn:207152418816}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}}, log_meta_storage:{log_tail:{lsn:5660672}, readable_log_tail:{lsn:5660672}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207622152192}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/meta", dir_fd:180, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61444096, block_header_serialize_buf_:0x2b0e0db389f8, flashback_version:0}, log_storage:{log_tail:{lsn:207588560071}, readable_log_tail:{lsn:207588560071}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207555047424}, min_scn:{val:1729060516440786001, v:0}, curr_block_id:3093, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/log", dir_fd:182, min_block_id:3073, max_block_id:3094, curr_writable_block_id:3093}, logical_block_size_:67104768, curr_block_writable_size_:33592121, block_header_serialize_buf_:0x2b0e0db3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433990608, this:0x2b0e0db36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:80, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:79, majority_max_lsn:{lsn:57811903042}, saved_end_lsn:{lsn:57811903042}, last_submit_prepare_req_time_us:1722494433991756, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0db369b0}) [2024-10-16 14:57:18.473810] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=100] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:18.473824] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=11] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=218496667, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:18.473996] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106793][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8E2-0-0] [lt=7] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:18.474015] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106793][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8E2-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:18.474031] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106793][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8E2-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:18.480649] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3178}, to_seq_no={branch:0, seq:1036}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285497} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.482420] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3C35-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=85, avg_log_gen_to_freeze_time=5, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=321, avg_log_submit_to_slide_time=596) [2024-10-16 14:57:18.486501] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061838486430000, start_id=1729061838486430000, end_id=1729061838486430001) [2024-10-16 14:57:18.487559] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F4E-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:18.487573] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F4E-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11825, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:18.487571] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:18.487578] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F4E-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=365, avg_log_submit_to_slide_time=11818) [2024-10-16 14:57:18.501069] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:57:18.501932] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=15] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:18.501965] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=33] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:18.501998] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=16] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:18.502073] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061845590307000, latest_log_ts:{val:1729061828964666001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:18.502132] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=57] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:18.502159] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:18.502167] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061828964666001, v:0}) [2024-10-16 14:57:18.502170] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:18.502174] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:18.502182] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:18.502211] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:18.502219] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:18.502225] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:18.512121] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1587625984, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:18.513452] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061838512831000, start_id=1729061838512831000, end_id=1729061838512831001) [2024-10-16 14:57:18.513480] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061838513476], gts:1729061838512831000, latest_srr:[mts=1729061838513476]}) [2024-10-16 14:57:18.515077] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1034}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285507} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.520391] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3176}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202221} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.524450] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1111}, to_seq_no={branch:0, seq:1106}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285509} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.527087] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285510} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.529386] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22267][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2162}, to_seq_no={branch:0, seq:1102}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202223} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.529712] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:18.530081] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:11}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285511} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.530093] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061838528816000, start_id=1729061838528816000, end_id=1729061838528816001) [2024-10-16 14:57:18.531564] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:57:18.531580] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4382) [2024-10-16 14:57:18.531595] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1441575413, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:18.532758] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061838532312000, start_id=1729061838532312000, end_id=1729061838532312001) [2024-10-16 14:57:18.532809] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:18.532831] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:18.534496] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061838533406000, start_id=1729061838533406000, end_id=1729061838533406001) [2024-10-16 14:57:18.534648] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=6] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:18.534704] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061838534608000, start_id=1729061838534608000, end_id=1729061838534608001) [2024-10-16 14:57:18.535202] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2298}, to_seq_no={branch:0, seq:1095}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285512} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.537165] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2262}, to_seq_no={branch:0, seq:1203}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202224} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.537238] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=902713344, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=171028480, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:18.537286] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=22] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:18.537297] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:18.537330] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=969699328, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=104042496, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:18.537364] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:18.537372] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:18.537401] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=938364928, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=135376896, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:18.537439] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:18.537446] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011852, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:18.537489] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=6] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=956841984, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=116899840, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:18.537527] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:18.537535] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:18.537979] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2121}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285513} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.540901] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4D95-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=91, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:18.540912] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4D95-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=91, stat_interval=1000000, avg (count/cost)=719, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:18.540914] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=46] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=146247, hit_cnt=87, read_cnt=87, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:18.541517] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285515} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.545068] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285516} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.548937] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=16][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:18.548831] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109593494, max_lsn:13645391119, max_scn:{val:1729061838512831000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6198843846}, reuse_lsn:{lsn:13645391119}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:13603448079}}, last_submit_lsn:{lsn:13645390997}, last_submit_end_lsn:{lsn:13645391119}, last_submit_log_id:109593494, last_submit_log_pid:17, max_flushed_lsn:{lsn:13645390997}, max_flushed_end_lsn:{lsn:13645391119}, max_flushed_log_pid:17, committed_end_lsn:{lsn:13645391119}, last_slide_log_id:109593494, last_slide_scn:{val:1729061838512831000, v:0}, last_slide_lsn:{lsn:13645390997}, last_slide_end_lsn:{lsn:13645391119}, last_slide_log_pid:17, last_slide_log_accum_checksum:290449244, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431ec530}, [StateMgr]={this:0x2b0e431f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109593495, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261552988, palf_role_change_cb:0x2b0e431f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:14, lock_type:0, added_list:0, removed_list:0, type:"UNLOCK_CONFIG_CHANGE"}, state:0, checking_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:30}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:30}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494257649102, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494257649102, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261753456, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1728296971089337004, v:0}, min_block_id:189, min_block_min_scn:{val:1728240898542283004, v:0}, base_lsn_for_block_gc:{lsn:13555163136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:13555163136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}}, log_meta_storage:{log_tail:{lsn:1011712}, readable_log_tail:{lsn:1011712}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13689372672}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/meta", dir_fd:198, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66093056, block_header_serialize_buf_:0x2b0e431f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:13645391119}, readable_log_tail:{lsn:13645391119}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13622267904}, min_scn:{val:1729042537979987002, v:0}, curr_block_id:203, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/log", dir_fd:200, min_block_id:189, max_block_id:204, curr_writable_block_id:203}, logical_block_size_:67104768, curr_block_writable_size_:43981553, block_header_serialize_buf_:0x2b0e431f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261552391, this:0x2b0e431f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:6198988121}, saved_end_lsn:{lsn:6198988121}, last_submit_prepare_req_time_us:1722494261553147, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431f09b0}) [2024-10-16 14:57:18.548974] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=140] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838516296, last_flushed_end_lsn:{lsn:13645391119}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838516423, last_flushed_end_lsn:{lsn:13645391119}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838535611, last_flushed_end_lsn:{lsn:13645391119}}]) [2024-10-16 14:57:18.548991] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=109593495, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838516296, last_flushed_end_lsn:{lsn:13645391119}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838516423, last_flushed_end_lsn:{lsn:13645391119}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061838535611, last_flushed_end_lsn:{lsn:13645391119}}]) [2024-10-16 14:57:18.552123] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285519} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.557897] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.560936] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=6, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=26, avg_log_submit_to_flush_time=973, avg_log_submit_to_slide_time=1327) [2024-10-16 14:57:18.563093] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:18.563121] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=27] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:18.563127] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.565404] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:18.565418] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:18.565493] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061838565462000, start_id=1729061838565462000, end_id=1729061838565462001) [2024-10-16 14:57:18.568332] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285522} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.568625] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.569477] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.569633] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.569640] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.571163] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D56-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:18.571186] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D56-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061838122276004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061838122276004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:18.572438] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D56-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061838572191000, start_id=1729061838572191000, end_id=1729061838572191001) [2024-10-16 14:57:18.575286] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D56-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061838574312000, start_id=1729061838574312000, end_id=1729061838574312001) [2024-10-16 14:57:18.575620] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG IO WORKER QUEUE SIZE](cur_stat_count=1, stat_interval=1000000, avg (count/cost)=1, this=0x2b0e0d9bb7a8, extra_info="log_io_worker_num:4, cb_thread_pool_tg_id:485") [2024-10-16 14:57:18.576069] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2142}, to_seq_no={branch:0, seq:1052}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285524} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.577404] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061838576422000, start_id=1729061838576422000, end_id=1729061838576422001) [2024-10-16 14:57:18.578595] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1001}, {id:1003}]) [2024-10-16 14:57:18.578794] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061838578630000, start_id=1729061838578630000, end_id=1729061838578630001) [2024-10-16 14:57:18.580763] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202233} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.587423] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2109}, to_seq_no={branch:0, seq:1036}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285528} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.591846] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1023}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202236} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.594065] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1069}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285530} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.595848] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=25] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4495, mem_used=8318976) [2024-10-16 14:57:18.596377] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1048}, to_seq_no={branch:0, seq:1043}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285531} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.598893] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1138}, to_seq_no={branch:0, seq:1132}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202238} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.599202] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:12}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285532} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.600071] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=574, replace_time=0, replace_start_pos=692054, replace_num=31457) [2024-10-16 14:57:18.600099] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=574, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:18.602131] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285533} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.609883] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2081}, to_seq_no={branch:0, seq:1022}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202242} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.609977] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285536} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.613414] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=164120) [2024-10-16 14:57:18.613853] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=28] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=164120, hit_cnt=95, read_cnt=95, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:18.614703] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1023}, to_seq_no={branch:0, seq:1018}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202244} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.616019] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:18.616035] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] try gc retain ctx [2024-10-16 14:57:18.616038] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:18.616054] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=2] get_max_applied_scn(scn={val:1729061838512831000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493432260}, palf_committed_end_scn_:{val:1729061838512831000, v:0}, last_check_scn_:{val:1729061838512831000, v:0}, max_applied_cb_scn_:{val:1729061838512831000, v:0}}) [2024-10-16 14:57:18.616071] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:18.616709] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=43] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061835510227002, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850721265000, latest_log_ts:{val:1729061834119131001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:18.616789] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=41] get_max_applied_scn(scn={val:1729061838512831000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645391119}, palf_committed_end_scn_:{val:1729061838512831000, v:0}, last_check_scn_:{val:1729061838512831000, v:0}, max_applied_cb_scn_:{val:1729061838512831000, v:0}}) [2024-10-16 14:57:18.616799] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:18.617435] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061835510227002, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:18.618108] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=32] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061835510227002, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:18.618144] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=30] get_min_unreplayed_log_info(lsn={lsn:14352346471}, scn={val:1729061838123335005, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061838587807, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352346471}, next_to_submit_scn_:{val:1729061838123335005, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352346349}, end_lsn:{lsn:14352346471}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061838123335004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061838123335004, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:2813842154, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061838123335004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656721, miss_cnt:17, cache_read_size:8050136692, hit_ratio:"9.999997286804174212e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:18.618779] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=33] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061835510227002, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:18.620010] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202245} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.622860] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202246} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.623371] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285541} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.623477] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B6F-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061838623444000, start_id=1729061838623444000, end_id=1729061838623444001) [2024-10-16 14:57:18.623728] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B73-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061838623444000, start_id=1729061838623444000, end_id=1729061838623444001) [2024-10-16 14:57:18.623861] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B70-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061838623444000, start_id=1729061838623444000, end_id=1729061838623444001) [2024-10-16 14:57:18.631500] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1055}, to_seq_no={branch:0, seq:1050}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285544} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.635646] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2633-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510306504039, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410507, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:18.636757] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C8F-0-0] [lt=15][errcode=-4201] hashtable not init [2024-10-16 14:57:18.636766] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C8F-0-0] [lt=10][errcode=-4016] get cursor info failed(cursor_id=1108, get_sessid()=3221565766) [2024-10-16 14:57:18.637000] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061838636750000, start_id=1729061838636750000, end_id=1729061838636750001) [2024-10-16 14:57:18.637068] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1068}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285545} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.639763] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1040}, to_seq_no={branch:0, seq:1035}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285546} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.645323] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=108, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:18.645531] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=107, stat_interval=1000000, avg (count/cost)=254, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:18.645546] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=108, stat_interval=1000000, avg (count/cost)=38, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:18.645577] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=108, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:18.646526] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1074}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285549} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.648844] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:18.648860] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:18.648871] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=10] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:18.648877] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:18.650975] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=104, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}]) [2024-10-16 14:57:18.651110] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=133] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:18.651178] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:18.657948] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.659862] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1056}, to_seq_no={branch:0, seq:1051}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285554} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.663155] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.663688] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1049}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285556} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.666595] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061838666328000, start_id=1729061838666328000, end_id=1729061838666328001) [2024-10-16 14:57:18.666667] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1085}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285557} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.666869] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=10] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:18.667685] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061838667388000, start_id=1729061838667388000, end_id=1729061838667388001) [2024-10-16 14:57:18.669629] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061838668606000, start_id=1729061838668606001, end_id=1729061838668606002) [2024-10-16 14:57:18.670003] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1109}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202262} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.670879] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1057}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285558} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.671169] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061838670772000, start_id=1729061838670772000, end_id=1729061838670772001) [2024-10-16 14:57:18.673396] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1075}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285559} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.678779] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1086}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202265} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.680177] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061838679321000, start_id=1729061838679321001, end_id=1729061838679321002) [2024-10-16 14:57:18.685281] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=12] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:18.686808] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061838684661000, start_id=1729061838684661000, end_id=1729061838684661001) [2024-10-16 14:57:18.686881] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=29] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=20, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:18.686912] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=361, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:18.686918] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=31] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4101) [2024-10-16 14:57:18.686926] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:18.686924] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:18.686936] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:18.686981] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:18.687117] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3979) [2024-10-16 14:57:18.687140] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3979) [2024-10-16 14:57:18.687500] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F31-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:18.687508] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F31-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=827, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:18.689793] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1086}, to_seq_no={branch:0, seq:1079}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285567} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.689793] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1086}, to_seq_no={branch:0, seq:1080}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202269} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.692307] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:12}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202270} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.695342] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:18.699366] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1042}, to_seq_no={branch:0, seq:1037}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202273} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.705933] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202275} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.712423] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1204781056, global_cache_size=1587625984, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:18.716120] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:18.716140] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:18.717144] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=3342959, this={dio_aligned_buf:{buf_write_offset:1439, buf_padding_size:2657, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80962444, truncate_used_ts:29163380}, log_block_size:67108864, dir_fd:148, io_fd:171}, aligned_buf_len=4096, aligned_block_offset=3342336, buf_len=816, total_write_size_=98543, total_write_size_after_dio_=790528, ob_pwrite_used_ts=54748, count=169, each_pwrite_cost=323) [2024-10-16 14:57:18.720113] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2242}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202279} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.720433] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:18.723468] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=25] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:18.724543] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1055}, to_seq_no={branch:0, seq:1050}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285579} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.729272] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202283} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.730228] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061838729860000, start_id=1729061838729860000, end_id=1729061838729860001) [2024-10-16 14:57:18.731045] INFO [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:194) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1}, delta=178276, log_service_ts=1729061838552680, min_tx_service_ts=4611686018427387, timestamp={val:1729061838552680001, v:0}) [2024-10-16 14:57:18.731069] INFO [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:214) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=21] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=0, Near_To_GTS_Cnt=23, Other_Error_Cnt=0, Submit_Succ_Cnt=154, last_scn="{val:1729061836641544000, v:0}", last_lsn={lsn:17959661983}, last_gts={val:1729061838648347000, v:0}, min_start_scn="{val:0, v:0}", min_start_status=1, sys_ls_end_scn={val:0, v:0}) [2024-10-16 14:57:18.732804] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061838732055000, start_id=1729061838732055000, end_id=1729061838732055001) [2024-10-16 14:57:18.734612] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061838734179000, start_id=1729061838734179000, end_id=1729061838734179001) [2024-10-16 14:57:18.735612] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=15] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:18.739379] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285584} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.741785] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1107}, to_seq_no={branch:0, seq:8}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202286} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.744895] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:12}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202287} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.748652] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=2] gc stale ls task succ [2024-10-16 14:57:18.748679] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=14] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:18.748690] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:18.752239] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1057}, to_seq_no={branch:0, seq:1051}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202291} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.752390] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2108}, to_seq_no={branch:0, seq:1050}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285590} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.756514] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1057}, to_seq_no={branch:0, seq:1052}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202293} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.757992] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A89-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:18.758030] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.758173] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=354, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:18.758187] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:18.758212] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:18.758302] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=15][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5024, dropped:302, tid:12153}]) [2024-10-16 14:57:18.758620] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62649988, miss_cnt:32, cache_read_size:8050389564, hit_ratio:"9.999994892260209989e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:18.759171] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=38][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.759186] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.759192] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618587477495808},{"BIGINT":1729058400},{"VARCHAR":"?ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.759247] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.759523] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1104}, to_seq_no={branch:0, seq:1099}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202294} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.762228] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.762246] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.762253] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618587628490752},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.762328] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=44][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.762567] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1036}, to_seq_no={branch:0, seq:1031}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202295} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.763180] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.764369] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=121, stat_interval=1000000, avg (count/cost)=242, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:18.764387] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=121, stat_interval=1000000, avg (count/cost)=39, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:18.764414] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=121, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:18.764947] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.764961] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.764968] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830053053026304},{"BIGINT":1729058400},{"VARCHAR":"AÐ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.765031] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=39][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.765470] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1200}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285594} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.767384] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061838767285000, start_id=1729061838767285000, end_id=1729061838767285001) [2024-10-16 14:57:18.767385] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257662, offset=39004806}, end_cursor:ObLogCursor{file_id=197, log_id=49257663, offset=39005089}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd291c050, len:1402, log_data_len:283, seq:49257662, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:18.767696] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:18.768164] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:18.769632] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB358-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061838769386000, start_id=1729061838769386000, end_id=1729061838769386001) [2024-10-16 14:57:18.770075] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.770090] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.770096] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618587867566080},{"BIGINT":1729058400},{"VARCHAR":"@Â1", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.770179] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.770562] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3154}, to_seq_no={branch:0, seq:2095}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202297} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.770871] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257663, offset=39005089}, end_cursor:ObLogCursor{file_id=197, log_id=49257664, offset=39005371}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2af8050, len:1119, log_data_len:282, seq:49257663, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:18.771044] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=33][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.771056] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.771062] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910196337463296},{"BIGINT":1729058400},{"VARCHAR":"Aà", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.771107] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.771341] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285596} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.772266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061838771504000, start_id=1729061838771504000, end_id=1729061838771504001) [2024-10-16 14:57:18.773813] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257664, offset=39005371}, end_cursor:ObLogCursor{file_id=197, log_id=49257665, offset=39005653}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bc4050, len:837, log_data_len:282, seq:49257664, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:18.774087] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257665, offset=39005653}, end_cursor:ObLogCursor{file_id=197, log_id=49257666, offset=39005935}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bb8050, len:555, log_data_len:282, seq:49257665, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:18.774152] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656722, miss_cnt:17, cache_read_size:8050136814, hit_ratio:"9.999997286804217511e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:18.774963] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=124, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:18.775500] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=40] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560600, miss_cnt:1, cache_read_size:152589760325, hit_ratio:"9.999999928346539058e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:18.777630] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061838775797000, start_id=1729061838775797000, end_id=1729061838775797001) [2024-10-16 14:57:18.777689] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=25, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:18.777701] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=10600, avg_group_log_size=530) [2024-10-16 14:57:18.777710] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11150) [2024-10-16 14:57:18.777717] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:18.777926] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=280, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:18.777934] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10600) [2024-10-16 14:57:18.777947] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=96, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:18.777959] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11150) [2024-10-16 14:57:18.777970] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:18.778108] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D569F-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:18.778114] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D569F-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=948, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:18.778117] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D569F-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=6, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=530, avg_log_submit_to_slide_time=912) [2024-10-16 14:57:18.779222] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:18.780427] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=23] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601296, offset=9599736}, end_cursor:ObLogCursor{file_id=192, log_id=47601297, offset=9600020}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a58050, len:1288, log_data_len:284, seq:47601296, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:18.781853] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.781865] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.781871] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618588458962944},{"BIGINT":1729058400},{"VARCHAR":"@»Ä", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.781921] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.782181] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3125}, to_seq_no={branch:0, seq:2109}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202300} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.782221] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48D-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061838782081000, start_id=1729061838782081000, end_id=1729061838782081001) [2024-10-16 14:57:18.783280] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601297, offset=9600020}, end_cursor:ObLogCursor{file_id=192, log_id=47601298, offset=9600294}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b7c050, len:1004, log_data_len:274, seq:47601297, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:18.784191] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061838783160000, start_id=1729061838783160000, end_id=1729061838783160001) [2024-10-16 14:57:18.785131] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=50] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598196, miss_cnt:9, cache_read_size:149777026809, hit_ratio:"9.999999345921699589e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:18.785592] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601298, offset=9600294}, end_cursor:ObLogCursor{file_id=192, log_id=47601299, offset=9600568}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a48050, len:730, log_data_len:274, seq:47601298, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:18.785801] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=24] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601299, offset=9600568}, end_cursor:ObLogCursor{file_id=192, log_id=47601300, offset=9600842}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2abc050, len:456, log_data_len:274, seq:47601299, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:18.787109] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.787143] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=32][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.787150] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910196362629120},{"BIGINT":1729058400},{"VARCHAR":"?ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.787210] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.787413] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:8}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285601} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.788292] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.788320] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.788335] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.788348] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.788548] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.788562] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.788576] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.788579] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.788802] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.788823] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.789711] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:18.793897] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA869-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:18.793922] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA869-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061838789698001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061838789698001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:18.795108] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA869-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061838794980000, start_id=1729061838794980000, end_id=1729061838794980001) [2024-10-16 14:57:18.797261] WDIAG [PALF] is_prev_log_pid_match (log_sliding_window.cpp:3884) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=11][errcode=-4018] get_prev_log_info_ failed(tmp_ret=-4018, log_id=218496733, palf_id=1003, self="10.0.4.7:2890", log_id=218496733, lsn={lsn:207588667240}, prev_lsn={lsn:207588665616}, prev_log_pid=81) [2024-10-16 14:57:18.798257] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.798270] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.798276] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830053178855424},{"BIGINT":1729058400},{"VARCHAR":"AÐ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.798331] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.798721] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3502}, to_seq_no={branch:0, seq:2440}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285604} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.798837] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA869-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061838798504000, start_id=1729061838798504000, end_id=1729061838798504001) [2024-10-16 14:57:18.799396] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:18.801903] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=483, replace_time=0, replace_start_pos=723511, replace_num=31457) [2024-10-16 14:57:18.801917] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=483, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:18.804159] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.804173] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.804179] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618589549481985},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.804230] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.804513] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1037}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202306} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.806743] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.806778] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=34][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.806785] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545345651855361},{"BIGINT":1729058400},{"VARCHAR":"Aô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.806839] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.807154] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2143}, to_seq_no={branch:0, seq:1086}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285607} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.809092] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.809106] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.809112] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618589729837056},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.809159] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.809356] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1052}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202308} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.810333] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=10][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:18.810393] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=14][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1001 and flag like "%DUPLICATE%", tenant_id=1001, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:18.813592] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:18.813656] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1 and flag like "%DUPLICATE%", tenant_id=1, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:18.815523] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1001}, {id:1003}]) [2024-10-16 14:57:18.815867] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.815884] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.815890] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618589956329472},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.815954] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.816415] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=201, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:18.816385] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2359}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202310} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.816431] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=527, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:18.818378] INFO [LIB] stat (utility.h:1165) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=42] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=125, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:18.818394] INFO [LIB] stat (utility.h:1165) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=125, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:18.818587] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=202975) [2024-10-16 14:57:18.818616] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=202975) [2024-10-16 14:57:18.819450] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.819464] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.819470] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049492062208},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.819523] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.819703] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202311} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.830196] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.830213] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.830220] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830053292101632},{"BIGINT":1729058400},{"VARCHAR":"B", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.830291] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.830469] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1050}, to_seq_no={branch:0, seq:1044}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285615} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.831489] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.831504] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.831511] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049554976768},{"BIGINT":1729058400},{"VARCHAR":"@òÀö =p¤@òÀøL˜_@òÀú/Ÿ¾@òÀü2ÊW¨@òÀýù kº@òÁ \ú¬Ú@òÁ y¦µ @òÁ¿±[W@òÁö”Ft@òÁ Îp:û@òÁ"Ý/ @òÁ$àuöý@òÁ'FÜ]@òÁ)4ê@òÁH¬1@òÁJ k¹Œ@òÁL Iº^@òÁNŠÚ¹õ@òÁPU2a|@òÁRlô!@òÁT§æ›@òÁVõÂ\@òÁ‚škP±@òÁ„ÿ—$@òÁ‡ =p¤@òÁ‰¹#¢œ@òÁ‹z—@òÁ?@òÁŽÿ.Hé@òÁ¸o", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.831567] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.831788] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202316} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.831842] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061838830783000, start_id=1729061838830783001, end_id=1729061838830783002) [2024-10-16 14:57:18.838859] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.838879] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.838886] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049580142592},{"BIGINT":1729058400},{"VARCHAR":"@٠ƱÄ2Ê@Ù ÈmÅÖ9@Ù ÈmÅÖ9@Ù É«ŸU›@Ù Õb¶®@Ù òߤ?æ@Ù .Hè§@Ù `$ x@Ù g·éù@Ù n䎊r@Ù yN;Í6@Ù É…ðo@Ù • '»@Ù šT`ªe@Ù ñº^5?@Ù ö¶®}V@Ù öÂ&€@Ù öÊW§‡@Ù ÷’:)Ç@Ù øzáG®@Ù ú¸ºÇ@Ù üXy=Ù@Ù wŸ¾w@Ù ‰¥œà@Ù ŠîcŠ@Ù ‹ûµt@Ù ™™™š@Ù ŽQÎ@Ù Žz—@Ù óÕfÏB@Ù üªdÂø@Ù ü­«ŸV@Ù ü­«ŸV@Ù ÿ»/ìW@Ù ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.838951] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.839346] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1043}, to_seq_no={branch:0, seq:1038}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202317} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.843833] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG IO WORKER QUEUE SIZE](cur_stat_count=2, stat_interval=1000000, avg (count/cost)=1, this=0x2b0e0d9bb068, extra_info="log_io_worker_num:4, cb_thread_pool_tg_id:485") [2024-10-16 14:57:18.844573] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.844588] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.844594] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910255061913600},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.844655] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.844867] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285621} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.848018] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.848034] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.848040] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545341667266560},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.848151] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.848305] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.848316] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.848324] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1031}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285622} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.848321] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618590157656064},{"BIGINT":1729058400},{"VARCHAR":"@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð@@ÁTÉ…ð¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.848362] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.848528] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1033}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202320} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.849408] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3007, mem_used=6239232) [2024-10-16 14:57:18.850572] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.850587] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.850593] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049605308416},{"BIGINT":1729058400},{"VARCHAR":"@¿«Aÿ.Hé@¿«B:)Çz@¿«B‚@·€@¿«B\(ö@¿«B\(ö@¿«J~ùÛ#@¿«J~ùÛ#@¿«M/Ÿ¾@¿«M—ö+k@¿«M—ö+k@¿«M¸ºÇ@¿«M¸ºÇ@¿«M¸ºÇ@¿«MæšÔ,@¿«o\(õÃ@¿«ob¶®}@¿«ob¶®}@¿«oiDg8@¿«oiDg8@¿«oÒòå@¿«oØ­«Ÿ@¿«oØ­«Ÿ@¿« -à @¿« -à @¿« -à @¿« 4mÅÖ@¿« :û~‘@¿« N¤¨Á@¿« N¤¨Á@¿«ÄÂø7µ@¿«ÄÂø7µ@¿«ÄÂø7µ@¿«ÄÂø7µ@¿«ÄÂø7µ@¿«ÄÂø7µ@¿«Å '»@¿«Å '»@¿«ç§†Â'@¿«ê’£S@¿«ëô‡ü¹@¿«ì¾ß¤@¿«ì¾ß¤@¿«ì¾ß¤@¿«ìPHð@¿«ìPHð@¿«ìVÕÏ«@¿«þzáH@¿«ÿ¼@¿«ÿ¼@¿«ÿ¼@¿«ÿ¼@¿«ÿ¼@¿¬P:û~‘@¿¬P:û~‘@¿¬P:û~‘@¿®¦µ @¿®¦µ ¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.850658] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=39][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.850852] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202321} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.851288] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115172921, max_lsn:14352346715, max_scn:{val:1729061838625138001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6301879685}, reuse_lsn:{lsn:14352346715}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14310403675}}, last_submit_lsn:{lsn:14352346593}, last_submit_end_lsn:{lsn:14352346715}, last_submit_log_id:115172921, last_submit_log_pid:17, max_flushed_lsn:{lsn:14352346593}, max_flushed_end_lsn:{lsn:14352346715}, max_flushed_log_pid:17, committed_end_lsn:{lsn:14352346715}, last_slide_log_id:115172921, last_slide_scn:{val:1729061838625138001, v:0}, last_slide_lsn:{lsn:14352346593}, last_slide_end_lsn:{lsn:14352346715}, last_slide_log_pid:17, last_slide_log_accum_checksum:2199257047, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43d32530}, [StateMgr]={this:0x2b0e43d36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.9:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6302436797}, scan_disk_log_finished:true, last_check_start_id:115172922, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433837169, palf_role_change_cb:0x2b0e43d3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, reconfig_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, persistent_config_version:{proposal_id:17, config_seq:26}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061836747080, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061836747080, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43d33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1728301961314447004, v:0}, min_block_id:199, min_block_min_scn:{val:1728247158357854003, v:0}, base_lsn_for_block_gc:{lsn:14293315584}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:14293315584}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}}, log_meta_storage:{log_tail:{lsn:1064960}, readable_log_tail:{lsn:1064960}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14360420352}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/meta", dir_fd:192, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66039808, block_header_serialize_buf_:0x2b0e43d389f8, flashback_version:0}, log_storage:{log_tail:{lsn:14352346715}, readable_log_tail:{lsn:14352346715}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14293315584}, min_scn:{val:1729013741169122001, v:0}, curr_block_id:213, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/log", dir_fd:207, min_block_id:199, max_block_id:214, curr_writable_block_id:213}, logical_block_size_:67104768, curr_block_writable_size_:8073637, block_header_serialize_buf_:0x2b0e43d3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433836510, this:0x2b0e43d36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:16, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:15, majority_max_lsn:{lsn:6302238429}, saved_end_lsn:{lsn:6302238429}, last_submit_prepare_req_time_us:1722494433837290, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43d369b0}) [2024-10-16 14:57:18.851436] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=146] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:18.851447] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=115172922, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:18.855164] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.855181] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.855187] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910256999682048},{"BIGINT":1729058400},{"VARCHAR":"?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õÂ?ô\(õ¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.855244] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.855426] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202323} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.858114] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.858175] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A8B-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:18.860384] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.860397] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.860404] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041174757376},{"BIGINT":1729058400},{"VARCHAR":"@7", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.860470] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.860738] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8477}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285624} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.862933] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.862950] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.862957] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618590283485184},{"BIGINT":1729058400},{"VARCHAR":"@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­@ ~Åm\ú­¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.863014] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.863218] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1070}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202325} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.863242] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.863641] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.863649] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.863655] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041262837760},{"BIGINT":1729058400},{"VARCHAR":"@"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.863697] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.864058] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2112}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285625} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.866476] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.866494] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.866500] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049668222976},{"BIGINT":1729058400},{"VARCHAR":"Aê¢FãAê¢Vàß8Aê¢eÚ]ë…Aê¢t««Ø®Aꢃ€‰ AꢒœÛ#Aꢢšr¬Aꢱ‘Ø8†Aê¢ÀežÇãAê¢Ï7\~úAê¢ÞU¶Ü]Aê¢îR®n˜Aê¢ýKå\ûAê£ ¡µtAê£ðRE¢Aê£*dGEAê£: &FtAê£I뇔Aê£WÕÝæ›Aê£f©©­CAê£uÆškAꣅÆQ¯¸A꣔¼€ï5A꣣ùÛAꣲb:|…Aê£Á€a¿HAê£Ñ}Ý/Aê£àv¹¨XAê£ïJ$FtAê£þë…Aê¤ 8ýŠ Aê¤7DÔþAê¤,/íaAê¤;%·Aê¤IÔr¤©Aê¤XñZ ÕAê¤hïø1ùAê¤wçN!A꤆¼EœàAꤕ”l°òAꤤ²=qAꤴ°K¢œAê¤Ã©OùrAê¤Òz|ɆAê¤áN›&éAê¤ðk!o", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.866543] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.867042] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1020}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202326} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.867893] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.867906] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.867911] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910255099662336},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.867953] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.868224] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2210}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285626} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.869701] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061838869093000, start_id=1729061838869093001, end_id=1729061838869093002) [2024-10-16 14:57:18.870006] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.870020] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.870026] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049693388800},{"BIGINT":1729058400},{"VARCHAR":"Aš«qW§‡Aš«€õ+ÓÃAš«å0oiAš« Í^OvAš«°æ…ˆAš«À½H´9Aš«Ð½C{JAš«àµÝ—öAš«ð›Í;Aš¬", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.870076] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.870322] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1024}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202327} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.873893] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBCD-0-0] [lt=14] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061838841644, keepalive_ts:-1}) [2024-10-16 14:57:18.873912] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBCD-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061838841644, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:18.882253] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.882259] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.882267] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.882271] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.882278] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":681984530369335296},{"BIGINT":1729058400},{"VARCHAR":"Ac×€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.882274] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618590447063040},{"BIGINT":1729058400},{"VARCHAR":"A6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»šA6 [»š¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.882334] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.882335] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.882539] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:1053}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202330} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.882547] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:1052}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285630} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.883040] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=15] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:18.883916] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061838883888000, start_id=1729061838883888000, end_id=1729061838883888001) [2024-10-16 14:57:18.885218] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.885235] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.885241] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545341742764032},{"BIGINT":1729058400},{"VARCHAR":"A°|", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.885290] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.885495] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:8}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285631} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.886846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061838886012000, start_id=1729061838886012000, end_id=1729061838886012001) [2024-10-16 14:57:18.887846] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061838684661000, v:0}, new_max_version={val:1729061838684661000, v:0}, last_min_version={val:1729061838283939001, v:0}, last_max_version={val:1729061838283939001, v:0}, total_time_us=1832, delta_us=203183) [2024-10-16 14:57:18.887862] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061838684661000, v:0}, min_version={val:1729061838684661000, v:0}, max_version={val:1729061838684661000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:18.891188] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3CE6-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=142, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:18.891200] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3CE6-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=142, stat_interval=1000000, avg (count/cost)=2027, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:18.893161] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.893213] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.893223] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675512069381541888},{"BIGINT":1729058400},{"VARCHAR":"AWÏ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.893291] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=36][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.893556] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1093}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285634} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.893985] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.893997] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.894002] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618590585475072},{"BIGINT":1729058400},{"VARCHAR":"?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú?˜“t¼j~ú¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.894042] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.894338] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1090}, to_seq_no={branch:0, seq:1085}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202335} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.894494] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=31] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 211.00; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 8192.00, iops: 5.98, rt: 101.33; ) [2024-10-16 14:57:18.894513] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:18.894556] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 208.00; ) [2024-10-16 14:57:18.894565] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:18.894574] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:18.898734] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:18.898802] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1004 and flag like "%DUPLICATE%", tenant_id=1004, exec_tenant_id=1003, need_member_list=false) [2024-10-16 14:57:18.898853] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061838898852, bytes: 0, bw: 0.000000 MB/s, add_ts: 1008085, add_bytes: 0 [2024-10-16 14:57:18.899832] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.899844] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.899849] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049806635008},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.899899] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.900448] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2129}, to_seq_no={branch:0, seq:1068}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202337} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.900614] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.900626] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.900632] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":655630500856713216},{"BIGINT":1729058400},{"VARCHAR":"AWÏ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.900676] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.900889] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285636} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.902876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061838902343000, start_id=1729061838902343000, end_id=1729061838902343001) [2024-10-16 14:57:18.907038] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=53][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.907066] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.907074] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545345240813568},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.907128] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.911372] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.911396] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.911403] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041413832704},{"BIGINT":1729058400},{"VARCHAR":"A_¿", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.911472] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=40][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.911687] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285638} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.912753] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1587625984, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:18.913118] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=138, avg_log_batch_cnt=1, total_group_log_size=224084, avg_group_log_size=1623) [2024-10-16 14:57:18.913130] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=224085) [2024-10-16 14:57:18.914518] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:9780}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202339} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.919505] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:18.920491] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.920506] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.920512] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049869549568},{"BIGINT":1729058400},{"VARCHAR":"AcùÛ ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.920583] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=38][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.920804] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2115}, to_seq_no={branch:0, seq:1074}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202342} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.921230] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=145, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:18.921244] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=145, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:18.921265] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=145, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:18.922041] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.922069] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.922075] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":663258968113139712},{"BIGINT":1729058400},{"VARCHAR":"AO¾", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.922126] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.922409] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285642} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.923211] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.923226] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.923232] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910257062596608},{"BIGINT":1729058400},{"VARCHAR":"A6bƒ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.923272] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.923489] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202343} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.924767] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.924776] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.924781] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":656974733458919424},{"BIGINT":1729058400},{"VARCHAR":"AO¾", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.924819] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.925053] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285643} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.927773] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.927786] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.927792] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":656974733509251072},{"BIGINT":1729058400},{"VARCHAR":"AO¾", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.927834] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.928114] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285644} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.930249] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] submit log success(service_type_=0, last_id=1729061855590307000, limited_id=1729061855590307000) [2024-10-16 14:57:18.930259] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] submit log with lock success(service_type_=0, last_id=1729061855590307000, limited_id=1729061855590307000) [2024-10-16 14:57:18.930264] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061838929180000, start_id=1729061838929180000, end_id=1729061838929180001) [2024-10-16 14:57:18.930800] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] update id meta(ret=0, service_type=0, limited_id=1729061855590307000, latest_scn={val:1729061838929180001, v:0}, *this={tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061855590307000, latest_log_ts:{val:1729061838929180001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:18.930841] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=41] handle submit callback(service_type_=0, success=true, used_ts=594, log_ts={val:1729061838929180001, v:0}, limited_id=1729061855590307000) [2024-10-16 14:57:18.930848] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:18.930853] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:18.930987] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061838930240000, start_id=1729061838930240000, end_id=1729061838930240001) [2024-10-16 14:57:18.931553] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.931566] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.931571] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545467748044800},{"BIGINT":1729058400},{"VARCHAR":"A±|", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.931624] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.931816] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1037}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285645} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.932805] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061838932398000, start_id=1729061838932398000, end_id=1729061838932398001) [2024-10-16 14:57:18.933298] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.933311] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.933318] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049894715392},{"BIGINT":1729058400},{"VARCHAR":"AEšÅ€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.933359] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.933646] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22267][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2167}, to_seq_no={branch:0, seq:1121}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202346} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.934826] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061838934617000, start_id=1729061838934617000, end_id=1729061838934617001) [2024-10-16 14:57:18.937040] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1287, mem_used=4159488) [2024-10-16 14:57:18.940373] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.940389] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.940395] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545345291145216},{"BIGINT":1729058400},{"VARCHAR":"@¯ ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.940459] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.940643] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202349} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.940950] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.940964] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.940968] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654967884924702720},{"BIGINT":1729058400},{"VARCHAR":"AWÏ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.941011] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.941193] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2116}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285649} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.942919] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.942930] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.942928] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:18.942941] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27628320, tenant_id=1004) [2024-10-16 14:57:18.942950] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:18.942935] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618590925213696},{"BIGINT":1729058400},{"VARCHAR":"@®>", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.942979] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.943197] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1038}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202350} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.943658] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=5][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.943668] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.943673] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684547669581484032},{"BIGINT":1729058400},{"VARCHAR":"AWÏ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.943714] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.944110] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1041}, to_seq_no={branch:0, seq:1035}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285650} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.945813] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.945828] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.945833] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049970212864},{"BIGINT":1729058400},{"VARCHAR":"B(ÅOR", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.945872] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.946109] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:1055}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202351} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.946511] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.946521] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.946525] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684620839126683648},{"BIGINT":1729058400},{"VARCHAR":"AWÏ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.946564] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.946742] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1068}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285651} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.952013] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:18.953221] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061838953219, bytes: 4446795340020, bw: 3.449407 MB/s, add_ts: 1001609, add_bytes: 3622785 [2024-10-16 14:57:18.955521] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:18.955538] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:18.955545] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618591063625728},{"BIGINT":1729058400},{"VARCHAR":"A—G!À", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:18.955619] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:18.955826] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1073}, to_seq_no={branch:0, seq:1068}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202355} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.958080] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A8D-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:18.958260] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.960747] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:18.963304] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:18.965084] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2189}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202358} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.965681] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2079}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285655} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.969677] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061838969638000, start_id=1729061838969638000, end_id=1729061838969638001) [2024-10-16 14:57:18.973424] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3024}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285657} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.974067] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1068}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202360} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.976504] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285658} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.976863] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15C6-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:18.976927] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15C8-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:18.977879] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061838977014000, start_id=1729061838977014000, end_id=1729061838977014001) [2024-10-16 14:57:18.984449] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4200}, to_seq_no={branch:0, seq:1033}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202362} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.985609] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061838984794000, start_id=1729061838984794001, end_id=1729061838984794002) [2024-10-16 14:57:18.985642] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=52] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=342, stat_interval=1000000, avg (count/cost)=30, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:18.985653] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=342, stat_interval=1000000, avg (count/cost)=73, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:18.992151] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1056}, to_seq_no={branch:0, seq:1051}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202365} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:18.992367] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:18.992388] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=207) [2024-10-16 14:57:18.999279] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=14] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:18.999446] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2129}, to_seq_no={branch:0, seq:2120}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285665} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.003361] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=52, replace_time=0, replace_start_pos=754968, replace_num=31457) [2024-10-16 14:57:19.003396] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=52, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:19.003937] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1809}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285667} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.004190] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1810}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202371} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.006564] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202372} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.006742] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:19.007491] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.007517] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.007537] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=16] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:19.007681] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836738668000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850098662000, latest_log_ts:{val:1729061833508923001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:19.007731] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=48] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836738668000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850098662000, latest_log_ts:{val:1729061833508923001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:19.007761] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:19.007768] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061833508923001, v:0}) [2024-10-16 14:57:19.007772] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:19.007779] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:19.007788] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:19.007796] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:19.008511] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:1, active_tx_count:1, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.008531] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=20] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.008558] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:19.008611] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836738668000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:19.008647] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=34] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836738668000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:19.008674] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:19.008681] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:19.008691] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:19.008699] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:19.009416] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.009430] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=13] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.009440] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:19.009474] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836738668000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:19.009504] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836738668000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:19.009530] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:19.009535] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:19.009544] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:19.009551] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:19.010251] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.010266] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.010274] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:19.010310] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836738668000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:19.010352] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=40] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836738668000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:19.010386] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=32] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:19.010395] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:19.010404] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:19.010413] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:19.010417] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:19.011432] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5388}, to_seq_no={branch:0, seq:4328}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285669} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.012962] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202373} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.013839] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839013297000, start_id=1729061839013297000, end_id=1729061839013297001) [2024-10-16 14:57:19.013886] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=40] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:19.013903] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061839013900], gts:1729061839013297000, latest_srr:[mts=1729061839013900]}) [2024-10-16 14:57:19.014266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061839013297000, start_id=1729061839013297001, end_id=1729061839013297002) [2024-10-16 14:57:19.016333] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:19.016351] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:19.019382] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF6B-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1636) [2024-10-16 14:57:19.019401] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF6A-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:19.020420] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:19.022001] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202374} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.023791] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1042}, to_seq_no={branch:0, seq:1036}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285671} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.024720] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1055}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202375} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.027642] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285673} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.030834] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1601}, to_seq_no={branch:0, seq:1596}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202377} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.033493] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1037}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202378} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.034172] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1036}, to_seq_no={branch:0, seq:1031}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285675} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.036350] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:19.036362] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:19.036375] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=291835856, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:19.038678] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285678} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.040385] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1027}, to_seq_no={branch:0, seq:1022}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202380} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.043130] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1014}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202381} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.046550] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1037}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202382} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.047503] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618575372734464},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285681} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.047699] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285681} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.056003] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675512509582135296},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285684} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.056251] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285684} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.058353] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.058762] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830041929732096},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285685} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.058999] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285685} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.059092] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2114}, to_seq_no={branch:0, seq:1057}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202387} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.063365] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.063391] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1056}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202389} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.065448] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4808-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061839033229, keepalive_ts:-1}) [2024-10-16 14:57:19.065465] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4808-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061839033229, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:19.065744] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1056}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202390} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.068424] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1080}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202391} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.070657] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061839070532000, start_id=1729061839070532000, end_id=1729061839070532001) [2024-10-16 14:57:19.070928] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830042093309952},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285689} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.071133] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1083}, to_seq_no={branch:0, seq:1078}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285689} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.072373] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1152}, to_seq_no={branch:0, seq:1147}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202393} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.073563] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910255238074368},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285690} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.073801] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1020}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285690} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.074817] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:11}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202394} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.081814] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830042219139072},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285694} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.082073] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285694} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.085054] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:19.085071] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=152413380, tenant_id=1) [2024-10-16 14:57:19.085084] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:19.086916] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061839086210000, start_id=1729061839086210000, end_id=1729061839086210001) [2024-10-16 14:57:19.097072] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545341818261504},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285701} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.097573] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:8}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285701} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.098990] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202402} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.099085] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061839098849000, start_id=1729061839098849000, end_id=1729061839098849001) [2024-10-16 14:57:19.100181] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618575846690817},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285702} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.100463] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1034}, to_seq_no={branch:0, seq:1029}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285702} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.105620] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202405} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.109901] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:19.109920] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:19.113011] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1589706752, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:19.116366] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1636, avg_group_log_size=163) [2024-10-16 14:57:19.116378] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1636) [2024-10-16 14:57:19.116403] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:19.116408] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:19.120325] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C91-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510306988718, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410508, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.123047] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:19.123882] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B74-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061839123295000, start_id=1729061839123295000, end_id=1729061839123295001) [2024-10-16 14:57:19.124073] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B78-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061839123295000, start_id=1729061839123295000, end_id=1729061839123295001) [2024-10-16 14:57:19.124444] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B75-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061839124354000, start_id=1729061839124354000, end_id=1729061839124354001) [2024-10-16 14:57:19.124500] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0B77-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839124354000, start_id=1729061839124354000, end_id=1729061839124354001) [2024-10-16 14:57:19.125513] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:19.126628] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=6] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=42, mem_used=2079744) [2024-10-16 14:57:19.128059] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10459, avg_group_log_size=550) [2024-10-16 14:57:19.128074] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10459) [2024-10-16 14:57:19.128243] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=255, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:19.128256] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=105, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:19.128270] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:19.128472] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCFAE-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:19.128478] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCFAE-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=670, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:19.128729] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797AC2-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10581) [2024-10-16 14:57:19.130498] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061839128595000, start_id=1729061839128595000, end_id=1729061839128595001) [2024-10-16 14:57:19.133332] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21658][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE873-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307001727, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410509, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.133500] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061839131746000, start_id=1729061839131746000, end_id=1729061839131746001) [2024-10-16 14:57:19.133779] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11052) [2024-10-16 14:57:19.134758] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCFAF-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=384, avg_log_submit_to_slide_time=670) [2024-10-16 14:57:19.135026] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061839133765000, start_id=1729061839133765000, end_id=1729061839133765001) [2024-10-16 14:57:19.136015] INFO [STORAGE] runTimerTask (ob_ddl_heart_beat_task.cpp:49) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8] send to rs all task status succeed [2024-10-16 14:57:19.141963] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7608}, to_seq_no={branch:0, seq:1349}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202411} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.147645] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830042521128960},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285706} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.147924] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1024}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285706} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.153460] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830042584043520},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285708} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.153696] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1079}, to_seq_no={branch:0, seq:1074}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285708} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.156935] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202417} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.157583] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830042634375168},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285709} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.157790] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285709} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.158414] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.158648] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:19.158662] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:19.159031] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=206, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:19.159044] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=732) [2024-10-16 14:57:19.159056] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=720, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:19.159066] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=732) [2024-10-16 14:57:19.159073] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:19.159776] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3B61-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:19.159785] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3B61-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=1206, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:19.160649] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061839160334000, start_id=1729061839160334000, end_id=1729061839160334001) [2024-10-16 14:57:19.162141] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830042747621376},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285710} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.162388] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285710} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.163336] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5015-0-0] [lt=8] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:19.163363] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5015-0-0] [lt=22] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:19.163426] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.164266] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=290662) [2024-10-16 14:57:19.164891] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2115}, to_seq_no={branch:0, seq:1044}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202419} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.165354] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:19.165386] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830042823118848},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285711} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.165837] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=21] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285711} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.171393] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2122}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202420} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.171574] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061839170989000, start_id=1729061839170989000, end_id=1729061839170989001) [2024-10-16 14:57:19.178616] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839177304000, start_id=1729061839177304000, end_id=1729061839177304001) [2024-10-16 14:57:19.181787] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=297266) [2024-10-16 14:57:19.181993] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106968][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D4EAD-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=298889) [2024-10-16 14:57:19.182005] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830043112525824},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285716} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.182286] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285716} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.184955] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AD-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:19.184992] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AD-0-0] [lt=24] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061838625138004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061838625138004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:19.186902] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A08-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061839177966, keepalive_ts:-1}) [2024-10-16 14:57:19.186921] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A08-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061839177966, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:19.187120] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C12-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:19.187319] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2222}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202424} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.189132] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830043250937856},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285720} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.189278] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AD-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061839188859000, start_id=1729061839188859000, end_id=1729061839188859001) [2024-10-16 14:57:19.189410] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=10] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1684}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285720} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.189800] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:19.194961] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1058}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202426} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.197485] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1032}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202427} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.200092] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830043364184064},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285724} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.200430] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285724} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.200559] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:1057}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202428} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.201763] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061839201589000, start_id=1729061839201589000, end_id=1729061839201589001) [2024-10-16 14:57:19.204051] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2273}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202429} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.205019] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=8] Cache replace map node details(ret=0, replace_node_count=272, replace_time=0, replace_start_pos=786425, replace_num=31457) [2024-10-16 14:57:19.205036] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=272, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:19.210693] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1083}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202431} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.212833] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21656][T1_L0_G0][T1][YB4A0A000408-00061E9B592E5872-0-0] [lt=14] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:19.212928] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21656][T1_L0_G0][T1][YB4A0A000408-00061E9B592E5872-0-0] [lt=12][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:19.212946] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:19.213683] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.213707] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=23] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.213730] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:19.213849] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836511707002, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850721265000, latest_log_ts:{val:1729061834119131001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:19.213893] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=41] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:19.213910] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:19.213917] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061834119131001, v:0}) [2024-10-16 14:57:19.213921] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.213928] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:19.213938] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:19.213945] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:19.214694] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.214708] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.214719] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:19.214789] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836511707002, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:19.214827] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=37] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836511707002, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:19.214859] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:19.214865] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:19.214871] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:19.214877] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:19.215562] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.215574] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.215584] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:19.215659] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836511707002, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:19.215688] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836511707002, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:19.215711] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:19.215716] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:19.215722] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:19.215727] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:19.216448] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.216460] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:19.216469] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:19.216536] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836511707002, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:19.216564] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061836511707002, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:19.216587] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:19.216592] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:19.216598] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:19.216611] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:19.216615] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:19.216934] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=195, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:19.216948] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=195, stat_interval=1000000, avg (count/cost)=639, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:19.217035] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=312, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:19.217046] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=392, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:19.217064] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:19.217072] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1419, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:19.217076] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:19.217137] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:19.217143] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1369, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:19.217146] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=1112, avg_log_submit_to_slide_time=1354) [2024-10-16 14:57:19.222090] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830043565510656},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285730} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.222328] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1057}, to_seq_no={branch:0, seq:1052}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285730} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.224846] INFO [SERVER] refresh_avail_memory (ob_table_load_assigned_memory_manager.cpp:117) [21531][T1_TntSharedTim][T1][Y0-0000000000000000-0-0] [lt=13] ObTableLoadAssignedMemoryManager::refresh_avail_memory(MTL_ID()=1, avail_memory=107374182, chunk_count_=0, avail_sort_memory_=107374182, avail_memory_=107374182) [2024-10-16 14:57:19.226586] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202435} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.226823] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] worker thread began to shrink(tenant_->id()=1002, group_id_=1, token=0) [2024-10-16 14:57:19.227047] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106967][T1002_L0_G1][T1002][Y0-0000000000000000-0-0] [lt=16] worker thread exit(tenant_->id()=1002, workers_.get_size()=4) [2024-10-16 14:57:19.227062] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106967][T1002_L0_G1][T0][Y0-0000000000000000-0-0] [lt=11] tenant thread end_run(id_=1002, ret=0, thread_count_=192) [2024-10-16 14:57:19.229445] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830043641008128},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285732} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.230194] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1048}, to_seq_no={branch:0, seq:1038}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285732} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.231421] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202437} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.233179] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E07-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061839232328000, start_id=1729061839232328000, end_id=1729061839232328001) [2024-10-16 14:57:19.238214] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830043741671424},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285735} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.238548] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2100}, to_seq_no={branch:0, seq:2095}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285735} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.240939] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1DFC-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061839239679000, start_id=1729061839239679000, end_id=1729061839239679001) [2024-10-16 14:57:19.246204] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830043842334720},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285737} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.246471] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD03-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:19.248416] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD102B-0-0] [lt=10] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:19.249285] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6333}, to_seq_no={branch:0, seq:2150}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285737} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.250887] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1074}, to_seq_no={branch:0, seq:1069}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202442} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.256386] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202444} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.258161] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A93-0-0] [lt=20] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:19.258175] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A93-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10737, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:19.258180] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A93-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=212, avg_log_submit_to_slide_time=10731) [2024-10-16 14:57:19.258208] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:19.258467] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044018495488},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285740} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.258552] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.258786] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2144}, to_seq_no={branch:0, seq:1099}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285740} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.262751] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2092}, to_seq_no={branch:0, seq:1039}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202446} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.263509] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.267563] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044228210689},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285743} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.267804] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1023}, to_seq_no={branch:0, seq:1018}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285743} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.269324] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2111}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202448} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.269469] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] dump tenant info(tenant={id:1, tenant_meta:{unit:{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, super_block:{tenant_id:1, replay_start_point:ObLogCursor{file_id=12, log_id=1497048, offset=40358755}, ls_meta_entry:[2738](ver=0,mode=0,seq=7639021), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"3.000000000000000000e+00", unit_max_cpu:"3.000000000000000000e+00", total_worker_cnt:54, min_worker_cnt:32, max_worker_cnt:150, stopped:0, worker_us:1096708777872, recv_hp_rpc_cnt:5717674, recv_np_rpc_cnt:154800912, recv_lp_rpc_cnt:0, recv_mysql_cnt:62153705, recv_task_cnt:6664548, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:19623806307, workers:32, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=7 cnt[3]=0 cnt[4]=0 cnt[5]=228764008 cnt[6]=10623 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52540841, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143148686}{group_id:10, queue_size:0, recv_req_cnt:19682005, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143048442}{group_id:20, queue_size:0, recv_req_cnt:401, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015260372185}{group_id:15, queue_size:0, recv_req_cnt:5821, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723554329}{group_id:5, queue_size:0, recv_req_cnt:35746001, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494262337428}{group_id:19, queue_size:0, recv_req_cnt:7245659, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061834968674}{group_id:9, queue_size:0, recv_req_cnt:15758030, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143222379}{group_id:1, queue_size:0, recv_req_cnt:153510489, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061836676474}, rpc_stat_info: pcode=0x51c:cnt=299 pcode=0x1502:cnt=216 pcode=0x717:cnt=182 pcode=0x150b:cnt=60 pcode=0x710:cnt=50, token_change_ts:1722494237022007, tenant_role:1}) [2024-10-16 14:57:19.270151] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=681] dump throttled time info(id_=1, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 10, group: OBCG_LOC_CACHE, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 9, group: OBCG_DETECT_RS, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1, tenant_throttled_time: 0;) [2024-10-16 14:57:19.270175] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=24] dump tenant info(tenant={id:508, tenant_meta:{unit:{tenant_id:508, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140910089, is_removed:false}, super_block:{tenant_id:508, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:15572488569, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:51691458, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6536460920, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:15, queue_size:0, recv_req_cnt:53884, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060722522359}, rpc_stat_info: pcode=0x5c0:cnt=67, token_change_ts:1722494140914497, tenant_role:0}) [2024-10-16 14:57:19.270612] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=436] dump throttled time info(id_=508, throttled_time_log=group_id: 15, group: OBCG_WR, throttled_time: 0;tenant_id: 508, tenant_throttled_time: 0;) [2024-10-16 14:57:19.270618] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=6] dump tenant info(tenant={id:509, tenant_meta:{unit:{tenant_id:509, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140901337, is_removed:false}, super_block:{tenant_id:509, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:12839934798, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6441799995, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1722494140916657, tenant_role:0}) [2024-10-16 14:57:19.271008] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=389] dump throttled time info(id_=509, throttled_time_log=tenant_id: 509, tenant_throttled_time: 0;) [2024-10-16 14:57:19.271012] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=4] dump tenant info(tenant={id:1001, tenant_meta:{unit:{tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1001, replay_start_point:ObLogCursor{file_id=11, log_id=1059563, offset=59267180}, ls_meta_entry:[7452](ver=0,mode=0,seq=7431948), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:468363444061, recv_hp_rpc_cnt:13589232, recv_np_rpc_cnt:126529043, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2784363264, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=9 cnt[3]=0 cnt[4]=0 cnt[5]=167010890 cnt[6]=3244 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52540840, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494145075365}{group_id:20, queue_size:0, recv_req_cnt:306, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015263681355}{group_id:5, queue_size:0, recv_req_cnt:58923341, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726724812431820}{group_id:19, queue_size:0, recv_req_cnt:7662166, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061784948758}{group_id:1, queue_size:0, recv_req_cnt:227492385, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061834576390}, rpc_stat_info: pcode=0x1502:cnt=346 pcode=0x51c:cnt=223 pcode=0x717:cnt=187 pcode=0x710:cnt=88 pcode=0x150b:cnt=60, token_change_ts:1722494144838773, tenant_role:1}) [2024-10-16 14:57:19.271483] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=470] dump throttled time info(id_=1001, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1001, tenant_throttled_time: 0;) [2024-10-16 14:57:19.271497] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] dump tenant info(tenant={id:1002, tenant_meta:{unit:{tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1002, replay_start_point:ObLogCursor{file_id=191, log_id=47529733, offset=57901112}, ls_meta_entry:[10485](ver=0,mode=0,seq=7657455), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:998828954404, recv_hp_rpc_cnt:198477751, recv_np_rpc_cnt:379057277, recv_lp_rpc_cnt:0, recv_mysql_cnt:280946906, recv_task_cnt:146658, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:4103754187, workers:6, nesting workers:8, req_queue:total_size=1 queue[0]=0 queue[1]=0 queue[2]=1 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=673 cnt[3]=0 cnt[4]=0 cnt[5]=64428685 cnt[6]=226112 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131354453, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1728209719937557}{group_id:20, queue_size:0, recv_req_cnt:1673, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264869418}{group_id:15, queue_size:0, recv_req_cnt:5733, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723564477}{group_id:5, queue_size:0, recv_req_cnt:401892241, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1727167492226332}{group_id:19, queue_size:0, recv_req_cnt:1399881, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061796075772}{group_id:1, queue_size:0, recv_req_cnt:977156041, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1729061839226821}, rpc_stat_info: pcode=0x710:cnt=1994 pcode=0x52b:cnt=1624 pcode=0x1502:cnt=1126 pcode=0x1501:cnt=900 pcode=0x1516:cnt=850, token_change_ts:1729004298741539, tenant_role:1}) [2024-10-16 14:57:19.271964] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=466] dump throttled time info(id_=1002, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1002, tenant_throttled_time: 0;) [2024-10-16 14:57:19.271972] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] dump tenant info(tenant={id:1003, tenant_meta:{unit:{tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1003, replay_start_point:ObLogCursor{file_id=11, log_id=1067656, offset=59494798}, ls_meta_entry:[3704](ver=0,mode=0,seq=7380334), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:503354681488, recv_hp_rpc_cnt:13609623, recv_np_rpc_cnt:126131745, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2783960029, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=4 cnt[3]=0 cnt[4]=0 cnt[5]=196317174 cnt[6]=6245 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52540801, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722517512781421}{group_id:20, queue_size:0, recv_req_cnt:308, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015261556381}{group_id:5, queue_size:0, recv_req_cnt:63971551, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726083164255294}{group_id:19, queue_size:0, recv_req_cnt:7660866, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061802906653}{group_id:1, queue_size:0, recv_req_cnt:228853434, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061836329739}, rpc_stat_info: pcode=0x1502:cnt=358 pcode=0x51c:cnt=253 pcode=0x717:cnt=187 pcode=0x710:cnt=98 pcode=0x150b:cnt=60, token_change_ts:1722494150252707, tenant_role:1}) [2024-10-16 14:57:19.272412] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=438] dump throttled time info(id_=1003, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1003, tenant_throttled_time: 0;) [2024-10-16 14:57:19.272427] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] dump tenant info(tenant={id:1004, tenant_meta:{unit:{tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1004, replay_start_point:ObLogCursor{file_id=197, log_id=49173916, offset=15367427}, ls_meta_entry:[12150](ver=0,mode=0,seq=7655944), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:603321444608, recv_hp_rpc_cnt:12341725, recv_np_rpc_cnt:113755047, recv_lp_rpc_cnt:0, recv_mysql_cnt:11227, recv_task_cnt:1052, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2741344749, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=99 cnt[3]=0 cnt[4]=0 cnt[5]=72426136 cnt[6]=155203 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131354311, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494154950936}{group_id:20, queue_size:0, recv_req_cnt:798, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264132685}{group_id:15, queue_size:0, recv_req_cnt:5423, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723614999}{group_id:21, queue_size:0, recv_req_cnt:15, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269726417109}{group_id:5, queue_size:0, recv_req_cnt:31029487, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1724705543331737}{group_id:19, queue_size:0, recv_req_cnt:1325441, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061785498526}{group_id:16, queue_size:0, recv_req_cnt:18, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269721623294}{group_id:1, queue_size:0, recv_req_cnt:509697803, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061817163496}, rpc_stat_info: pcode=0x1502:cnt=402 pcode=0x1501:cnt=200 pcode=0x1516:cnt=192 pcode=0x717:cnt=179 pcode=0x150b:cnt=120, token_change_ts:1725266206494273, tenant_role:1}) [2024-10-16 14:57:19.272930] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=502] dump throttled time info(id_=1004, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 21, group: OBCG_UNLOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 16, group: OBCG_TRANSFER, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1004, tenant_throttled_time: 0;) [2024-10-16 14:57:19.279634] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] no need to update min/max version(new_version={val:1729061838929180002, v:0}, min_version={val:1729061838929180002, v:0}, max_version={val:1729061838929180002, v:0}) [2024-10-16 14:57:19.280683] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061839280619000, start_id=1729061839280619000, end_id=1729061839280619001) [2024-10-16 14:57:19.281648] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9B3769AB91-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:19.282144] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:204022435727}], scn_array=[cnt:1, {val:1729061839269048000, v:0}], curr_size=1624, accum_size=173743, time_cost=12713) [2024-10-16 14:57:19.282811] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839281688000, start_id=1729061839281688000, end_id=1729061839281688001) [2024-10-16 14:57:19.283386] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10] start to print blacklist info [2024-10-16 14:57:19.283559] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=15] ls blacklist refresh finish(cost_time=7276) [2024-10-16 14:57:19.285198] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=1361) [2024-10-16 14:57:19.285970] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:19.286770] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061839285499000, start_id=1729061839285499001, end_id=1729061839285499002) [2024-10-16 14:57:19.286832] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=7, avg_log_batch_cnt=1, total_group_log_size=1706, avg_group_log_size=243) [2024-10-16 14:57:19.286842] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1280) [2024-10-16 14:57:19.287670] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839285499000, start_id=1729061839285499000, end_id=1729061839285499001) [2024-10-16 14:57:19.287718] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=4990, avg_group_log_size=453) [2024-10-16 14:57:19.288219] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:19.288358] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C14-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1333) [2024-10-16 14:57:19.288377] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C14-0-0] [lt=8][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172925, log_task={header:{begin_lsn:{lsn:14352347081}, end_lsn:{lsn:14352347203}, log_id:115172925, min_scn:{val:1729061839124354000, v:0}, max_scn:{val:1729061839124354000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352346959}, prev_proposal_id:17, committed_end_lsn:{lsn:14352347081}, data_checksum:940009244, accum_checksum:1428390380, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061839187116, freeze_ts:1729061839187116, submit_ts:1729061839187141, flushed_ts:1729061839187321, gen_to_freeze cost time:0, gen_to_submit cost time:25, submit_to_flush cost time:180}, fs_cb_begin_ts=1729061839288370, log_life_time=101254) [2024-10-16 14:57:19.288956] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B00-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=347, avg_log_submit_to_slide_time=627) [2024-10-16 14:57:19.289617] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044534394880},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285746} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.289957] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3647}, to_seq_no={branch:0, seq:2505}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285746} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.293137] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B1E-0-0] [lt=16] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.114681053161621094e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.994376630833028685e+03) [2024-10-16 14:57:19.293215] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B1E-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061839292293000, start_id=1729061839292293000, end_id=1729061839292293001) [2024-10-16 14:57:19.294284] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044584726528},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285747} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.294522] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3181}, to_seq_no={branch:0, seq:1028}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285747} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.296618] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2114}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202453} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.298323] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044672806912},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285749} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.298530] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1036}, to_seq_no={branch:0, seq:1031}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285749} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.300575] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2201}, to_seq_no={branch:0, seq:1141}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202454} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.305337] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044773470208},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285752} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.305722] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1073}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285752} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.306976] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061839306199000, start_id=1729061839306199000, end_id=1729061839306199001) [2024-10-16 14:57:19.308434] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044798636032},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285753} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.308732] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1038}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285753} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.309213] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=9] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5329, mem_used=10398720) [2024-10-16 14:57:19.311125] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044836384768},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285754} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.311456] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:1055}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285754} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.313443] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1196392448, global_cache_size=1589706752, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:19.313806] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Wash memory, (tenant_id=1002, cache_size=95715328, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:57:19.313930] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash time detail, (compute_wash_size_time=165, refresh_score_time=96, wash_time=250) [2024-10-16 14:57:19.315862] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044861550592},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285755} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.316220] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3103}, to_seq_no={branch:0, seq:2056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285755} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.316961] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=7] Cache wash clean map node details(ret=0, clean_node_count=49, clean_time=0, clean_start_pos=314570, clean_num=62914) [2024-10-16 14:57:19.317563] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:19.317593] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1636) [2024-10-16 14:57:19.320590] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:19.325305] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:19.325577] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1240}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202467} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.325621] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27] rpc statistics(total_trans_msg_count=402, total_batch_msg_count=0) [2024-10-16 14:57:19.327828] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=13] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:19.328768] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654830044999962624},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285759} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.329147] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1075}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285759} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.330466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061839329676000, start_id=1729061839329676000, end_id=1729061839329676001) [2024-10-16 14:57:19.330862] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=195, avg_log_batch_cnt=0, total_group_log_size=316644, avg_group_log_size=1623) [2024-10-16 14:57:19.333095] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654830045100625920},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285760} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.334248] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061839333920000, start_id=1729061839333920000, end_id=1729061839333920001) [2024-10-16 14:57:19.334398] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:10}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285760} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.335075] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061839334983000, start_id=1729061839334983000, end_id=1729061839334983001) [2024-10-16 14:57:19.344957] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA2-0-0] [lt=12] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:19.344978] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA2-0-0] [lt=17] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=22) [2024-10-16 14:57:19.348519] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789D7-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061839347791000, start_id=1729061839347791000, end_id=1729061839347791001) [2024-10-16 14:57:19.349483] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830045390032896},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285764} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.349801] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1041}, to_seq_no={branch:0, seq:1036}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285764} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.354648] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202476} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.357496] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202477} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.358616] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.359107] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=22] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:19.359119] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:19.363570] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.364106] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1057}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202480} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.367624] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654830045629108224},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285771} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.368081] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1194}, to_seq_no={branch:0, seq:1187}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285771} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.372256] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1070}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202482} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.374640] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830045704605696},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285773} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.374876] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3A-0-0] [lt=0] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:19.374898] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3A-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061839369187001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061839369187001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:19.375918] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C3A-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061839375539000, start_id=1729061839375539000, end_id=1729061839375539001) [2024-10-16 14:57:19.376962] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3180}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285773} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.378757] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C3A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061839377795000, start_id=1729061839377795000, end_id=1729061839377795001) [2024-10-16 14:57:19.378902] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061839377795000, start_id=1729061839377795001, end_id=1729061839377795002) [2024-10-16 14:57:19.380528] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839380046000, start_id=1729061839380046000, end_id=1729061839380046001) [2024-10-16 14:57:19.382141] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830045767520256},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285775} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.382421] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1120}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285775} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.383484] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061839383287000, start_id=1729061839383287000, end_id=1729061839383287001) [2024-10-16 14:57:19.383569] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=17] gc stale ls task succ [2024-10-16 14:57:19.387181] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C16-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:19.389975] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202488} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.390212] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830045868183552},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285778} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.391152] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2083}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285778} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.399261] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2098}, to_seq_no={branch:0, seq:1047}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202492} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.402183] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202493} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.404165] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654830046056927232},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285782} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.404410] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:1057}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285782} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.404892] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202494} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.406692] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=536, replace_time=0, replace_start_pos=817882, replace_num=31457) [2024-10-16 14:57:19.406720] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=536, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:19.407165] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061839406674000, start_id=1729061839406674000, end_id=1729061839406674001) [2024-10-16 14:57:19.407540] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061839406674000, start_id=1729061839406674001, end_id=1729061839406674002) [2024-10-16 14:57:19.409131] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:9}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202495} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.409518] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830046203727873},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285784} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.409825] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285784} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.412191] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830046270836736},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285785} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.412425] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285785} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.412684] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=199, avg_log_batch_cnt=0, total_group_log_size=321625, avg_group_log_size=1616) [2024-10-16 14:57:19.417950] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830046384082944},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285788} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.418221] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1015}, to_seq_no={branch:0, seq:1009}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285788} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.426925] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1138}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202501} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.429576] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1025}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202502} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.433300] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1049}, to_seq_no={branch:0, seq:9}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202503} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.438876] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830046962896896},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285798} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.439253] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285798} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.442664] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:1055}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202506} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.442765] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654830047000645632},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285799} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.443014] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1955}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285799} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.445594] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047076143104},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285800} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.446044] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A69-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:19.446058] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A69-0-0] [lt=12] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:19.446316] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2113}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285800} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.448911] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047126474752},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285801} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.449111] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285801} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.451189] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:1057}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202510} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.452091] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] [COMPACTION DAG STATUS] (start_time=1729061829448040, end_time=1729061839452090, dag_status={}) [2024-10-16 14:57:19.452105] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:19.452113] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:19.452117] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:19.452120] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:19.452124] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:19.452127] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:19.452133] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:19.452137] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:19.452140] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:19.452144] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452150] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452154] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452158] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452168] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452173] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452176] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452180] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452184] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452188] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452192] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452196] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452200] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452204] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452207] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452211] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452215] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452222] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452226] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452230] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452233] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452237] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452241] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452245] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452249] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452252] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452256] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452260] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452263] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452268] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452271] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452275] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452279] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452283] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452287] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452296] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452300] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452304] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452308] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452311] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452315] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452319] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452323] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452326] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452330] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452334] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452338] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452341] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452345] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452349] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452352] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452356] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452360] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452364] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452368] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452371] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452375] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452379] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:19.452384] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:19.452387] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:19.452390] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:19.452393] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:19.452396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:19.452398] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:19.452400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:19.452403] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:19.452405] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:19.452412] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:19.452416] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:19.458419] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:19.458700] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.460764] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106746][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8B8-0-0] [lt=13] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:19.460791] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106746][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8B8-0-0] [lt=23] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:19.460809] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106746][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8B8-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:19.462208] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1022}, to_seq_no={branch:0, seq:1017}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202512} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.463624] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.465783] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21810][T1001_TntShared][T1001][Y0-0000000000000000-0-0] [lt=15] clear dtl interm result cost(us)(clear_cost=15, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:19.469838] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=230, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:19.469857] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=56, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:19.470102] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=47, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:19.470115] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=13] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=1522, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:19.470120] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:19.470123] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=-529, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:19.470127] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=1040, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:19.472903] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3142}, to_seq_no={branch:0, seq:2107}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202515} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.476092] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:19.476103] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=11] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:19.476113] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:19.476109] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A22-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:19.476118] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:19.476122] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:19.476125] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:19.476123] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A22-0-0] [lt=12] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:19.476128] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:19.476130] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:19.476133] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:19.476135] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:19.476138] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:19.479179] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047613014017},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285811} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.479578] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285811} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.483667] INFO [STORAGE.TRANS] refresh_elr_tenant_config_ (ob_tx_elr_util.cpp:47) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C81-0-0] [lt=16] refresh tenant config success(tenant_id=1004, *this={last_refresh_ts:1729061839483155, can_tenant_elr:true}) [2024-10-16 14:57:19.483842] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C81-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061839483155000, start_id=1729061839483155000, end_id=1729061839483155001) [2024-10-16 14:57:19.484474] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839484214000, start_id=1729061839484214000, end_id=1729061839484214001) [2024-10-16 14:57:19.484613] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF78-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=25, avg_log_submit_to_flush_time=743, avg_log_submit_to_slide_time=1089) [2024-10-16 14:57:19.487860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061839486339000, start_id=1729061839486339000, end_id=1729061839486339001) [2024-10-16 14:57:19.488030] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:19.488048] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:19.488062] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70454030, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:19.496533] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202525} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.497717] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654830048019861504},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285816} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.497931] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1167}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285816} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.498546] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3DF8-0-0] [lt=15] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=216, avg_log_gen_to_freeze_time=44, avg_log_gen_to_submit_time=67, avg_log_submit_to_flush_time=408, avg_log_submit_to_slide_time=608) [2024-10-16 14:57:19.500551] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1070}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202527} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.500683] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048107941888},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285817} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.500903] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1072}, to_seq_no={branch:0, seq:1067}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285817} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.507637] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1150}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202529} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.509232] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061839508525000, start_id=1729061839508525001, end_id=1729061839508525002) [2024-10-16 14:57:19.509962] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048359600128},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285823} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.510212] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285823} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.513041] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048409931776},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285824} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.513359] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202531} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.513363] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1068}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285824} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.514307] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839513891000, start_id=1729061839513891000, end_id=1729061839513891001) [2024-10-16 14:57:19.514329] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=500, get_gts_with_stc_cnt=77, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=75, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:19.514481] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=521, get_gts_with_stc_cnt=5, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:19.514488] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061839514474], gts:1729061839513891000, latest_srr:[mts=1729061839514474]}) [2024-10-16 14:57:19.515928] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048435097600},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285825} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.516124] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285825} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.517328] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:19.517439] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061839517352000, start_id=1729061839517352000, end_id=1729061839517352001) [2024-10-16 14:57:19.517876] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1338}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202533} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.522464] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=7] replay status is not follower(min_unreplayed_lsn={lsn:7385763025}, base_lsn={lsn:7314419712}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:19.522519] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=55] get_replay_process success(id={id:1}, replayed_log_size=71343313, unreplayed_log_size=0) [2024-10-16 14:57:19.522530] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:57923794664}, base_lsn={lsn:57911414784}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:19.522556] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=26] get_replay_process success(id={id:1001}, replayed_log_size=12379880, unreplayed_log_size=0) [2024-10-16 14:57:19.522561] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=152596172825, unreplayed_log_size=0) [2024-10-16 14:57:19.522569] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=6] get_replay_process success(id={id:1003}, replayed_log_size=149811691062, unreplayed_log_size=0) [2024-10-16 14:57:19.522583] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1002, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=288478, last_replayed_log_size(MB)=288476, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:19.524183] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1058}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202535} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.526857] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048699338752},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285831} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.527180] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1025}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285831} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.529438] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:19.529996] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048737087488},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285832} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.530439] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285832} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.530544] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061839528934000, start_id=1729061839528934000, end_id=1729061839528934001) [2024-10-16 14:57:19.534336] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839533926000, start_id=1729061839533926000, end_id=1729061839533926001) [2024-10-16 14:57:19.534389] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:19.534402] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:19.535460] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061839534983000, start_id=1729061839534983000, end_id=1729061839534983001) [2024-10-16 14:57:19.542774] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3C15-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=216, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:19.542790] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3C15-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=216, stat_interval=1000000, avg (count/cost)=1101, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:19.542787] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=350751, hit_cnt=201, read_cnt=201, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:19.546131] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048900665344},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285838} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.546372] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1031}, to_seq_no={branch:0, seq:1026}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285838} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.554235] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1073}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202544} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.556500] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048950996992},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285840} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.556743] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285840} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.558776] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.563685] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.564214] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=7, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=24, avg_log_submit_to_flush_time=1144, avg_log_submit_to_slide_time=1570) [2024-10-16 14:57:19.565300] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:19.565318] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:19.566495] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=22] BlockGCTimerTask success(ret=0, cost_time_us=9, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:19.568375] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=25] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:19.569127] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.569148] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.569160] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.569620] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.575220] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1073}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202547} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.577580] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=10] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:57:19.578703] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1001}, {id:1002}]) [2024-10-16 14:57:19.578815] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D57-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:19.578838] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D57-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061839123295004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061839123295004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:19.580674] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061839580571000, start_id=1729061839580571000, end_id=1729061839580571001) [2024-10-16 14:57:19.580815] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D57-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061839580571000, start_id=1729061839580571000, end_id=1729061839580571001) [2024-10-16 14:57:19.582399] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B28-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061839580571000, start_id=1729061839580571000, end_id=1729061839580571001) [2024-10-16 14:57:19.584005] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1DFB-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061839582516000, start_id=1729061839582516001, end_id=1729061839582516002) [2024-10-16 14:57:19.584243] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830049064243200},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285844} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.584555] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3524}, to_seq_no={branch:0, seq:1948}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285844} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.586052] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D57-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061839585658000, start_id=1729061839585658000, end_id=1729061839585658001) [2024-10-16 14:57:19.586480] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061839585658000, start_id=1729061839585658001, end_id=1729061839585658002) [2024-10-16 14:57:19.587357] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C1A-0-0] [lt=22] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:19.587368] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C1A-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30445, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:19.587373] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C1A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=371, avg_log_submit_to_slide_time=30439) [2024-10-16 14:57:19.587391] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=33] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:19.587745] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830049089409024},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285845} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.587994] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2083}, to_seq_no={branch:0, seq:1047}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285845} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.595971] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=4512, mem_used=8318976) [2024-10-16 14:57:19.598807] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1031}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202554} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.602716] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1080}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202556} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.608418] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] Cache replace map node details(ret=0, replace_node_count=508, replace_time=0, replace_start_pos=849339, replace_num=31457) [2024-10-16 14:57:19.608439] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] replace map num details(ret=0, replace_node_count=508, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:19.608981] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830049328484352},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285853} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.609269] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2116}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285853} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.610532] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202558} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.610628] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061839610052000, start_id=1729061839610052000, end_id=1729061839610052001) [2024-10-16 14:57:19.613700] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=324871) [2024-10-16 14:57:19.614042] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=324871, hit_cnt=191, read_cnt=191, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:19.624702] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B7A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061839623917000, start_id=1729061839623917000, end_id=1729061839623917001) [2024-10-16 14:57:19.624802] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B79-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061839623917000, start_id=1729061839623917000, end_id=1729061839623917001) [2024-10-16 14:57:19.624830] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B7D-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061839623917000, start_id=1729061839623917000, end_id=1729061839623917001) [2024-10-16 14:57:19.627317] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910255326154752},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285860} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.627734] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285860} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.629643] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202566} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.633756] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202568} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.637273] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=15] LogLoopThread round_cost_time(us)(round_cost_time=9) [2024-10-16 14:57:19.638804] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910255401652224},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285864} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.639066] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1087}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285864} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.645461] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=205, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:19.645638] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=204, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:19.645656] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=205, stat_interval=1000000, avg (count/cost)=47, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:19.645687] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=205, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:19.649869] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654910255502315520},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285869} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.650015] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1077}, to_seq_no={branch:0, seq:1072}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202574} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.650109] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1077}, to_seq_no={branch:0, seq:1071}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285869} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.651276] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:19.657588] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4393}, to_seq_no={branch:0, seq:2709}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202576} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.658838] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.660006] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910255577812992},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285873} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.661094] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285873} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.664296] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.667804] INFO [PALF] check_log_checksum_ (log_group_entry_header.cpp:425) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=17] This is a padding log, no need check log checksum(bool_ret=true, data_len=4600) [2024-10-16 14:57:19.667822] INFO [PALF] calculate_log_checksum_ (log_group_entry_header.cpp:118) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=14] This is a padding log, set log data checksum to 0(data_checksum=0, data_len=4600) [2024-10-16 14:57:19.668176] INFO [CLOG] fsync_after_rename_ (ob_server_log_block_mgr.cpp:961) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] fsync_after_rename_ success(ret=0, this={dir::"/mnt/myoceanbase/oceanbase/data//clog/log_pool", dir_fd:15, meta_fd:16, log_pool_meta:{curr_total_size:32212254720, next_total_size:32212254720, status:0}, min_block_id:15037, max_block_id:15335, min_log_disk_size_for_all_tenants_:15032385536, is_inited:true}, dest_dir_fd=177) [2024-10-16 14:57:19.668193] INFO [CLOG] create_block_at (ob_server_log_block_mgr.cpp:282) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] create_new_block_at success(ret=0, this={dir::"/mnt/myoceanbase/oceanbase/data//clog/log_pool", dir_fd:15, meta_fd:16, log_pool_meta:{curr_total_size:32212254720, next_total_size:32212254720, status:0}, min_block_id:15037, max_block_id:15335, min_log_disk_size_for_all_tenants_:15032385536, is_inited:true}, dest_dir_fd=177, dest_block_path="3175") [2024-10-16 14:57:19.668802] INFO [PALF] open (log_block_handler.cpp:223) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] open block success(ret=0, this={dio_aligned_buf:{buf_write_offset:0, buf_padding_size:0, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111159821, truncate_used_ts:41410783}, log_block_size:67108864, dir_fd:177, io_fd:142}) [2024-10-16 14:57:19.668814] INFO [PALF] switch_next_block (log_block_mgr.cpp:137) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] switch_next_block success(this={log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/log", dir_fd:177, min_block_id:3154, max_block_id:3176, curr_writable_block_id:3175}) [2024-10-16 14:57:19.669296] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=0, this={dio_aligned_buf:{buf_write_offset:0, buf_padding_size:0, align_size:4096, aligned_buf_size:8192, aligned_used_ts:5455, truncate_used_ts:261}, log_block_size:67108864, dir_fd:175, io_fd:170}, aligned_buf_len=4096, aligned_block_offset=0, buf_len=4096, total_write_size_=4096, total_write_size_after_dio_=4096, ob_pwrite_used_ts=460, count=1, each_pwrite_cost=460) [2024-10-16 14:57:19.669311] INFO [PALF] update_block_header_ (log_storage.cpp:785) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] append_block_header_ success(ret=0, block_id=0, log_block_header_={magic:18754, version:1, min_lsn:{lsn:213124743168}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}) [2024-10-16 14:57:19.669320] INFO [PALF] update_manifest_used_for_meta_storage (log_storage.cpp:551) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] update_manifest_used_for_meta_storage success(ret=0, this={log_tail:{lsn:5767168}, readable_log_tail:{lsn:5767168}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213124743168}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/meta", dir_fd:175, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61337600, block_header_serialize_buf_:0x2b0e0dac69f8, flashback_version:0}) [2024-10-16 14:57:19.669334] INFO [PALF] update_manifest (log_engine.cpp:1056) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] update_max_block_id_for_switch_block_cb success(ret=0, palf_id=1002, is_inited=true, block_id=3176) [2024-10-16 14:57:19.669339] INFO [PALF] inner_switch_block_ (log_storage.cpp:741) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] inner_switch_block_ success(ret=0, log_block_header_={magic:18754, version:1, min_lsn:{lsn:212990533632}, min_scn:{val:1729059200214792001, v:0}, curr_block_id:3174, palf_id:1002}, block_id=3175) [2024-10-16 14:57:19.669618] INFO [PALF] update_block_header_ (log_storage.cpp:785) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] append_block_header_ success(ret=0, block_id=3175, log_block_header_={magic:18754, version:1, min_lsn:{lsn:213057638400}, min_scn:{val:1729061839666529002, v:0}, curr_block_id:3175, palf_id:1002}) [2024-10-16 14:57:19.670019] INFO [PALF] check_log_checksum_ (log_group_entry_header.cpp:425) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] This is a padding log, no need check log checksum(bool_ret=true, data_len=4600) [2024-10-16 14:57:19.670029] INFO [PALF] generate_padding_header_ (log_entry_header.cpp:209) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=8] generate_padding_header_ success(this={magic:19528, version:1, log_size:4568, scn_:{val:1729061839666529001, v:0}, data_checksum:3610668542, flag:2}, log_data="", base_header_len=16, padding_data_len=4568) [2024-10-16 14:57:19.670038] INFO [PALF] generate_padding_log_buf (log_entry_header.cpp:157) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] generate_padding_log_buf success(header={magic:19528, version:1, log_size:4568, scn_:{val:1729061839666529001, v:0}, data_checksum:3610668542, flag:2}, padding_data_len=4568, scn={val:1729061839666529001, v:0}, out_buf=0x2b0e7f805038, padding_valid_data_len=48) [2024-10-16 14:57:19.670046] INFO [PALF] construct_padding_log_entry_ (log_iterator_impl.h:827) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] generate padding log entry successfully(this={buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:56, curr_read_buf_start_pos:0, curr_read_buf_end_pos:4656, log_storage_:{IteratorStorage:{start_lsn:{lsn:213057633744}, end_lsn:{lsn:213057638400}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061839665470000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:1536, scn_:{val:1729061839665470000, v:0}, data_checksum:4047029688, flag:0}}, init_mode_version:0, accumulate_checksum:3433154679, curr_entry_is_padding:1, padding_entry_size:4600, padding_entry_scn:{val:1729061839666529001, v:0}}) [2024-10-16 14:57:19.670063] INFO [CLOG] fetch_and_submit_single_log_ (ob_log_replay_service.cpp:1129) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] no need to replay padding log entry(id={id:1002}, submit_task={ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061839669987, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213057633744}, next_to_submit_scn_:{val:1729061839665470001, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:56, curr_read_buf_start_pos:0, curr_read_buf_end_pos:4656, log_storage_:{IteratorStorage:{start_lsn:{lsn:213057633744}, end_lsn:{lsn:213057638400}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:4600, prev_entry_scn:{val:1729061839666529001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:4568, scn_:{val:1729061839666529001, v:0}, data_checksum:3610668542, flag:2}}, init_mode_version:0, accumulate_checksum:3433154679, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560779, miss_cnt:1, cache_read_size:152590078382, hit_ratio:"9.999999928346631206e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}, header={version:1, log_type:25, flag:0, need_pre_replay_barrier:false, need_post_replay_barrier:false, is_compressed:false, replay_hint:0}) [2024-10-16 14:57:19.672677] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3183}, to_seq_no={branch:0, seq:2121}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202578} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.674308] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061835670652, last_first_register_time_us=1722494150998907, register_time_us=1729061835670652, curr_time_us=1729061839674306) [2024-10-16 14:57:19.674339] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061839674324, last_submit_register_req_time_us=1729061839674324, register_parent_reason=3) [2024-10-16 14:57:19.674353] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:19.675793] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910255640727552},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285877} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.676192] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2143}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285877} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.679869] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33705, start_service_tstamp_=1722494263684720, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:19.686716] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061839686686000, start_id=1729061839686686000, end_id=1729061839686686001) [2024-10-16 14:57:19.687193] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061839486339000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=200849, in_cluster_service=true, cluster_version={val:1729061839285499000, v:0}, min_cluster_version={val:1729061839285499000, v:0}, max_cluster_version={val:1729061839285499000, v:0}, get_cluster_version_err=0, cluster_version_delta=401689, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65248680, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=336959903, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:19.688232] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061839687712000, start_id=1729061839687712000, end_id=1729061839687712001) [2024-10-16 14:57:19.688287] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:19.688305] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4393) [2024-10-16 14:57:19.688313] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:19.688363] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:19.689117] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061839687712000, start_id=1729061839687712001, end_id=1729061839687712002) [2024-10-16 14:57:19.689566] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=257, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:19.689579] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:19.689590] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=236, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:19.690291] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B1B-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061839689163000, start_id=1729061839689163000, end_id=1729061839689163001) [2024-10-16 14:57:19.690368] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=50] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:19.690485] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=19] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4393) [2024-10-16 14:57:19.690510] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4393) [2024-10-16 14:57:19.690659] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B04-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:19.690669] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B04-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=808, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:19.691913] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910255716225024},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285882} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.692485] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3817}, to_seq_no={branch:0, seq:2757}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285882} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.695401] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:19.697380] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1058}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202586} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.709507] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=3][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:19.709545] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910255879802880},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285890} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.710036] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=24] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2204}, to_seq_no={branch:0, seq:1052}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285890} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.710125] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:19.710389] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1156}, to_seq_no={branch:0, seq:1150}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202590} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.711389] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061839710606000, start_id=1729061839710606000, end_id=1729061839710606001) [2024-10-16 14:57:19.715421] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=10] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:19.715443] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=208) [2024-10-16 14:57:19.717405] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:19.717422] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:19.717676] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1208975360, global_cache_size=1581383680, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:19.720508] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:19.722739] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910256068546560},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285895} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.722994] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DC94A-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1133}, to_seq_no={branch:0, seq:1128}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285895} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.725298] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15CC-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:19.730681] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061839729665000, start_id=1729061839729665000, end_id=1729061839729665001) [2024-10-16 14:57:19.733587] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=4] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1004, min_version={val:1729061839123295003, v:0}, base_version={val:1729061839013297003, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061839123295004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061839733576, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061839123295003, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061839649814, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061839123295004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061839691071, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:19.734460] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061839733953000, start_id=1729061839733953000, end_id=1729061839733953001) [2024-10-16 14:57:19.734611] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3178}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202599} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.735554] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061839735011000, start_id=1729061839735011000, end_id=1729061839735011001) [2024-10-16 14:57:19.737960] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061839737220000, start_id=1729061839737220000, end_id=1729061839737220001) [2024-10-16 14:57:19.748772] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:19.748799] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=16] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:19.748807] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:19.751310] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:11277}, to_seq_no={branch:0, seq:7054}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202600} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.755515] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=21] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:57:19.755903] INFO [STORAGE.TRANS] statistics (ob_location_adapter.cpp:72) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D94-0-0] [lt=21] location adapter statistics(renew_access=0, total_access=578, error_count=0, renew_rate=0.000000000e+00) [2024-10-16 14:57:19.757369] INFO [RPC] process (ob_batch_processor.cpp:106) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3B87832C-0-0] [lt=18] batch rpc statistics(clog_batch_nodelay_cnt=0, clog_batch_cnt=0, trx_batch_cnt=1, sql_batch_cnt=0) [2024-10-16 14:57:19.758898] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.759071] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A9D-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:19.759086] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A9D-0-0] [lt=8][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178237, log_task={header:{begin_lsn:{lsn:14538461455}, end_lsn:{lsn:14538461577}, log_id:115178237, min_scn:{val:1729061839623917000, v:0}, max_scn:{val:1729061839623917000, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538461333}, prev_proposal_id:23, committed_end_lsn:{lsn:14538461455}, data_checksum:1715074492, accum_checksum:32747118, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061839658942, freeze_ts:1729061839658942, submit_ts:1729061839658946, flushed_ts:1729061839659121, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:175}, fs_cb_begin_ts=1729061839759081, log_life_time=100139) [2024-10-16 14:57:19.759135] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62649997, miss_cnt:32, cache_read_size:8050390662, hit_ratio:"9.999994892260943846e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:19.759246] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=169, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:19.759260] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:19.759289] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:19.759448] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5024, dropped:738, tid:12153}]) [2024-10-16 14:57:19.761163] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D95-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307629557, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893775, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.763406] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D97-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307631800, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893776, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.764535] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.764583] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=203, stat_interval=1000000, avg (count/cost)=343, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:19.764597] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=203, stat_interval=1000000, avg (count/cost)=37, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:19.764619] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=203, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:19.768225] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:19.772962] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D99-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307641355, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893777, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.773387] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=12] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:19.774790] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D9B-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307643176, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893778, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.774973] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=219, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:19.775703] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.775717] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.775723] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895501811712},{"BIGINT":1729058400},{"VARCHAR":"@¥", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.775779] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.776862] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D9D-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307645257, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893779, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.778422] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0D9F-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307646818, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893780, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.778470] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5054}, to_seq_no={branch:0, seq:2154}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202605} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.778545] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257666, offset=39005935}, end_cursor:ObLogCursor{file_id=197, log_id=49257667, offset=39006218}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2994050, len:4369, log_data_len:283, seq:49257666, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:19.779364] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=17] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:19.780716] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061839779044000, start_id=1729061839779044000, end_id=1729061839779044001) [2024-10-16 14:57:19.780777] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=21, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:19.780789] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=10491, avg_group_log_size=524) [2024-10-16 14:57:19.780798] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10491) [2024-10-16 14:57:19.780804] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:19.781285] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB359-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061839779044000, start_id=1729061839779044000, end_id=1729061839779044001) [2024-10-16 14:57:19.781651] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=213, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:19.781662] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10491) [2024-10-16 14:57:19.781672] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=247, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:19.781686] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10491) [2024-10-16 14:57:19.781693] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=6, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:19.781911] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D56B3-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:19.781917] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D56B3-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=792, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:19.781921] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D56B3-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=16, avg_log_gen_to_submit_time=41, avg_log_submit_to_flush_time=480, avg_log_submit_to_slide_time=750) [2024-10-16 14:57:19.782518] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257667, offset=39006218}, end_cursor:ObLogCursor{file_id=197, log_id=49257668, offset=39006500}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ab0050, len:4086, log_data_len:282, seq:49257667, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:19.785029] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DA1-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307653424, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893781, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.786801] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DA3-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307655197, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893782, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.786911] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257668, offset=39006500}, end_cursor:ObLogCursor{file_id=197, log_id=49257669, offset=39006782}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b4c050, len:3804, log_data_len:282, seq:49257668, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:19.787992] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C1E-0-0] [lt=18][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172930, log_task={header:{begin_lsn:{lsn:14352347691}, end_lsn:{lsn:14352347813}, log_id:115172930, min_scn:{val:1729061839623917001, v:0}, max_scn:{val:1729061839623917001, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352347569}, prev_proposal_id:17, committed_end_lsn:{lsn:14352347691}, data_checksum:940009244, accum_checksum:758148485, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061839687611, freeze_ts:1729061839687611, submit_ts:1729061839687614, flushed_ts:1729061839690355, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:2741}, fs_cb_begin_ts=1729061839787986, log_life_time=100375) [2024-10-16 14:57:19.788067] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656732, miss_cnt:17, cache_read_size:8050138034, hit_ratio:"9.999997286804650498e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:19.788182] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061839787915000, start_id=1729061839787915000, end_id=1729061839787915001) [2024-10-16 14:57:19.788287] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.788351] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.788359] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.788368] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.788463] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257669, offset=39006782}, end_cursor:ObLogCursor{file_id=197, log_id=49257670, offset=39007064}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a9c050, len:3522, log_data_len:282, seq:49257669, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:19.788596] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.788606] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.788615] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.788629] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.788832] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=34] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560802, miss_cnt:1, cache_read_size:152590137213, hit_ratio:"9.999999928346643419e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:19.789421] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.789442] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.790325] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:19.790320] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601300, offset=9600842}, end_cursor:ObLogCursor{file_id=192, log_id=47601301, offset=9601126}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2aac050, len:4278, log_data_len:284, seq:47601300, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:19.790938] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.790953] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.790967] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545341755346944},{"BIGINT":1729058400},{"VARCHAR":"@ª²", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.791024] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.791250] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2435}, to_seq_no={branch:0, seq:2430}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202607} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.792699] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22067][T1002_TeRec][T1002][YB4A0A000407-00061E9B225FA48E-0-0] [lt=13] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=1110000, MAX_GAP=900000000000, new_sync_scn={val:1729061838772614000, v:0}, new_replayable_scn={val:1729061838772614000, v:0}, new_readable_scn={val:1729061838771504000, v:0}, old_tenant_info={tenant_id:1002, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061837767698000, v:0}, replayable_scn:{val:1729061837767698000, v:0}, standby_scn:{val:1729061837760001000, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"NOARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:57:19.792876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48E-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061839792253000, start_id=1729061839792253000, end_id=1729061839792253001) [2024-10-16 14:57:19.793834] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601301, offset=9601126}, end_cursor:ObLogCursor{file_id=192, log_id=47601302, offset=9601400}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ae0050, len:3994, log_data_len:274, seq:47601301, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:19.800357] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.800368] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.800375] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618575574061056},{"BIGINT":1729058400},{"VARCHAR":"A ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.800435] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.800490] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601302, offset=9601400}, end_cursor:ObLogCursor{file_id=192, log_id=47601303, offset=9601674}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29d8050, len:3720, log_data_len:274, seq:47601302, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:19.800713] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601303, offset=9601674}, end_cursor:ObLogCursor{file_id=192, log_id=47601304, offset=9601948}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a18050, len:3446, log_data_len:274, seq:47601303, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:19.800821] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598383, miss_cnt:9, cache_read_size:149777362693, hit_ratio:"9.999999345922588878e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:19.800835] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4301}, to_seq_no={branch:0, seq:2121}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202608} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.803351] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86A-0-0] [lt=26] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:19.803376] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86A-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061839769745000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061839769745000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:19.805441] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA86A-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061839804964000, start_id=1729061839804964000, end_id=1729061839804964001) [2024-10-16 14:57:19.809888] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=22, replace_time=0, replace_start_pos=880796, replace_num=31457) [2024-10-16 14:57:19.809920] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=22, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:19.810053] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA86A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061839809987000, start_id=1729061839809987000, end_id=1729061839809987001) [2024-10-16 14:57:19.810668] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:19.814836] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DA5-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307683229, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893783, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.815634] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1001}, {id:1003}]) [2024-10-16 14:57:19.816349] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DA7-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307684744, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893784, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.817595] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:19.817829] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=310, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:19.817845] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=452, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:19.818224] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=45, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:57:19.818236] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=650, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:57:19.818240] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:57:19.818243] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=-468, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:57:19.818247] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=227, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:57:19.821812] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DA9-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307690206, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893785, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.824458] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.824474] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.824482] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895539560448},{"BIGINT":1729058400},{"VARCHAR":"@¡|", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.824544] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.824823] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2174}, to_seq_no={branch:0, seq:2169}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202611} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.827171] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:19.827184] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:19.827198] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52189794, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:19.827406] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DAB-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307695800, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893786, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.827999] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061839827945000, start_id=1729061839827945000, end_id=1729061839827945001) [2024-10-16 14:57:19.828065] INFO [LIB] stat (utility.h:1165) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=197, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:19.828079] INFO [LIB] stat (utility.h:1165) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=197, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:19.828365] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=336978) [2024-10-16 14:57:19.828398] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=336978) [2024-10-16 14:57:19.838509] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DAD-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307706902, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893787, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.840037] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DAF-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307708433, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893788, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.842728] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.842742] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.842749] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042454020097},{"BIGINT":1729058400},{"VARCHAR":"A¢×3â", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.842810] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.844013] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DB0-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307712408, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893789, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.844233] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4471}, to_seq_no={branch:0, seq:1205}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202613} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.849217] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DB2-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307717612, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893790, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.849545] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3069, mem_used=6239232) [2024-10-16 14:57:19.854384] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DB4-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307722779, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893791, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.855372] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=33954924, this={dio_aligned_buf:{buf_write_offset:708, buf_padding_size:3388, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111299271, truncate_used_ts:40614675}, log_block_size:67108864, dir_fd:182, io_fd:127}, aligned_buf_len=8192, aligned_block_offset=33951744, buf_len=1624, total_write_size_=921218, total_write_size_after_dio_=3182592, ob_pwrite_used_ts=133652, count=552, each_pwrite_cost=242) [2024-10-16 14:57:19.857625] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DB6-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307726019, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893792, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.858718] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.858730] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.858738] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895589892096},{"BIGINT":1729058400},{"VARCHAR":"@—ì", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.858801] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.858981] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.859248] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DB8-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307727642, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893793, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.859265] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4304}, to_seq_no={branch:0, seq:3186}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202615} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.859446] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2A9F-0-0] [lt=22] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:19.861057] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DBA-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307729453, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893794, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.863633] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=37] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:19.864247] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:19.864291] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061837234027000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:19.864598] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.875172] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DBC-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307743566, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893795, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.878691] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.878708] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.878714] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895602475008},{"BIGINT":1729058400},{"VARCHAR":"AŒ÷ëÐ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.878782] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.878975] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1030}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202619} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.882895] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DBE-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307751289, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893796, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.883133] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:19.884315] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DC0-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307752710, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893797, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.888742] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061839887925000, start_id=1729061839887925000, end_id=1729061839887925001) [2024-10-16 14:57:19.890326] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DC2-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307758721, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893798, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.890735] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061839687712001, v:0}, new_max_version={val:1729061839687712001, v:0}, last_min_version={val:1729061839486339000, v:0}, last_max_version={val:1729061839486339000, v:0}, total_time_us=2869, delta_us=203022) [2024-10-16 14:57:19.890757] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061839687712001, v:0}, min_version={val:1729061839687712001, v:0}, max_version={val:1729061839687712001, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:19.891389] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061839890448000, start_id=1729061839890448000, end_id=1729061839890448001) [2024-10-16 14:57:19.896305] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.896321] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.896327] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895615057920},{"BIGINT":1729058400},{"VARCHAR":"AxC", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.896388] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.896749] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=31] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 403.25; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 16175.73, iops: 176.60, rt: 122.15; ) [2024-10-16 14:57:19.896771] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:19.896815] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 655.50; ) [2024-10-16 14:57:19.896824] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:19.896831] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:19.897382] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5287}, to_seq_no={branch:0, seq:2106}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202621} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.906003] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=8] PNIO [ratelimit] time: 1729061839906001, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007149, add_bytes: 0 [2024-10-16 14:57:19.913364] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2277-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307781756, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893799, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.916684] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2279-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307785077, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893800, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.917976] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1587625984, tenant_max_wash_size=7525990, tenant_min_wash_size=7525990, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:19.919620] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:19.919842] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.919854] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.919866] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042533711872},{"BIGINT":1729058400},{"VARCHAR":"ÁŸ¥˜l", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.919927] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.922241] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:11343}, to_seq_no={branch:0, seq:2111}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202623} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.924450] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DC7-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307792844, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893801, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.926171] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DC9-0-0] [lt=43][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307794566, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893802, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.926529] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=179, avg_log_batch_cnt=1, total_group_log_size=307750, avg_group_log_size=1719) [2024-10-16 14:57:19.926541] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=307749) [2024-10-16 14:57:19.930120] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DCB-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307798515, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893803, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.930794] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061839929006000, start_id=1729061839929006000, end_id=1729061839929006001) [2024-10-16 14:57:19.931564] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:19.932104] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=198, stat_interval=1000000, avg (count/cost)=252, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:19.932121] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=199, stat_interval=1000000, avg (count/cost)=55, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:19.932156] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=199, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:19.934585] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=47] get number(ret=0, service_type_=0, range=1, base_id=1729061839933158000, start_id=1729061839933158000, end_id=1729061839933158001) [2024-10-16 14:57:19.935657] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=1526] get number(ret=0, service_type_=0, range=1, base_id=1729061839934906000, start_id=1729061839934906000, end_id=1729061839934906001) [2024-10-16 14:57:19.937143] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1381, mem_used=4159488) [2024-10-16 14:57:19.938117] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DCD-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307806511, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893804, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.942297] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26142][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB25CC6B7-0-0] [lt=32] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:57:19.947568] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DCF-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307815962, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893805, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.948945] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DD1-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307817341, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893806, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.949442] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1370) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2024-10-16 14:57:19.949458] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.949464] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545341805678592},{"BIGINT":1729058400},{"VARCHAR":"Anè3à", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.949519] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.949802] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5320}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202627} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.951288] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061839950941000, start_id=1729061839950941000, end_id=1729061839950941001) [2024-10-16 14:57:19.956807] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DD3-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307825202, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893807, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.957788] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061839957787, bytes: 4446799451744, bw: 3.903415 MB/s, add_ts: 1004568, add_bytes: 4111724 [2024-10-16 14:57:19.958518] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AA1-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:19.959038] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:19.959055] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:19.959061] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.964612] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:19.965106] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D0-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061839956313, keepalive_ts:-1}) [2024-10-16 14:57:19.965125] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D0-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061839956313, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:19.968957] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DD5-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307837351, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893808, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.969049] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3E72-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=182, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:19.969062] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3E72-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=182, stat_interval=1000000, avg (count/cost)=997, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:19.972755] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DD7-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307841149, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893809, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.981923] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061839981110000, start_id=1729061839981110000, end_id=1729061839981110001) [2024-10-16 14:57:19.984628] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC732-0-0] [lt=18] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:19.984645] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC732-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:19.985008] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0eeda3bb10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202633} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778320400 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, prev=this=0x2b0f65dcf858 trans_version={val:1729061778327579000, v:0} scn={val:1729061778327579000, v:0} tx_id={txid:392201602} prev=0x2b0f65dc6f18 next=(nil) modify_count=1 acc_checksum=1281247613 version=1729061778320400 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:8560}, snapshot_version={val:1729061839983227000, v:0}, *this={this=0x2b0f65dc6c10 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f65dcf858 latest_compact_node=(nil) max_trans_version={val:1729061778327579000, v:0} max_trans_id=392201602 max_elr_trans_version={val:1729061778327579000, v:0} max_elr_trans_id=392201602 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778327579000, v:0} min_modify_scn={val:1729061778327579000, v:0}}) [2024-10-16 14:57:19.985036] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0eeda3bb10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202633} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778320400 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, ctx={this:0x2b0e2b854080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606596, tablet_id:{id:209264}, table_iter:0x2b0e2b853210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606596, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061839983227000, v:0}, tx_id:{txid:392202633}, scn:{branch:0, seq:1059}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202633}, tx_desc:{this:0x2b0e0fccbf70, tx_id:{txid:392202633}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061839984679, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061839982169, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1062}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:19.985087] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=50][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202633} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202633}, data:0x2b0e2b850438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778320400, seq_no:{branch:0, seq:1062}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:19.985101] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654830042806341633},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202633} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.985113] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.985119] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042806341633},{"BIGINT":1729058400},{"VARCHAR":"B$", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.985168] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.987974] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5687}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202633} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:19.988454] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DD9-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307856848, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893810, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.991127] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061839991117000, start_id=1729061839991117000, end_id=1729061839991117001) [2024-10-16 14:57:19.992443] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DDB-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307860838, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893811, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:19.995230] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=358, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:19.995248] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=358, stat_interval=1000000, avg (count/cost)=80, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:19.995723] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f5272b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202635} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1075}, prev=this=0x2b0f7f483c30 trans_version={val:1729061778332999001, v:0} scn={val:1729061778332999001, v:0} tx_id={txid:392201604} prev=0x2b0f7f483af0 next=(nil) modify_count=1 acc_checksum=3657787693 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4338}, snapshot_version={val:1729061839994253000, v:0}, *this={this=0x2b0f7f4837e8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f483c30 latest_compact_node=(nil) max_trans_version={val:1729061778332999001, v:0} max_trans_id=392201604 max_elr_trans_version={val:1729061778332999001, v:0} max_elr_trans_id=392201604 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778332999001, v:0} min_modify_scn={val:1729061778332999001, v:0}}) [2024-10-16 14:57:19.995761] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f5272b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202635} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1075}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606262, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606262, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061839994253000, v:0}, tx_id:{txid:392202635}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202635}, tx_desc:{this:0x2b0ec37c5a90, tx_id:{txid:392202635}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061839995327, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061839994253, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1075}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:19.995825] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202635} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202635}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1075}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:19.995842] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830042923782144},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202635} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:19.995857] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:19.995862] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042923782144},{"BIGINT":1729058400},{"VARCHAR":"?à", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:19.995917] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:19.997157] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2135}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202635} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.001976] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DDD-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307870369, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893812, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.005184] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DDF-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307873579, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893813, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.011492] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=26, replace_time=0, replace_start_pos=912253, replace_num=31457) [2024-10-16 14:57:20.011512] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=26, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:20.012937] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DE1-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307881331, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893814, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.014437] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DE3-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307882832, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893815, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.014692] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061840014475000, start_id=1729061840014475000, end_id=1729061840014475001) [2024-10-16 14:57:20.014715] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061840014687], gts:1729061840014475000, latest_srr:[mts=1729061840014687]}) [2024-10-16 14:57:20.014725] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:20.014747] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=532, get_gts_with_stc_cnt=82, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=78, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:20.015306] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:20.015402] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DA7-0-0] [lt=11] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061839981110000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061855590307000, latest_log_ts:{val:1729061838929180001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:20.015460] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DA7-0-0] [lt=55] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:20.017576] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:20.017594] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:20.018950] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF88-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2192) [2024-10-16 14:57:20.018972] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF89-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:20.020149] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DE5-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307888544, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893816, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.029444] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eda130 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202639} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2114}, prev=this=0x2b0f22e786d0 trans_version={val:1729061778344796001, v:0} scn={val:1729061778344796001, v:0} tx_id={txid:392201608} prev=0x2b0f22e78590 next=(nil) modify_count=1 acc_checksum=1614307826 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2126}, snapshot_version={val:1729061840028040000, v:0}, *this={this=0x2b0f22e78288 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e786d0 latest_compact_node=(nil) max_trans_version={val:1729061778344796001, v:0} max_trans_id=392201608 max_elr_trans_version={val:1729061778344796001, v:0} max_elr_trans_id=392201608 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778344796001, v:0} min_modify_scn={val:1729061778344796001, v:0}}) [2024-10-16 14:57:20.029479] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eda130 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202639} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2114}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606271, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606271, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840028040000, v:0}, tx_id:{txid:392202639}, scn:{branch:0, seq:1056}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202639}, tx_desc:{this:0x2b0ec37a2150, tx_id:{txid:392202639}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840029098, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840026985, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2114}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.029537] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202639} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202639}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:2114}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.029555] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":675513895715721216},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202639} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.029569] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.029575] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895715721216},{"BIGINT":1729058400},{"VARCHAR":"A¾@›", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.029630] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.029866] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2117}, to_seq_no={branch:0, seq:1057}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202639} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.046669] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DEA-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307915062, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893817, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.049591] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DEC-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307917985, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893818, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.051179] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=49] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:20.051760] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=12] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:20.051801] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061830183137000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:20.052163] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DEE-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307920559, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893819, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.053817] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DF0-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307922212, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893820, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.056021] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DF2-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307924417, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893821, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.059100] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.064226] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DF4-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307932620, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893822, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.064675] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.065980] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DF6-0-0] [lt=29][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307934375, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893823, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.069529] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb1ad00 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202647} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0fada9d040 trans_version={val:1729061778397370000, v:0} scn={val:1729061778397370000, v:0} tx_id={txid:392201616} prev=0x2b0fada9cf00 next=(nil) modify_count=1 acc_checksum=3212147865 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:10899}, snapshot_version={val:1729061840068497000, v:0}, *this={this=0x2b0fada9cbf8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fada9d040 latest_compact_node=(nil) max_trans_version={val:1729061778397370000, v:0} max_trans_id=392201616 max_elr_trans_version={val:1729061778397370000, v:0} max_elr_trans_id=392201616 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778397370000, v:0} min_modify_scn={val:1729061778397370000, v:0}}) [2024-10-16 14:57:20.069573] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=44][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb1ad00 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202647} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606253, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606253, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840068497000, v:0}, tx_id:{txid:392202647}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202647}, tx_desc:{this:0x2b0ec37c3af0, tx_id:{txid:392202647}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840068497, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840068497, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.069640] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=67][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202647} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202647}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.069658] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513895791218688},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202647} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.069674] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.069680] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895791218688},{"BIGINT":1729058400},{"VARCHAR":"Á”ú° ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.069731] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.069945] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202647} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.071665] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061840070622000, start_id=1729061840070622000, end_id=1729061840070622001) [2024-10-16 14:57:20.074066] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DF8-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307942460, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893824, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.075587] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DFA-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307943982, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893825, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.081156] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get_max_applied_scn(scn={val:1729061839981110000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618299575}, palf_committed_end_scn_:{val:1729061839981110000, v:0}, last_check_scn_:{val:1729061839981110000, v:0}, max_applied_cb_scn_:{val:1729061839981110000, v:0}}) [2024-10-16 14:57:20.081183] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:20.083475] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DFC-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307951868, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893826, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.083657] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:20.084446] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:20.084473] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:20.084503] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=25] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:20.084648] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=12] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061847008855000, latest_log_ts:{val:1729061830358352001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:20.084693] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=42] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:20.084704] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:57:20.084711] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061830358352001, v:0}) [2024-10-16 14:57:20.084715] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:20.084718] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:20.084729] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:20.084738] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:57:20.084746] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:20.084751] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:20.086313] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6de86b0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202650} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3176}, prev=this=0x2b0ec6d419a0 trans_version={val:1729061778412320000, v:0} scn={val:1729061778412320000, v:0} tx_id={txid:392201619} prev=0x2b0ec6d39060 next=(nil) modify_count=1 acc_checksum=244592946 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3600}, snapshot_version={val:1729061840084111000, v:0}, *this={this=0x2b0ec6d38d58 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d419a0 latest_compact_node=(nil) max_trans_version={val:1729061778412320000, v:0} max_trans_id=392201619 max_elr_trans_version={val:1729061778412320000, v:0} max_elr_trans_id=392201619 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778412320000, v:0} min_modify_scn={val:1729061778412320000, v:0}}) [2024-10-16 14:57:20.086342] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6de86b0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202650} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3176}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606312, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606312, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840084111000, v:0}, tx_id:{txid:392202650}, scn:{branch:0, seq:2119}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202650}, tx_desc:{this:0x2b0ec37adf10, tx_id:{txid:392202650}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840086228, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840081995, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:3176}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.086408] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=66][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202650} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202650}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:3176}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.086425] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830043213189120},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202650} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.086439] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.086445] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043213189120},{"BIGINT":1729058400},{"VARCHAR":"A™Íž„", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.086504] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.087688] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5293}, to_seq_no={branch:0, seq:2120}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202650} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.088804] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061840087287000, start_id=1729061840087287000, end_id=1729061840087287001) [2024-10-16 14:57:20.090402] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0DFE-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307958795, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893827, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.092718] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061840092066000, start_id=1729061840092066000, end_id=1729061840092066001) [2024-10-16 14:57:20.094522] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E00-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307962916, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893828, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.098504] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E02-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307966899, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893829, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.101857] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eda810 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202653} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1067}, prev=this=0x2b0f22e78dd0 trans_version={val:1729061778423225000, v:0} scn={val:1729061778423225000, v:0} tx_id={txid:392201622} prev=0x2b0f22e78c90 next=(nil) modify_count=1 acc_checksum=1853355612 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5653}, snapshot_version={val:1729061840100483000, v:0}, *this={this=0x2b0f22e78988 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e78dd0 latest_compact_node=(nil) max_trans_version={val:1729061778423225000, v:0} max_trans_id=392201622 max_elr_trans_version={val:1729061778423225000, v:0} max_elr_trans_id=392201622 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778423225000, v:0} min_modify_scn={val:1729061778423225000, v:0}}) [2024-10-16 14:57:20.101893] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=35][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eda810 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202653} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1067}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606260, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606260, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840100483000, v:0}, tx_id:{txid:392202653}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202653}, tx_desc:{this:0x2b0ec37d37f0, tx_id:{txid:392202653}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840101550, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840100484, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1067}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.101947] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202653} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202653}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:1067}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.101966] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":675513895841550336},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202653} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.101977] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.101983] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895841550336},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.102027] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.102401] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15CA-0-0] [lt=8] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:20.102916] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2251}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202653} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.109944] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:20.109962] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:20.117177] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E04-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307985570, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893830, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.117421] INFO [STORAGE.TRANS] handle_request (ob_timestamp_service.cpp:175) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] handle gts request statistics(total_rt=5319, total_cnt=3010, avg_rt=1.766522749916971113e+00, avg_cnt=3.010000000000000000e+02) [2024-10-16 14:57:20.117644] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2070, avg_group_log_size=207) [2024-10-16 14:57:20.117655] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2070) [2024-10-16 14:57:20.117703] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:20.117709] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:20.118501] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1200586752, global_cache_size=1589706752, tenant_max_wash_size=7525990, tenant_min_wash_size=7525990, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:20.118718] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:20.120063] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E06-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307988458, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893831, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.121446] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E08-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307989842, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893832, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.123107] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:20.123441] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C92-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510307991835, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410510, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.124672] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=9] current monitor number(seq_=-1) [2024-10-16 14:57:20.125102] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B82-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061840123975000, start_id=1729061840123975000, end_id=1729061840123975001) [2024-10-16 14:57:20.125108] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B7E-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061840123975000, start_id=1729061840123975000, end_id=1729061840123975001) [2024-10-16 14:57:20.125175] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0B81-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061840125139000, start_id=1729061840125139000, end_id=1729061840125139001) [2024-10-16 14:57:20.125197] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B7F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061840125139000, start_id=1729061840125139000, end_id=1729061840125139001) [2024-10-16 14:57:20.126690] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=117, mem_used=2079744) [2024-10-16 14:57:20.127539] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=51] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:57:20.127552] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:57:20.127556] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:20.128158] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061855590307000, latest_log_ts:{val:1729061838929180001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:20.128669] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=21, avg_log_batch_cnt=1, total_group_log_size=11040, avg_group_log_size=525) [2024-10-16 14:57:20.128685] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11040) [2024-10-16 14:57:20.128841] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=354, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:20.128858] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=115, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:20.128865] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:20.129267] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797AD7-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:20.129274] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797AD7-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=810, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:20.130956] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061840130613000, start_id=1729061840130613000, end_id=1729061840130613001) [2024-10-16 14:57:20.132847] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb1b3e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202659} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, prev=this=0x2b0fada9d740 trans_version={val:1729061778523297000, v:0} scn={val:1729061778523297000, v:0} tx_id={txid:392201628} prev=0x2b0fada9d600 next=(nil) modify_count=1 acc_checksum=467483279 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:40117}, snapshot_version={val:1729061840131670000, v:0}, *this={this=0x2b0fada9d2f8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fada9d740 latest_compact_node=(nil) max_trans_version={val:1729061778523297000, v:0} max_trans_id=392201628 max_elr_trans_version={val:1729061778523297000, v:0} max_elr_trans_id=392201628 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778523297000, v:0} min_modify_scn={val:1729061778523297000, v:0}}) [2024-10-16 14:57:20.132885] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb1b3e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202659} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606251, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606251, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840131670000, v:0}, tx_id:{txid:392202659}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202659}, tx_desc:{this:0x2b0ec37b3df0, tx_id:{txid:392202659}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840132731, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840131670, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1062}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.132950] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=64][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202659} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202659}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1062}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.132969] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":675513895917047808},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202659} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.132981] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.132990] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895917047808},{"BIGINT":1729058400},{"VARCHAR":"A™€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.133040] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.133393] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202659} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.134978] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061840134881000, start_id=1729061840134881000, end_id=1729061840134881001) [2024-10-16 14:57:20.135098] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061839934906000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=200187, in_cluster_service=true, cluster_version={val:1729061839735011000, v:0}, min_cluster_version={val:1729061839735011000, v:0}, max_cluster_version={val:1729061839735011000, v:0}, get_cluster_version_err=0, cluster_version_delta=400082, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65243996, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=449204896, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:20.135355] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11040) [2024-10-16 14:57:20.135391] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11040) [2024-10-16 14:57:20.136095] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=105] get number(ret=0, service_type_=0, range=1, base_id=1729061840135942000, start_id=1729061840135942000, end_id=1729061840135942001) [2024-10-16 14:57:20.136430] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797AD8-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=21, avg_log_gen_to_freeze_time=23, avg_log_gen_to_submit_time=45, avg_log_submit_to_flush_time=500, avg_log_submit_to_slide_time=773) [2024-10-16 14:57:20.138108] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22edaef0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202660} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0f22e794d0 trans_version={val:1729061778544963000, v:0} scn={val:1729061778544963000, v:0} tx_id={txid:392201629} prev=0x2b0f22e79390 next=(nil) modify_count=1 acc_checksum=212383794 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:18130}, snapshot_version={val:1729061840137010000, v:0}, *this={this=0x2b0f22e79088 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e794d0 latest_compact_node=(nil) max_trans_version={val:1729061778544963000, v:0} max_trans_id=392201629 max_elr_trans_version={val:1729061778544963000, v:0} max_elr_trans_id=392201629 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778544963000, v:0} min_modify_scn={val:1729061778544963000, v:0}}) [2024-10-16 14:57:20.138139] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22edaef0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202660} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606256, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606256, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840137010000, v:0}, tx_id:{txid:392202660}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202660}, tx_desc:{this:0x2b0ec37c1b50, tx_id:{txid:392202660}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840138070, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840137010, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.138194] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202660} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202660}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.138212] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830043376766976},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202660} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.138227] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.138232] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043376766976},{"BIGINT":1729058400},{"VARCHAR":"A¨", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.138276] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.140820] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E0A-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308009214, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893833, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.141107] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3179}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202660} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.150574] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E0C-0-0] [lt=29][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308018968, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893834, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.158303] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E0E-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308026696, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893835, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.158934] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f527fa8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202663} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3221}, prev=this=0x2b0f7f48e1a8 trans_version={val:1729061778617461000, v:0} scn={val:1729061778617461000, v:0} tx_id={txid:392201632} prev=0x2b0f7f48e068 next=(nil) modify_count=1 acc_checksum=2830461726 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:40885}, snapshot_version={val:1729061840158413000, v:0}, *this={this=0x2b0f7f48dd60 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f48e1a8 latest_compact_node=(nil) max_trans_version={val:1729061778617461000, v:0} max_trans_id=392201632 max_elr_trans_version={val:1729061778617461000, v:0} max_elr_trans_id=392201632 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778617461000, v:0} min_modify_scn={val:1729061778617461000, v:0}}) [2024-10-16 14:57:20.158966] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f527fa8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202663} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3221}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606390, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606390, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840158413000, v:0}, tx_id:{txid:392202663}, scn:{branch:0, seq:3218}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202663}, tx_desc:{this:0x2b0ec37c7a30, tx_id:{txid:392202663}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840158413, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840155196, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:3221}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.159021] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202663} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202663}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:3221}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.159036] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513895954796544},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202663} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.159047] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.159052] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895954796544},{"BIGINT":1729058400},{"VARCHAR":"A®dî", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.159095] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.159163] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.159294] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:20.159309] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:20.159515] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=571, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:20.159523] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2254) [2024-10-16 14:57:20.159532] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=150, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:20.159545] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2254) [2024-10-16 14:57:20.159552] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:20.160080] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E10-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308028474, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893836, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.160276] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4281}, to_seq_no={branch:0, seq:3219}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202663} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.161569] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3CA0-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:20.161579] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3CA0-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=1723, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:20.164219] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5016-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:20.164234] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5016-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:20.164735] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.165353] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=248198) [2024-10-16 14:57:20.167756] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E12-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308036150, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893837, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.168417] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb1bac0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202665} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3018}, prev=this=0x2b0fada9de40 trans_version={val:1729061778687353000, v:0} scn={val:1729061778687353000, v:0} tx_id={txid:392201634} prev=0x2b0fada9dd00 next=(nil) modify_count=1 acc_checksum=2370878127 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:22117}, snapshot_version={val:1729061840164796001, v:0}, *this={this=0x2b0fada9d9f8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fada9de40 latest_compact_node=(nil) max_trans_version={val:1729061778687353000, v:0} max_trans_id=392201634 max_elr_trans_version={val:1729061778687353000, v:0} max_elr_trans_id=392201634 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778687353000, v:0} min_modify_scn={val:1729061778687353000, v:0}}) [2024-10-16 14:57:20.168459] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=42][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb1bac0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202665} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3018}, ctx={this:0x2b0e2b854080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437608491, tablet_id:{id:209279}, table_iter:0x2b0e2b853210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437608491, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840164796001, v:0}, tx_id:{txid:392202665}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202665}, tx_desc:{this:0x2b0e0fcc40f0, tx_id:{txid:392202665}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840167815, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840164798, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:3018}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.168514] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202665} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202665}, data:0x2b0e2b850438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:3018}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.168528] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513895979962368},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202665} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.168539] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.168545] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513895979962368},{"BIGINT":1729058400},{"VARCHAR":"AÉ™™", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.168587] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.169555] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF227D-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308037949, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893838, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.170202] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5092}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202665} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.173830] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061840173007000, start_id=1729061840173007000, end_id=1729061840173007001) [2024-10-16 14:57:20.175315] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f45e158 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202666} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1022}, prev=this=0x2b0e5abd9dd8 trans_version={val:1729061778750778000, v:0} scn={val:1729061778750778000, v:0} tx_id={txid:392201635} prev=0x2b0e5abd9c98 next=(nil) modify_count=1 acc_checksum=1065238774 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:52519}, snapshot_version={val:1729061840174084000, v:0}, *this={this=0x2b0e5abd9990 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abd9dd8 latest_compact_node=(nil) max_trans_version={val:1729061778750778000, v:0} max_trans_id=392201635 max_elr_trans_version={val:1729061778750778000, v:0} max_elr_trans_id=392201635 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778750778000, v:0} min_modify_scn={val:1729061778750778000, v:0}}) [2024-10-16 14:57:20.175348] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f45e158 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202666} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1022}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606270, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606270, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840174084000, v:0}, tx_id:{txid:392202666}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202666}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392202666}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840175105, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840174084, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1022}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.175412] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202666} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202666}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:1022}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.175429] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830043477430272},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202666} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.175444] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.175450] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043477430272},{"BIGINT":1729058400},{"VARCHAR":"AÉ™™", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.175495] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.175719] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1025}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202666} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.182064] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=46] get number(ret=0, service_type_=0, range=1, base_id=1729061840180393000, start_id=1729061840180393000, end_id=1729061840180393001) [2024-10-16 14:57:20.186126] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=295813) [2024-10-16 14:57:20.186305] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=297437) [2024-10-16 14:57:20.186788] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15C4-0-0] [lt=13] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:20.187757] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C26-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:20.188619] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22edb5d0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202668} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4329}, prev=this=0x2b0f22e79fa8 trans_version={val:1729061778819350000, v:0} scn={val:1729061778819350000, v:0} tx_id={txid:392201637} prev=0x2b0f22e79e68 next=(nil) modify_count=1 acc_checksum=1933667134 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:35341}, snapshot_version={val:1729061840185474000, v:0}, *this={this=0x2b0f22e79788 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e79fa8 latest_compact_node=(nil) max_trans_version={val:1729061778819350000, v:0} max_trans_id=392201637 max_elr_trans_version={val:1729061778819350000, v:0} max_elr_trans_id=392201637 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778819350000, v:0} min_modify_scn={val:1729061778819350000, v:0}}) [2024-10-16 14:57:20.188651] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22edb5d0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202668} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4329}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437608071, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437608071, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840185474000, v:0}, tx_id:{txid:392202668}, scn:{branch:0, seq:2198}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202668}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392202668}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840187603, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840183277, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4329}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.188744] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=92][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202668} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202668}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4329}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.188760] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830043502596096},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202668} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.188776] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.188783] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043502596096},{"BIGINT":1729058400},{"VARCHAR":"Aà", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.188839] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.190066] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6740}, to_seq_no={branch:0, seq:2199}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202668} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.192721] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AE-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:20.192745] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AE-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061839623917005, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061839623917005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:20.193987] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061840193315000, start_id=1729061840193315000, end_id=1729061840193315001) [2024-10-16 14:57:20.196916] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061840196493000, start_id=1729061840196493000, end_id=1729061840196493001) [2024-10-16 14:57:20.197264] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f45e838 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202670} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2121}, prev=this=0x2b0e5abda4d8 trans_version={val:1729061778834674000, v:0} scn={val:1729061778834674000, v:0} tx_id={txid:392201639} prev=0x2b0e5abda398 next=(nil) modify_count=1 acc_checksum=306453124 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4524}, snapshot_version={val:1729061840196493000, v:0}, *this={this=0x2b0e5abda090 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abda4d8 latest_compact_node=(nil) max_trans_version={val:1729061778834674000, v:0} max_trans_id=392201639 max_elr_trans_version={val:1729061778834674000, v:0} max_elr_trans_id=392201639 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778834674000, v:0} min_modify_scn={val:1729061778834674000, v:0}}) [2024-10-16 14:57:20.197298] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f45e838 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202670} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2121}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606286, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606286, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840196493000, v:0}, tx_id:{txid:392202670}, scn:{branch:0, seq:2118}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202670}, tx_desc:{this:0x2b0ec37cf8b0, tx_id:{txid:392202670}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840196493, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840194376, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2121}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.197354] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202670} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202670}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:2121}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.197368] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830043540344832},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202670} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.197388] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.197394] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043540344832},{"BIGINT":1729058400},{"VARCHAR":"Aݹä", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.197434] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.197632] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:20.197645] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] [APPLY STAT CB APPEND COST TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=34, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:57:20.197652] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=1134, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:57:20.197655] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:57:20.197658] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=-795, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:57:20.197662] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=2] [APPLY STAT CB TOTAL TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=373, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:57:20.197826] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3176}, to_seq_no={branch:0, seq:2119}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202670} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.200654] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061840199684000, start_id=1729061840199684000, end_id=1729061840199684001) [2024-10-16 14:57:20.200815] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E14-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308069209, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893839, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.202356] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E16-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308070750, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893840, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.203708] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E18-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308072105, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893841, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.209957] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E1A-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308078351, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893842, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.213012] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=30, replace_time=0, replace_start_pos=943710, replace_num=31457) [2024-10-16 14:57:20.213031] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=30, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:20.214467] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22edbcb0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202674} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0f22e7a9d0 trans_version={val:1729061778851883001, v:0} scn={val:1729061778851883001, v:0} tx_id={txid:392201643} prev=0x2b0f22e7a890 next=(nil) modify_count=1 acc_checksum=1474395382 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4558}, snapshot_version={val:1729061840213524000, v:0}, *this={this=0x2b0f22e7a588 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e7a9d0 latest_compact_node=(nil) max_trans_version={val:1729061778851883001, v:0} max_trans_id=392201643 max_elr_trans_version={val:1729061778851883001, v:0} max_elr_trans_id=392201643 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778851883001, v:0} min_modify_scn={val:1729061778851883001, v:0}}) [2024-10-16 14:57:20.214505] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22edbcb0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202674} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606260, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606260, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840213524000, v:0}, tx_id:{txid:392202674}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202674}, tx_desc:{this:0x2b0ec37b3df0, tx_id:{txid:392202674}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840213524, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840213524, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.214570] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=64][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202674} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202674}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.214588] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830043628425216},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202674} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.214608] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.214617] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043628425216},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.214669] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.216492] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E1C-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308084886, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893843, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.216549] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2088}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202674} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.218311] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B251ED0A1-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308086706, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893844, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.218646] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=269, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:20.218659] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=157, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:20.218678] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:20.218689] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=649, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:20.218694] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=491, avg_log_submit_to_slide_time=632) [2024-10-16 14:57:20.218730] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:20.218734] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=734, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:20.218738] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:20.219388] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3E92-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=121, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:20.219398] INFO [LIB] stat (utility.h:1165) [106968][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3E92-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=121, stat_interval=1000000, avg (count/cost)=704, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:20.221392] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f45ef18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202675} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2116}, prev=this=0x2b0e5abdabd8 trans_version={val:1729061778859421000, v:0} scn={val:1729061778859421000, v:0} tx_id={txid:392201644} prev=0x2b0e5abdaa98 next=(nil) modify_count=1 acc_checksum=282838192 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5366}, snapshot_version={val:1729061840218957000, v:0}, *this={this=0x2b0e5abda790 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abdabd8 latest_compact_node=(nil) max_trans_version={val:1729061778859421000, v:0} max_trans_id=392201644 max_elr_trans_version={val:1729061778859421000, v:0} max_elr_trans_id=392201644 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778859421000, v:0} min_modify_scn={val:1729061778859421000, v:0}}) [2024-10-16 14:57:20.221425] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f45ef18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202675} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2116}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606303, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606303, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840218957000, v:0}, tx_id:{txid:392202675}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202675}, tx_desc:{this:0x2b0ec37c1b50, tx_id:{txid:392202675}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840221072, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840218957, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2116}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.221485] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202675} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202675}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:2116}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.221505] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=0] primary key duplication conflict(key={"BIGINT":675513896068042752},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202675} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.221517] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.221522] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513896068042752},{"BIGINT":1729058400},{"VARCHAR":"@»ò", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.221571] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.221848] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2119}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202675} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.224910] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0eeda3cb08 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202676} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778320400 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, prev=this=0x2b0f65dd9aa8 trans_version={val:1729061778870937001, v:0} scn={val:1729061778870937001, v:0} tx_id={txid:392201645} prev=0x2b0f65dd9968 next=(nil) modify_count=1 acc_checksum=1955627120 version=1729061778320400 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:11515}, snapshot_version={val:1729061840224321000, v:0}, *this={this=0x2b0f65dd9660 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f65dd9aa8 latest_compact_node=(nil) max_trans_version={val:1729061778870937001, v:0} max_trans_id=392201645 max_elr_trans_version={val:1729061778870937001, v:0} max_elr_trans_id=392201645 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778870937001, v:0} min_modify_scn={val:1729061778870937001, v:0}}) [2024-10-16 14:57:20.224941] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0eeda3cb08 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202676} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778320400 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606265, tablet_id:{id:209264}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606265, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840224321000, v:0}, tx_id:{txid:392202676}, scn:{branch:0, seq:1057}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202676}, tx_desc:{this:0x2b0ec37c7a30, tx_id:{txid:392202676}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840224321, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840223265, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1060}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.225010] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=68][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202676} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202676}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778320400, seq_no:{branch:0, seq:1060}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.225027] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830043687145473},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202676} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.225038] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.225044] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043687145473},{"BIGINT":1729058400},{"VARCHAR":"AªÞè", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.225085] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.229008] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] worker thread began to shrink(tenant_->id()=1002, group_id_=1, token=0) [2024-10-16 14:57:20.230425] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E1F-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308098819, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893845, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.230742] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7208}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202676} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.233917] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106968][T1002_L0_G1][T1002][Y0-0000000000000000-0-0] [lt=4] worker thread exit(tenant_->id()=1002, workers_.get_size()=3) [2024-10-16 14:57:20.233934] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106968][T1002_L0_G1][T0][Y0-0000000000000000-0-0] [lt=13] tenant thread end_run(id_=1002, ret=0, thread_count_=191) [2024-10-16 14:57:20.234931] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f528688 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202677} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1063}, prev=this=0x2b0f7f48efa8 trans_version={val:1729061778884468001, v:0} scn={val:1729061778884468001, v:0} tx_id={txid:392201646} prev=0x2b0f7f48ee68 next=(nil) modify_count=1 acc_checksum=908011547 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:10330}, snapshot_version={val:1729061840233641000, v:0}, *this={this=0x2b0f7f48eb60 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f48efa8 latest_compact_node=(nil) max_trans_version={val:1729061778884468001, v:0} max_trans_id=392201646 max_elr_trans_version={val:1729061778884468001, v:0} max_elr_trans_id=392201646 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778884468001, v:0} min_modify_scn={val:1729061778884468001, v:0}}) [2024-10-16 14:57:20.234959] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f528688 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202677} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1063}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606247, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606247, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840233641000, v:0}, tx_id:{txid:392202677}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202677}, tx_desc:{this:0x2b0ec37afeb0, tx_id:{txid:392202677}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840234703, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840233641, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1063}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.235020] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202677} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202677}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1063}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.235034] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513896080625664},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202677} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.235048] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.235054] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513896080625664},{"BIGINT":1729058400},{"VARCHAR":"@»ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.235099] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.235364] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202677} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.243591] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E21-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308111984, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893846, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.247403] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E23-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308115797, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893847, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.247824] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6de93a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202679} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3343}, prev=this=0x2b0ec6d43e00 trans_version={val:1729061778906047000, v:0} scn={val:1729061778906047000, v:0} tx_id={txid:392201648} prev=0x2b0ec6d43cc0 next=(nil) modify_count=1 acc_checksum=973569365 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:14277}, snapshot_version={val:1729061840245202000, v:0}, *this={this=0x2b0ec6d439b8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d43e00 latest_compact_node=(nil) max_trans_version={val:1729061778906047000, v:0} max_trans_id=392201648 max_elr_trans_version={val:1729061778906047000, v:0} max_elr_trans_id=392201648 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778906047000, v:0} min_modify_scn={val:1729061778906047000, v:0}}) [2024-10-16 14:57:20.247857] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6de93a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202679} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3343}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437607707, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437607707, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840245202000, v:0}, tx_id:{txid:392202679}, scn:{branch:0, seq:1060}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202679}, tx_desc:{this:0x2b0ec37a9fd0, tx_id:{txid:392202679}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840247485, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840244143, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3343}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.247916] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202679} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202679}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:3343}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.247934] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":675513896118374400},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202679} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.247949] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.247954] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513896118374400},{"BIGINT":1729058400},{"VARCHAR":"A<žh", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.248002] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.248326] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3346}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202679} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.251164] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E25-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308119558, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893848, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.259225] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.264795] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.275231] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f528d68 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202684} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1075}, prev=this=0x2b0f7f490558 trans_version={val:1729061778946875000, v:0} scn={val:1729061778946875000, v:0} tx_id={txid:392201653} prev=0x2b0f7f490418 next=(nil) modify_count=1 acc_checksum=1323880140 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6975}, snapshot_version={val:1729061840274473000, v:0}, *this={this=0x2b0f7f48fd38 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f490558 latest_compact_node=(nil) max_trans_version={val:1729061778946875000, v:0} max_trans_id=392201653 max_elr_trans_version={val:1729061778946875000, v:0} max_elr_trans_id=392201653 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778946875000, v:0} min_modify_scn={val:1729061778946875000, v:0}}) [2024-10-16 14:57:20.275271] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=39][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f528d68 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202684} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1075}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606351, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606351, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840274473000, v:0}, tx_id:{txid:392202684}, scn:{branch:0, seq:1072}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202684}, tx_desc:{this:0x2b0ec37b1e50, tx_id:{txid:392202684}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840274473, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840273402, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1075}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.275333] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202684} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202684}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1075}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.275353] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=0] primary key duplication conflict(key={"BIGINT":654830043867500544},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202684} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.275368] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.275375] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043867500544},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.275430] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.277686] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4214}, to_seq_no={branch:0, seq:1073}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202684} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.278213] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E27-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308146606, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893849, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.280358] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E29-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308148753, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893850, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.281225] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061839981110000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=300109, in_cluster_service=true, cluster_version={val:1729061839981109999, v:0}, min_cluster_version={val:1729061839981109999, v:0}, max_cluster_version={val:1729061839981109999, v:0}, get_cluster_version_err=0, cluster_version_delta=300110, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244246, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=214677390, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:20.283357] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E2B-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308151753, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893851, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.283801] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061840283369000, start_id=1729061840283369000, end_id=1729061840283369001) [2024-10-16 14:57:20.285418] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f45f5f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202685} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0e5abdb2d8 trans_version={val:1729061778953124000, v:0} scn={val:1729061778953124000, v:0} tx_id={txid:392201654} prev=0x2b0e5abdb198 next=(nil) modify_count=1 acc_checksum=251308727 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5165}, snapshot_version={val:1729061840284427000, v:0}, *this={this=0x2b0e5abdae90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abdb2d8 latest_compact_node=(nil) max_trans_version={val:1729061778953124000, v:0} max_trans_id=392201654 max_elr_trans_version={val:1729061778953124000, v:0} max_elr_trans_id=392201654 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778953124000, v:0} min_modify_scn={val:1729061778953124000, v:0}}) [2024-10-16 14:57:20.285462] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=43][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f45f5f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202685} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606263, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606263, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840284427000, v:0}, tx_id:{txid:392202685}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202685}, tx_desc:{this:0x2b0ec37bfbb0, tx_id:{txid:392202685}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840284427, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840284427, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.285524] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202685} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202685}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.285542] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513896193871872},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202685} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.285556] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.285565] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513896193871872},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.285618] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.285974] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1025}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202685} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.287842] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C28-0-0] [lt=11][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172935, log_task={header:{begin_lsn:{lsn:14352348301}, end_lsn:{lsn:14352348423}, log_id:115172935, min_scn:{val:1729061840125139000, v:0}, max_scn:{val:1729061840125139000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352348179}, prev_proposal_id:17, committed_end_lsn:{lsn:14352348301}, data_checksum:940009244, accum_checksum:3066795269, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061840187753, freeze_ts:1729061840187753, submit_ts:1729061840187769, flushed_ts:1729061840188339, gen_to_freeze cost time:0, gen_to_submit cost time:16, submit_to_flush cost time:570}, fs_cb_begin_ts=1729061840287834, log_life_time=100081) [2024-10-16 14:57:20.289076] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061840287568000, start_id=1729061840287568000, end_id=1729061840287568001) [2024-10-16 14:57:20.289140] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=4515, avg_group_log_size=410) [2024-10-16 14:57:20.289539] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF90-0-0] [lt=25] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:20.289665] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:20.289968] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F42-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=128, avg_log_gen_to_submit_time=153, avg_log_submit_to_flush_time=544, avg_log_submit_to_slide_time=988) [2024-10-16 14:57:20.293398] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E2D-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308161792, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893852, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.295211] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=3.116081619262695312e+01, this=0x2b0e0c29f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=155, Throttle Trigger(MB)=93, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.867614137417546226e+04) [2024-10-16 14:57:20.295361] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061840294798000, start_id=1729061840294798000, end_id=1729061840294798001) [2024-10-16 14:57:20.299955] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E2F-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308168350, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893853, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.302598] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:20.302618] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=16] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53593280, tenant_id=1003) [2024-10-16 14:57:20.302630] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:20.309337] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5390, mem_used=10398720) [2024-10-16 14:57:20.309567] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E31-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308177960, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893854, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.310806] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:20.310821] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:20.310840] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373500755, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:20.318060] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:20.318087] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2070) [2024-10-16 14:57:20.318799] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1200586752, global_cache_size=1589706752, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:20.319618] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6de9d88 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202690} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2123}, prev=this=0x2b0ec6d448d8 trans_version={val:1729061778989555000, v:0} scn={val:1729061778989555000, v:0} tx_id={txid:392201659} prev=0x2b0ec6d44798 next=(nil) modify_count=1 acc_checksum=4140921357 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6609}, snapshot_version={val:1729061840318232000, v:0}, *this={this=0x2b0ec6d44490 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d448d8 latest_compact_node=(nil) max_trans_version={val:1729061778989555000, v:0} max_trans_id=392201659 max_elr_trans_version={val:1729061778989555000, v:0} max_elr_trans_id=392201659 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061778989555000, v:0} min_modify_scn={val:1729061778989555000, v:0}}) [2024-10-16 14:57:20.319652] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6de9d88 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202690} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2123}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606324, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606324, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840318232000, v:0}, tx_id:{txid:392202690}, scn:{branch:0, seq:1066}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202690}, tx_desc:{this:0x2b0ec37b5d90, tx_id:{txid:392202690}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840319289, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840317167, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2123}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.319719] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=67][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202690} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202690}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:2123}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.319734] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830044093992960},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202690} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.319753] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.319759] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044093992960},{"BIGINT":1729058400},{"VARCHAR":"Aÿ>±]", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.319811] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.320209] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2126}, to_seq_no={branch:0, seq:1067}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202690} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.320666] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=19] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:20.330969] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E33-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308199363, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893855, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.331052] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061840329898000, start_id=1729061840329898000, end_id=1729061840329898001) [2024-10-16 14:57:20.335288] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061840333295000, start_id=1729061840333295000, end_id=1729061840333295001) [2024-10-16 14:57:20.335384] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:20.336109] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061840335505000, start_id=1729061840335505000, end_id=1729061840335505001) [2024-10-16 14:57:20.336757] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=150, avg_log_batch_cnt=0, total_group_log_size=263335, avg_group_log_size=1755) [2024-10-16 14:57:20.337800] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6dea468 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202695} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1043}, prev=this=0x2b0ec6d456d8 trans_version={val:1729061779043917000, v:0} scn={val:1729061779043917000, v:0} tx_id={txid:392201664} prev=0x2b0ec6d45598 next=(nil) modify_count=1 acc_checksum=1345972271 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6483}, snapshot_version={val:1729061840336566000, v:0}, *this={this=0x2b0ec6d44ed8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d456d8 latest_compact_node=(nil) max_trans_version={val:1729061779043917000, v:0} max_trans_id=392201664 max_elr_trans_version={val:1729061779043917000, v:0} max_elr_trans_id=392201664 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779043917000, v:0} min_modify_scn={val:1729061779043917000, v:0}}) [2024-10-16 14:57:20.337832] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6dea468 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202695} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1043}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606249, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606249, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840336566000, v:0}, tx_id:{txid:392202695}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202695}, tx_desc:{this:0x2b0ec37a6090, tx_id:{txid:392202695}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840337611, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840336569, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1043}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.337892] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202695} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202695}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:1043}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.337909] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513896307118080},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202695} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.337928] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.337934] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513896307118080},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.337982] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.338443] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1046}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202695} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.338481] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] rpc statistics(total_trans_msg_count=216, total_batch_msg_count=8) [2024-10-16 14:57:20.340176] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22443][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=15][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:20.340236] WDIAG read_multi_row (ob_table_access_helper.h:238) [22443][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=20][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:20.341027] INFO delete_row (ob_table_access_helper.h:312) [22443][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1003, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061840340246) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061840340246) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:20.345575] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f45fcd8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202697} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1059}, prev=this=0x2b0e5abdb9d8 trans_version={val:1729061779065463000, v:0} scn={val:1729061779065463000, v:0} tx_id={txid:392201666} prev=0x2b0e5abdb898 next=(nil) modify_count=1 acc_checksum=2610377267 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:17131}, snapshot_version={val:1729061840343909001, v:0}, *this={this=0x2b0e5abdb590 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abdb9d8 latest_compact_node=(nil) max_trans_version={val:1729061779065463000, v:0} max_trans_id=392201666 max_elr_trans_version={val:1729061779065463000, v:0} max_elr_trans_id=392201666 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779065463000, v:0} min_modify_scn={val:1729061779065463000, v:0}}) [2024-10-16 14:57:20.345623] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=47][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f45fcd8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202697} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1059}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606404, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606404, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840343909001, v:0}, tx_id:{txid:392202697}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202697}, tx_desc:{this:0x2b0ec37a40f0, tx_id:{txid:392202697}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840344969, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840343911, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1059}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.345685] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202697} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202697}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:1059}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.345703] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513896319700992},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202697} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.345718] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.345727] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513896319700992},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.345776] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.348331] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4189}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202697} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.348486] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E35-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308216880, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893856, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.358922] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E37-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308227315, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893857, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.359212] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AA9-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:20.359223] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AA9-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27797, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:20.359228] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AA9-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=217, avg_log_submit_to_slide_time=27791) [2024-10-16 14:57:20.359236] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AA9-0-0] [lt=8][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178243, log_task={header:{begin_lsn:{lsn:14538462178}, end_lsn:{lsn:14538462300}, log_id:115178243, min_scn:{val:1729061840123975002, v:0}, max_scn:{val:1729061840123975002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538462065}, prev_proposal_id:23, committed_end_lsn:{lsn:14538462178}, data_checksum:1715074492, accum_checksum:929754474, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061840258688, freeze_ts:1729061840258688, submit_ts:1729061840258692, flushed_ts:1729061840258933, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:241}, fs_cb_begin_ts=1729061840359208, log_life_time=100520) [2024-10-16 14:57:20.359271] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:20.359298] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.359302] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:20.359314] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:20.359375] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=8, avg_log_batch_cnt=1, total_group_log_size=2095, avg_group_log_size=261) [2024-10-16 14:57:20.359402] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2521) [2024-10-16 14:57:20.364854] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.369249] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2280-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308237641, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893858, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.374652] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E3C-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308243044, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893859, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.379237] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E3E-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308247631, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893860, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.380478] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:20.380501] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=14] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=210) [2024-10-16 14:57:20.382225] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061840381058000, start_id=1729061840381058000, end_id=1729061840381058001) [2024-10-16 14:57:20.382812] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E40-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308251207, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893861, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.382952] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3B-0-0] [lt=9] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:20.382972] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3B-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061840375793000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061840375793000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:20.383613] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:20.384520] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E42-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308252915, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893862, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.384617] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C3B-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061840383929000, start_id=1729061840383929000, end_id=1729061840383929001) [2024-10-16 14:57:20.386007] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=9] ====== tenant freeze timer task ====== [2024-10-16 14:57:20.386021] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=8388608, ctx.total_memstore_used_=41943040, ctx.total_memstore_hold_=41943040, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:20.386037] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:20.386949] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E44-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308255344, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893863, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.388047] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C2A-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:20.388315] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get_max_applied_scn(scn={val:1729061840287568000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390038693}, palf_committed_end_scn_:{val:1729061840287568000, v:0}, last_check_scn_:{val:1729061840287568000, v:0}, max_applied_cb_scn_:{val:1729061840287568000, v:0}}) [2024-10-16 14:57:20.388338] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:20.389484] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22edc698 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202707} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1058}, prev=this=0x2b0f22e85270 trans_version={val:1729061779147112000, v:0} scn={val:1729061779147112000, v:0} tx_id={txid:392201676} prev=0x2b0f22e85130 next=(nil) modify_count=1 acc_checksum=526669916 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:13013}, snapshot_version={val:1729061840388230000, v:0}, *this={this=0x2b0f22e84e28 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e85270 latest_compact_node=(nil) max_trans_version={val:1729061779147112000, v:0} max_trans_id=392201676 max_elr_trans_version={val:1729061779147112000, v:0} max_elr_trans_id=392201676 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779147112000, v:0} min_modify_scn={val:1729061779147112000, v:0}}) [2024-10-16 14:57:20.389516] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22edc698 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202707} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1058}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606258, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606258, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840388230000, v:0}, tx_id:{txid:392202707}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202707}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392202707}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840389287, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840388230, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1058}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.389612] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=95][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202707} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202707}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:1058}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.389627] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513896407781376},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202707} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.389639] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.389645] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513896407781376},{"BIGINT":1729058400},{"VARCHAR":"Ac‡ ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.389696] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.390267] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202707} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.391314] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2281-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308259708, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893864, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.391950] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C3B-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061840391368000, start_id=1729061840391368000, end_id=1729061840391368001) [2024-10-16 14:57:20.393284] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E47-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308261679, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893865, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.394205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061840393521000, start_id=1729061840393521000, end_id=1729061840393521001) [2024-10-16 14:57:20.396016] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061840395661000, start_id=1729061840395661000, end_id=1729061840395661001) [2024-10-16 14:57:20.397125] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E49-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308265520, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893866, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.397148] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f4603b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202708} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1079}, prev=this=0x2b0e5abdc0d8 trans_version={val:1729061779152470000, v:0} scn={val:1729061779152470000, v:0} tx_id={txid:392201677} prev=0x2b0e5abdbf98 next=(nil) modify_count=1 acc_checksum=1318911278 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4300}, snapshot_version={val:1729061840395661000, v:0}, *this={this=0x2b0e5abdbc90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abdc0d8 latest_compact_node=(nil) max_trans_version={val:1729061779152470000, v:0} max_trans_id=392201677 max_elr_trans_version={val:1729061779152470000, v:0} max_elr_trans_id=392201677 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779152470000, v:0} min_modify_scn={val:1729061779152470000, v:0}}) [2024-10-16 14:57:20.397177] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f4603b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202708} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1079}, ctx={this:0x2b0e2b6d2080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606253, tablet_id:{id:209273}, table_iter:0x2b0e2b6d1210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606253, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840395661000, v:0}, tx_id:{txid:392202708}, scn:{branch:0, seq:1076}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202708}, tx_desc:{this:0x2b0e62630150, tx_id:{txid:392202708}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840396721, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840394586, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c231e90, mem_ctx:0x2b0e0c233410, tx_scn:{branch:0, seq:1079}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.397231] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202708} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202708}, data:0x2b0e2b6ce438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:1079}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.397246] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830044609892352},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202708} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.397258] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.397263] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044609892352},{"BIGINT":1729058400},{"VARCHAR":"B27íR", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.397303] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.397811] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2136}, to_seq_no={branch:0, seq:1077}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202708} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.405837] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E4B-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308274230, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893867, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.407537] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E4D-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308275931, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893868, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.412705] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=110, avg_log_batch_cnt=0, total_group_log_size=196124, avg_group_log_size=1782) [2024-10-16 14:57:20.414638] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=16, replace_time=0, replace_start_pos=975167, replace_num=31457) [2024-10-16 14:57:20.414655] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=16, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:20.416470] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E4F-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308284863, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893869, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.417929] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E51-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308286324, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893870, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.419591] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E53-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308287987, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893871, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.426555] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E55-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308294949, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893872, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.433266] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f460a98 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202717} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2120}, prev=this=0x2b0e5abdced8 trans_version={val:1729061779207089000, v:0} scn={val:1729061779207089000, v:0} tx_id={txid:392201686} prev=0x2b0e5abdcd98 next=(nil) modify_count=1 acc_checksum=3549808714 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:19835}, snapshot_version={val:1729061840431684000, v:0}, *this={this=0x2b0e5abdca90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abdced8 latest_compact_node=(nil) max_trans_version={val:1729061779207089000, v:0} max_trans_id=392201686 max_elr_trans_version={val:1729061779207089000, v:0} max_elr_trans_id=392201686 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779207089000, v:0} min_modify_scn={val:1729061779207089000, v:0}}) [2024-10-16 14:57:20.433308] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=42][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f460a98 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202717} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2120}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606333, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606333, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840431684000, v:0}, tx_id:{txid:392202717}, scn:{branch:0, seq:1062}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202717}, tx_desc:{this:0x2b0ec37d37f0, tx_id:{txid:392202717}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840432742, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840430623, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2120}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.433370] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202717} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202717}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:2120}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.433388] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513896508444672},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202717} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.433403] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.433409] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513896508444672},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.433469] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.433774] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2123}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202717} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.434273] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061840014475002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=419791, in_cluster_service=true, cluster_version={val:1729061839623917004, v:0}, min_cluster_version={val:1729061839623917004, v:0}, max_cluster_version={val:1729061839623917004, v:0}, get_cluster_version_err=0, cluster_version_delta=810349, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228026, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=385825787, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:20.438783] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E57-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308307176, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893873, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.439640] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f529750 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202718} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4215}, prev=this=0x2b0f7f491730 trans_version={val:1729061779229298000, v:0} scn={val:1729061779229298000, v:0} tx_id={txid:392201687} prev=0x2b0f7f4915f0 next=(nil) modify_count=1 acc_checksum=3366069880 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:12024}, snapshot_version={val:1729061840438096000, v:0}, *this={this=0x2b0f7f4912e8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f491730 latest_compact_node=(nil) max_trans_version={val:1729061779229298000, v:0} max_trans_id=392201687 max_elr_trans_version={val:1729061779229298000, v:0} max_elr_trans_id=392201687 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779229298000, v:0} min_modify_scn={val:1729061779229298000, v:0}}) [2024-10-16 14:57:20.439673] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f529750 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202718} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4215}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606291, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840438096000, v:0}, tx_id:{txid:392202718}, scn:{branch:0, seq:3156}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202718}, tx_desc:{this:0x2b0ec37b3df0, tx_id:{txid:392202718}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840439155, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840434941, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4215}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.439738] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=64][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202718} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202718}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:4215}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.439753] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830044874133504},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202718} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.439767] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.439773] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:20.442262] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E59-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308310655, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893874, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.442594] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7448}, to_seq_no={branch:0, seq:3157}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202718} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.446877] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6A-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:20.446896] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6A-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:20.459335] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.460459] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:20.463314] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7478}, to_seq_no={branch:0, seq:1219}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202721} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.464979] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.470452] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=316, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:20.470467] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=149, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:20.489254] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=43] submit log success(service_type_=0, last_id=1729061857008855000, limited_id=1729061857008855000) [2024-10-16 14:57:20.489267] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] submit log with lock success(service_type_=0, last_id=1729061857008855000, limited_id=1729061857008855000) [2024-10-16 14:57:20.489271] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061840488170000, start_id=1729061840488170000, end_id=1729061840488170001) [2024-10-16 14:57:20.490002] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061857008855000, latest_scn={val:1729061840488170001, v:0}, *this={tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061857008855000, latest_log_ts:{val:1729061840488170001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:20.490036] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=34] handle submit callback(service_type_=0, success=true, used_ts=784, log_ts={val:1729061840488170001, v:0}, limited_id=1729061857008855000) [2024-10-16 14:57:20.490043] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:20.490047] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:20.490105] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C90-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061840489463000, start_id=1729061840489463000, end_id=1729061840489463001) [2024-10-16 14:57:20.494881] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5326}, to_seq_no={branch:0, seq:3207}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202725} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.497851] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] get number(ret=0, service_type_=0, range=1, base_id=1729061840497597000, start_id=1729061840497597000, end_id=1729061840497597001) [2024-10-16 14:57:20.498929] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061840498659000, start_id=1729061840498659000, end_id=1729061840498659001) [2024-10-16 14:57:20.503690] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202728} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.505723] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3CCD-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=82, avg_log_gen_to_freeze_time=6, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=308, avg_log_submit_to_slide_time=626) [2024-10-16 14:57:20.515051] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061840514562000, start_id=1729061840514562000, end_id=1729061840514562001) [2024-10-16 14:57:20.515074] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=19] ts response task statistics(alloc_count=261363586, free_count=130681792) [2024-10-16 14:57:20.518728] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCF97-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=467, avg_log_submit_to_slide_time=695) [2024-10-16 14:57:20.519050] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1200586752, global_cache_size=1589706752, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:20.530296] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061840187603001, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=342688, in_cluster_service=true, cluster_version={val:1729061839950941000, v:0}, min_cluster_version={val:1729061839950941000, v:0}, max_cluster_version={val:1729061839950941000, v:0}, get_cluster_version_err=0, cluster_version_delta=579350, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65243946, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=511860130, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:20.531181] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061840529359000, start_id=1729061840529359000, end_id=1729061840529359001) [2024-10-16 14:57:20.531642] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5097}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202734} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.531672] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=11] ====== check clog disk timer task ====== [2024-10-16 14:57:20.531687] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4384) [2024-10-16 14:57:20.531704] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1443107984, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:20.535208] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061840533469000, start_id=1729061840533469000, end_id=1729061840533469001) [2024-10-16 14:57:20.535262] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:20.535276] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:20.536135] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061840535318000, start_id=1729061840535318000, end_id=1729061840535318001) [2024-10-16 14:57:20.546941] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D501F-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=106, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:20.546957] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D501F-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=106, stat_interval=1000000, avg (count/cost)=1606, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:20.546957] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=191881, hit_cnt=102, read_cnt=102, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:20.552034] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5187}, to_seq_no={branch:0, seq:3138}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202737} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.557535] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3147}, to_seq_no={branch:0, seq:1013}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202738} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.559349] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.563201] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=10][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:0, dropped:22, tid:24944}]) [2024-10-16 14:57:20.566066] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:20.566080] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:20.566940] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513896785268736},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202739} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.567709] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.568560] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1085}, to_seq_no={branch:0, seq:1080}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202739} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.568972] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.568992] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.569004] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.569616] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.570283] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:57:20.570294] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:57:20.570298] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:20.570911] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061857008855000, latest_log_ts:{val:1729061840488170001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:20.572804] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E7B-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308441198, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893890, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.578817] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1002}, {id:1003}]) [2024-10-16 14:57:20.582305] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513896810434560},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202741} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.582358] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061840581556000, start_id=1729061840581556000, end_id=1729061840581556001) [2024-10-16 14:57:20.582470] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=18] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:20.582809] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3227}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202741} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.586257] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830045591359488},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202742} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.586532] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1025}, to_seq_no={branch:0, seq:1020}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202742} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.588222] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C2E-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:20.588236] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C2E-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40573, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:20.588240] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C2E-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=645, avg_log_submit_to_slide_time=40568) [2024-10-16 14:57:20.588248] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C2E-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172938, log_task={header:{begin_lsn:{lsn:14352348667}, end_lsn:{lsn:14352348789}, log_id:115172938, min_scn:{val:1729061840125139003, v:0}, max_scn:{val:1729061840125139003, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352348545}, prev_proposal_id:17, committed_end_lsn:{lsn:14352348667}, data_checksum:940009244, accum_checksum:371653710, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061840488120, freeze_ts:1729061840488120, submit_ts:1729061840488125, flushed_ts:1729061840489542, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:1417}, fs_cb_begin_ts=1729061840588215, log_life_time=100095) [2024-10-16 14:57:20.588258] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:20.590211] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D58-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:20.590234] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D58-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061840123975005, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061840123975005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:20.590319] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E7D-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308458712, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893891, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.590452] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513896835600384},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202743} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.591121] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2162}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202743} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.591290] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D58-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061840591145000, start_id=1729061840591145000, end_id=1729061840591145001) [2024-10-16 14:57:20.595160] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D58-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061840594327000, start_id=1729061840594327000, end_id=1729061840594327001) [2024-10-16 14:57:20.595763] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830045624913921},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202744} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.596079] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4528, mem_used=8318976) [2024-10-16 14:57:20.596732] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E7F-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308465125, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893892, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.596981] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2078}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202744} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.600067] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061840599549000, start_id=1729061840599549000, end_id=1729061840599549001) [2024-10-16 14:57:20.612377] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E81-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308480770, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893893, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.613387] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5023-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=149033) [2024-10-16 14:57:20.613436] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=147409, hit_cnt=78, read_cnt=78, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:20.616201] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=32, replace_time=0, replace_start_pos=1006624, replace_num=31457) [2024-10-16 14:57:20.616219] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=32, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:20.620769] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E83-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308489162, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893894, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.625171] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B83-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061840625007000, start_id=1729061840625007000, end_id=1729061840625007001) [2024-10-16 14:57:20.625410] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B87-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061840625007000, start_id=1729061840625007000, end_id=1729061840625007001) [2024-10-16 14:57:20.625495] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B84-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061840625007000, start_id=1729061840625007000, end_id=1729061840625007001) [2024-10-16 14:57:20.632795] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E85-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308501188, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893895, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.635408] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get_max_applied_scn(scn={val:1729061840535318000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264512191}, palf_committed_end_scn_:{val:1729061840594327000, v:0}, last_check_scn_:{val:1729061840594327000, v:0}, max_applied_cb_scn_:{val:1729061840535318000, v:0}}) [2024-10-16 14:57:20.635433] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:20.636582] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E87-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308504976, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893896, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.640329] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E89-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308508722, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893897, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.646723] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E8B-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308515104, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893898, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.649156] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2285-0-0] [lt=1][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308517549, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893899, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.651186] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E8D-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308519580, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893900, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.651487] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:20.652768] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E8F-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308521163, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893901, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.654181] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E91-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308522577, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893902, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.655543] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E93-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308523939, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893903, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.659355] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E95-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308527750, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893904, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.659487] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513896923680768},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202749} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.659658] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.660677] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:11840}, to_seq_no={branch:0, seq:3336}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202749} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.667739] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.670018] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061840669116000, start_id=1729061840669116000, end_id=1729061840669116001) [2024-10-16 14:57:20.674662] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2287-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308543056, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893905, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.675190] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061836672348, last_first_register_time_us=1722494150998907, register_time_us=1729061836672348, curr_time_us=1729061840675189) [2024-10-16 14:57:20.675226] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061840675208, last_submit_register_req_time_us=1729061840675208, register_parent_reason=3) [2024-10-16 14:57:20.675236] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:20.676427] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E9A-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308544819, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893906, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.679295] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=70, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:20.679642] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=70, stat_interval=1000000, avg (count/cost)=240, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:20.679657] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=70, stat_interval=1000000, avg (count/cost)=27, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:20.679689] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=70, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:20.680772] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E9C-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308549165, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893907, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.682752] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0E9E-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308551142, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893908, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.686747] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0EA0-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308555142, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893909, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.689300] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061840688742000, start_id=1729061840688742000, end_id=1729061840688742001) [2024-10-16 14:57:20.689367] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=21, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:20.689381] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4332) [2024-10-16 14:57:20.689389] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:20.690049] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=3] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:20.695462] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:20.698963] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=329, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:20.698979] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:20.698991] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=51, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:20.699025] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:20.703189] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0EA1-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308571584, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893910, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.705714] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22141][T1002_TntShared][T1002][Y0-0000000000000000-0-0] [lt=12] clear dtl interm result cost(us)(clear_cost=1004, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:20.718332] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:20.718348] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:20.719353] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183768576, global_cache_size=1589706752, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:20.721514] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0EA3-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308589908, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893911, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.721582] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:20.725640] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:20.725658] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=210) [2024-10-16 14:57:20.729452] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15CE-0-0] [lt=16] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:20.730280] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B11-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4919) [2024-10-16 14:57:20.731241] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061840714044000, start_id=1729061840714044000, end_id=1729061840714044001) [2024-10-16 14:57:20.735383] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061840731365000, start_id=1729061840731365000, end_id=1729061840731365001) [2024-10-16 14:57:20.737466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=45] get number(ret=0, service_type_=0, range=1, base_id=1729061840731365000, start_id=1729061840731365001, end_id=1729061840731365002) [2024-10-16 14:57:20.737546] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061840731365000, start_id=1729061840731365000, end_id=1729061840731365001) [2024-10-16 14:57:20.737665] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:20.748781] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:20.748806] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=12] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:20.748813] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:20.757584] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=17] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061836747080, last_first_register_time_us=1722494157071428, register_time_us=1729061836747080, curr_time_us=1729061840757582) [2024-10-16 14:57:20.757626] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=22] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061840757606, last_submit_register_req_time_us=1729061840757606, register_parent_reason=3) [2024-10-16 14:57:20.757634] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:20.761156] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AB1-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:20.761260] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3EC9-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=8, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=284, avg_log_submit_to_slide_time=967) [2024-10-16 14:57:20.761261] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.762698] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0EA5-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308631092, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893912, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.768304] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:20.768987] WDIAG [PALF] is_prev_log_pid_match (log_sliding_window.cpp:3884) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=9][errcode=-4018] get_prev_log_info_ failed(tmp_ret=-4018, log_id=218496975, palf_id=1003, self="10.0.4.7:2890", log_id=218496975, lsn={lsn:207589076192}, prev_lsn={lsn:207589074568}, prev_log_pid=81) [2024-10-16 14:57:20.770397] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2288-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308638790, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893913, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.770431] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1168, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:20.770447] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:20.770535] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.770651] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:20.776067] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:20.780344] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=14] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:20.782415] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061840781464000, start_id=1729061840781464000, end_id=1729061840781464001) [2024-10-16 14:57:20.782481] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:20.782494] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=10446, avg_group_log_size=522) [2024-10-16 14:57:20.782503] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10446) [2024-10-16 14:57:20.782509] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:20.782742] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=231, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:20.782753] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10446) [2024-10-16 14:57:20.782762] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=214, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:20.782776] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10446) [2024-10-16 14:57:20.782782] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:20.783420] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F936B-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:20.783429] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F936B-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=893, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:20.783433] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F936B-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=548, avg_log_submit_to_slide_time=863) [2024-10-16 14:57:20.788772] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.788805] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.788814] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.788823] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.788848] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.789271] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.789285] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.789298] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.789551] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.789563] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.789663] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:20.801953] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0EA7-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308670347, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893914, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.802916] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513897036926976},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202757} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.803561] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22267][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3420}, to_seq_no={branch:0, seq:2238}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202757} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.804501] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656743, miss_cnt:17, cache_read_size:8050139376, hit_ratio:"9.999997286805126784e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:20.814575] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Slow write(duration=11061, write_len=4096) [2024-10-16 14:57:20.814594] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601304, offset=9601948}, end_cursor:ObLogCursor{file_id=192, log_id=47601305, offset=9602232}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2988050, len:3172, log_data_len:284, seq:47601304, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:20.817027] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=51] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:20.817592] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=19] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:20.817652] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1003}, {id:1}, {id:1001}]) [2024-10-16 14:57:20.817653] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=25] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061834390737000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:20.819695] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=22, replace_time=0, replace_start_pos=1038081, replace_num=31457) [2024-10-16 14:57:20.819711] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=22, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:20.823095] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA48F-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061840820920000, start_id=1729061840820920000, end_id=1729061840820920001) [2024-10-16 14:57:20.824234] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=676, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:20.824260] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=154, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:20.828555] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061840827560000, start_id=1729061840827560000, end_id=1729061840827560001) [2024-10-16 14:57:20.828651] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:20.828678] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:20.829058] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Slow write(duration=24340, write_len=4096) [2024-10-16 14:57:20.829072] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257670, offset=39007064}, end_cursor:ObLogCursor{file_id=197, log_id=49257671, offset=39007347}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2aa8050, len:3240, log_data_len:283, seq:49257670, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:20.829242] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=50, stat_interval=1000000, avg (count/cost)=263, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:20.829260] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=74700) [2024-10-16 14:57:20.829272] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=50, stat_interval=1000000, avg (count/cost)=56, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:20.829264] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601305, offset=9602232}, end_cursor:ObLogCursor{file_id=192, log_id=47601306, offset=9602506}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a78050, len:2888, log_data_len:274, seq:47601305, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:20.829289] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=74700) [2024-10-16 14:57:20.829297] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=50, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:20.829325] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86B-0-0] [lt=6] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:20.829347] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86B-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061840790045000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061840790045000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:20.831486] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228A-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308699879, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893915, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:20.840812] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB35A-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061840837303000, start_id=1729061840837303000, end_id=1729061840837303001) [2024-10-16 14:57:20.844428] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:20.844463] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910318555287552},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222919} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.844552] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8208}, to_seq_no={branch:0, seq:8206}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222919} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.844863] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=11] LogLoopThread round_cost_time(us)(round_cost_time=8) [2024-10-16 14:57:20.846615] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA86B-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061840846250000, start_id=1729061840846250000, end_id=1729061840846250001) [2024-10-16 14:57:20.846652] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7633-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=54, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:20.847564] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061840847309000, start_id=1729061840847309000, end_id=1729061840847309001) [2024-10-16 14:57:20.849084] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560855, miss_cnt:1, cache_read_size:152590221887, hit_ratio:"9.999999928346670064e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:20.849721] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3129, mem_used=6239232) [2024-10-16 14:57:20.859633] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=42] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:20.860215] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:20.860259] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061831822855000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:20.860303] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E06-0-0] [lt=13] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12274, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:20.860510] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E06-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12274, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061840860495, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061840860495, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061840860495, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:20.860531] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E06-0-0] [lt=21] success to process renew task(task={tenant_id:1, table_id:12274, add_timestamp:1729061840859911}, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061840860495, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061840860495, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061840860495, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:20.860777] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Slow write(duration=11791, write_len=4096) [2024-10-16 14:57:20.860787] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257671, offset=39007347}, end_cursor:ObLogCursor{file_id=197, log_id=49257672, offset=39007629}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ca0050, len:2957, log_data_len:282, seq:49257671, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:20.860969] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601306, offset=9602506}, end_cursor:ObLogCursor{file_id=192, log_id=47601307, offset=9602780}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b0c050, len:2614, log_data_len:274, seq:47601306, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:20.860978] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AB3-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:20.861018] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650008, miss_cnt:32, cache_read_size:8050391995, hit_ratio:"9.999994892261840906e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:20.861313] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.861948] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB260-0-0] [lt=12][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:20.862308] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601307, offset=9602780}, end_cursor:ObLogCursor{file_id=192, log_id=47601308, offset=9603054}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2980050, len:2340, log_data_len:274, seq:47601307, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:20.862472] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598433, miss_cnt:9, cache_read_size:149777443881, hit_ratio:"9.999999345922826466e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:20.863526] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=21][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5024, dropped:92, tid:12153}]) [2024-10-16 14:57:20.867866] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA86B-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061840867132000, start_id=1729061840867132000, end_id=1729061840867132001) [2024-10-16 14:57:20.868005] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257672, offset=39007629}, end_cursor:ObLogCursor{file_id=197, log_id=49257673, offset=39007911}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ae4050, len:2675, log_data_len:282, seq:49257672, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:20.868166] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=25] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257673, offset=39007911}, end_cursor:ObLogCursor{file_id=197, log_id=49257674, offset=39008193}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2954050, len:2393, log_data_len:282, seq:49257673, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:20.869717] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:20.870614] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.870772] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4454) [2024-10-16 14:57:20.870983] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F49-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:20.870993] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F49-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=999, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:20.876362] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392222920}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:20.876381] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:146) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] transaction rpc statistics(total_rt=8321, total_process=390, avg_rt=21) [2024-10-16 14:57:20.876421] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23] all savepoint rollback succeed(tx.tx_id={txid:392222920}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:20.876434] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222920}, start_ts=1729061840867555, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222920}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:17748}, op_sn:3, alloc_ts:1729061840849385, active_ts:1729061840867132, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062440867132, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042006989155882, first_scn:{branch:0, seq:14548}, last_scn:{branch:0, seq:14548}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840849384, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:20.877707] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBCE-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061840844263, keepalive_ts:-1}) [2024-10-16 14:57:20.877720] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBCE-0-0] [lt=12] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061840844263, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:20.880057] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB260-0-0] [lt=10][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:20.883207] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:20.885900] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:20.886107] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D43-0-0] [lt=11] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061840669116000, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838772614000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850098662000, latest_log_ts:{val:1729061833508923001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:20.886246] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D43-0-0] [lt=90] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061840837303001, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838772614000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:20.886334] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D43-0-0] [lt=40] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061840731365001, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838772614000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:20.886411] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D43-0-0] [lt=33] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061840850442000, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838772614000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:20.886450] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D43-0-0] [lt=38] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:20.890462] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061840889142000, start_id=1729061840889142000, end_id=1729061840889142001) [2024-10-16 14:57:20.892866] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=29] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:20.893736] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061840514562000, v:0}, new_max_version={val:1729061840514562000, v:0}, last_min_version={val:1729061840287568000, v:0}, last_max_version={val:1729061840287568000, v:0}, total_time_us=4070, delta_us=379173) [2024-10-16 14:57:20.893755] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061840514562000, v:0}, min_version={val:1729061840514562000, v:0}, max_version={val:1729061840514562000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:20.899129] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 3742.50; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 12288.00, iops: 1.00, rt: 103.00; ) [2024-10-16 14:57:20.899152] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:20.899193] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 9072.50; ) [2024-10-16 14:57:20.899202] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:20.899216] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:20.913614] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061840913613, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007612, add_bytes: 0 [2024-10-16 14:57:20.919651] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1198489600, global_cache_size=1589706752, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:20.919719] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:20.919932] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash memory, (tenant_id=1002, cache_size=95715328, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:57:20.920057] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash time detail, (compute_wash_size_time=116, refresh_score_time=75, wash_time=197) [2024-10-16 14:57:20.922854] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=8] Cache wash clean map node details(ret=0, clean_node_count=22, clean_time=0, clean_start_pos=377484, clean_num=62914) [2024-10-16 14:57:20.927466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [29808][T1002_L0_G0][T1002][YB4A0A000407-00061E9B2ECA2231-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061840926527000, start_id=1729061840926527000, end_id=1729061840926527001) [2024-10-16 14:57:20.932693] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061840931804000, start_id=1729061840931804000, end_id=1729061840931804001) [2024-10-16 14:57:20.933300] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:143) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3B87832C-0-0] [lt=5] transaction rpc statistics(total_rt=3248, total_process=396, avg_rt=8) [2024-10-16 14:57:20.933433] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=47] get number(ret=0, service_type_=0, range=1, base_id=1729061840932868000, start_id=1729061840932868000, end_id=1729061840932868001) [2024-10-16 14:57:20.934203] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392222921}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:20.934231] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392222921}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:20.934239] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222921}, start_ts=1729061840933961, retries=0, savepoint={branch:0, seq:5}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222921}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1057}, op_sn:3, alloc_ts:1729061840932868, active_ts:1729061840933924, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062440933924, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012331918644772, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840932867, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:20.935913] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=47, stat_interval=1000000, avg (count/cost)=224, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:20.935931] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=47, stat_interval=1000000, avg (count/cost)=93, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:20.935964] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=47, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:20.936839] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061840936049000, start_id=1729061840936049000, end_id=1729061840936049001) [2024-10-16 14:57:20.937295] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1644, mem_used=4159488) [2024-10-16 14:57:20.937709] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061840937338000, start_id=1729061840937338000, end_id=1729061840937338001) [2024-10-16 14:57:20.945128] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] ====== tenant freeze timer task ====== [2024-10-16 14:57:20.945141] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27838040, tenant_id=1004) [2024-10-16 14:57:20.945150] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:20.945880] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:20.945895] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:20.945902] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=145) [2024-10-16 14:57:20.949260] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061840948973000, start_id=1729061840948973000, end_id=1729061840948973001) [2024-10-16 14:57:20.954321] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392222922}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:20.954995] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f52ab20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202760} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4321}, prev=this=0x2b0f7f4945b8 trans_version={val:1729061779436173000, v:0} scn={val:1729061779436173000, v:0} tx_id={txid:392201729} prev=0x2b0f7f494478 next=(nil) modify_count=1 acc_checksum=3384279382 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3206}, snapshot_version={val:1729061840952168000, v:0}, *this={this=0x2b0f7f493db8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4945b8 latest_compact_node=(nil) max_trans_version={val:1729061779436173000, v:0} max_trans_id=392201729 max_elr_trans_version={val:1729061779436173000, v:0} max_elr_trans_id=392201729 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779436173000, v:0} min_modify_scn={val:1729061779436173000, v:0}}) [2024-10-16 14:57:20.955026] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f52ab20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202760} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4321}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606593, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606593, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840952168000, v:0}, tx_id:{txid:392202760}, scn:{branch:0, seq:2137}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202760}, tx_desc:{this:0x2b0ec37a40f0, tx_id:{txid:392202760}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061840954352, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840950032, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4321}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:20.955081] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202760} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202760}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:4321}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:20.955095] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830045910126594},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202760} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:20.955111] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:20.955116] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045910126594},{"BIGINT":1729058400},{"VARCHAR":"B"šÐ:", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:20.955163] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:20.955183] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=49] all savepoint rollback succeed(tx.tx_id={txid:392222922}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:20.955198] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222922}, start_ts=1729061840950171, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222922}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:13982}, op_sn:3, alloc_ts:1729061840936049, active_ts:1729061840950032, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062440950032, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007073759179, first_scn:{branch:0, seq:10812}, last_scn:{branch:0, seq:10812}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840936050, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:20.959082] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061840959080, bytes: 4446800700375, bw: 1.189250 MB/s, add_ts: 1001293, add_bytes: 1248631 [2024-10-16 14:57:20.959111] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AB5-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:20.960607] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=12] ====== tenant freeze timer task ====== [2024-10-16 14:57:20.961376] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.970675] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:20.971019] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:20105}, to_seq_no={branch:0, seq:2138}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202760} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:20.972855] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E99-0-0] [lt=15] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061840940608, keepalive_ts:-1}) [2024-10-16 14:57:20.972879] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E99-0-0] [lt=23] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061840940608, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:20.973021] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E98-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061840940570, keepalive_ts:-1}) [2024-10-16 14:57:20.973028] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E98-0-0] [lt=6] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061840940570, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:20.979910] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061840978626000, start_id=1729061840978626000, end_id=1729061840978626001) [2024-10-16 14:57:20.979985] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=43, avg_log_batch_cnt=1, total_group_log_size=69828, avg_group_log_size=1623) [2024-10-16 14:57:20.979995] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=69828) [2024-10-16 14:57:20.985408] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061840978626000, start_id=1729061840978626000, end_id=1729061840978626001) [2024-10-16 14:57:21.002870] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f52b200 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202761} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:15363}, prev=this=0x2b0f7f495090 trans_version={val:1729061779439868001, v:0} scn={val:1729061779439868001, v:0} tx_id={txid:392201730} prev=0x2b0f7f494f50 next=(nil) modify_count=1 acc_checksum=3703861773 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2153}, snapshot_version={val:1729061840986336000, v:0}, *this={this=0x2b0f7f494870 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f495090 latest_compact_node=(nil) max_trans_version={val:1729061779439868001, v:0} max_trans_id=392201730 max_elr_trans_version={val:1729061779439868001, v:0} max_elr_trans_id=392201730 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779439868001, v:0} min_modify_scn={val:1729061779439868001, v:0}}) [2024-10-16 14:57:21.002910] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=39][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f52b200 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202761} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:15363}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606547, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606547, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061840986336000, v:0}, tx_id:{txid:392202761}, scn:{branch:0, seq:5897}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202761}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392202761}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841002758, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061840986336, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:15363}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.002984] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=74][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202761} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202761}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:15363}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.003005] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513897087258624},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202761} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.003021] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.003027] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513897087258624},{"BIGINT":1729058400},{"VARCHAR":"AØäoÔÀ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.003081] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.004144] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:17479}, to_seq_no={branch:0, seq:5898}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202761} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.010856] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3ED4-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=50, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:21.010874] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3ED4-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=50, stat_interval=1000000, avg (count/cost)=5023, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:21.011028] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=93, stat_interval=1000000, avg (count/cost)=84, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:21.011037] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=93, stat_interval=1000000, avg (count/cost)=96, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:21.015097] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061841014488000, start_id=1729061841014488000, end_id=1729061841014488001) [2024-10-16 14:57:21.015468] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb1dd20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222923} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac0cf0 trans_version={val:1729061780810787001, v:0} scn={val:1729061780810787001, v:0} tx_id={txid:392222302} prev=0x2b0fadac0bb0 next=(nil) modify_count=1 acc_checksum=3437063583 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841014488001, v:0}, *this={this=0x2b0fadac08a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac0cf0 latest_compact_node=(nil) max_trans_version={val:1729061780810787001, v:0} max_trans_id=392222302 max_elr_trans_version={val:1729061780810787001, v:0} max_elr_trans_id=392222302 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780810787001, v:0} min_modify_scn={val:1729061780810787001, v:0}}) [2024-10-16 14:57:21.015510] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb1dd20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222923} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841014488001, v:0}, tx_id:{txid:392222923}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222923}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222923}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841014488, active_ts:1729061841014488, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841014487, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.015583] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=72][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222923} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222923}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.015606] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654910318593036288},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222923} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.015619] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.015625] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318593036288},{"BIGINT":1729058400},{"VARCHAR":"AG˜Y€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.015683] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.015717] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222923} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.015936] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061841014488000, start_id=1729061841014488000, end_id=1729061841014488001) [2024-10-16 14:57:21.015952] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:21.015972] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061841015969], gts:1729061841014488000, latest_srr:[mts=1729061841015969]}) [2024-10-16 14:57:21.018337] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f52b8e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222924} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f7f4ba270 trans_version={val:1729061780811885001, v:0} scn={val:1729061780811885001, v:0} tx_id={txid:392222303} prev=0x2b0f7f4ba130 next=(nil) modify_count=1 acc_checksum=791202986 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841017317001, v:0}, *this={this=0x2b0f7f4b9e28 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4ba270 latest_compact_node=(nil) max_trans_version={val:1729061780811885001, v:0} max_trans_id=392222303 max_elr_trans_version={val:1729061780811885001, v:0} max_elr_trans_id=392222303 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780811885001, v:0} min_modify_scn={val:1729061780811885001, v:0}}) [2024-10-16 14:57:21.018365] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f52b8e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222924} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209285}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841017317001, v:0}, tx_id:{txid:392222924}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222924}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222924}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841017317, active_ts:1729061841017317, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841017316, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.018419] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222924} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222924}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.018433] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513902875398144},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222924} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.018444] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.018449] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513902875398144},{"BIGINT":1729058400},{"VARCHAR":"A ëX", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.018490] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.018517] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222924} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.019464] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f462618 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222925} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0e5abfda08 trans_version={val:1729061780813156001, v:0} scn={val:1729061780813156001, v:0} tx_id={txid:392222304} prev=0x2b0e5abfd8c8 next=(nil) modify_count=1 acc_checksum=603884949 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841018751000, v:0}, *this={this=0x2b0e5abfd5c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abfda08 latest_compact_node=(nil) max_trans_version={val:1729061780813156001, v:0} max_trans_id=392222304 max_elr_trans_version={val:1729061780813156001, v:0} max_elr_trans_id=392222304 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780813156001, v:0} min_modify_scn={val:1729061780813156001, v:0}}) [2024-10-16 14:57:21.019493] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f462618 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222925} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841018751000, v:0}, tx_id:{txid:392222925}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222925}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222925}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841018751, active_ts:1729061841018751, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841018752, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.019539] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=46][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222925} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222925}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.019555] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654910318618202112},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222925} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.019568] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.019572] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318618202112},{"BIGINT":1729058400},{"VARCHAR":"BO¼¨=ž", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.019624] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.019644] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222925} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.019967] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:21.019982] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:21.020325] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:21.020400] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFA6-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:21.021398] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] Cache replace map node details(ret=0, replace_node_count=11, replace_time=0, replace_start_pos=1069538, replace_num=31457) [2024-10-16 14:57:21.021411] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=11, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:21.021771] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb1e400 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222926} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac13f0 trans_version={val:1729061780814238001, v:0} scn={val:1729061780814238001, v:0} tx_id={txid:392222305} prev=0x2b0fadac12b0 next=(nil) modify_count=1 acc_checksum=380282238 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841021371000, v:0}, *this={this=0x2b0fadac0fa8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac13f0 latest_compact_node=(nil) max_trans_version={val:1729061780814238001, v:0} max_trans_id=392222305 max_elr_trans_version={val:1729061780814238001, v:0} max_elr_trans_id=392222305 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780814238001, v:0} min_modify_scn={val:1729061780814238001, v:0}}) [2024-10-16 14:57:21.021800] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb1e400 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222926} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841021371000, v:0}, tx_id:{txid:392222926}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222926}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222926}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841021371, active_ts:1729061841021371, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841021372, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.021844] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=44][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222926} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222926}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.021861] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":675513902900563968},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222926} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.021874] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.021878] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513902900563968},{"BIGINT":1729058400},{"VARCHAR":"@A€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.021914] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.021944] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222926} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.022759] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0e885b3e40 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202762} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6201}, prev=this=0x2b0e88551bb0 trans_version={val:1729061779445285000, v:0} scn={val:1729061779445285000, v:0} tx_id={txid:392201731} prev=0x2b0e88551a70 next=(nil) modify_count=1 acc_checksum=1256873080 version=1729061779336102 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2127}, snapshot_version={val:1729061841017317000, v:0}, *this={this=0x2b0e88551768 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e88551bb0 latest_compact_node=(nil) max_trans_version={val:1729061779445285000, v:0} max_trans_id=392201731 max_elr_trans_version={val:1729061779445285000, v:0} max_elr_trans_id=392201731 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779445285000, v:0} min_modify_scn={val:1729061779445285000, v:0}}) [2024-10-16 14:57:21.022783] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0e885b3e40 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202762} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6201}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437610444, tablet_id:{id:209276}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437610444, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841017317000, v:0}, tx_id:{txid:392202762}, scn:{branch:0, seq:1063}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202762}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392202762}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841022455, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841016255, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:6201}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.022828] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=45][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202762} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202762}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779336102, seq_no:{branch:0, seq:6201}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.022841] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=0] primary key duplication conflict(key={"BIGINT":654830045964652545},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202762} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.022850] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.022855] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045964652545},{"BIGINT":1729058400},{"VARCHAR":"BR±HñdÀ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.022886] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.025208] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8330}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202762} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.028617] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061841027745000, start_id=1729061841027745000, end_id=1729061841027745001) [2024-10-16 14:57:21.028760] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392222927}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.030450] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392222927}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.030466] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222927}, start_ts=1729061841028324, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222927}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2128}, op_sn:3, alloc_ts:1729061841023518, active_ts:1729061841027745, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441027745, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332009800303, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841022457, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.036407] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:21.036424] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:21.036435] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291844581, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:21.036890] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1729061841035983000, start_id=1729061841035983000, end_id=1729061841035983001) [2024-10-16 14:57:21.036970] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f462cf8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222928} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0e5abfe108 trans_version={val:1729061780819268001, v:0} scn={val:1729061780819268001, v:0} tx_id={txid:392222307} prev=0x2b0e5abfdfc8 next=(nil) modify_count=1 acc_checksum=2322580161 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841035983000, v:0}, *this={this=0x2b0e5abfdcc0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abfe108 latest_compact_node=(nil) max_trans_version={val:1729061780819268001, v:0} max_trans_id=392222307 max_elr_trans_version={val:1729061780819268001, v:0} max_elr_trans_id=392222307 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780819268001, v:0} min_modify_scn={val:1729061780819268001, v:0}}) [2024-10-16 14:57:21.036996] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f462cf8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222928} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841035983000, v:0}, tx_id:{txid:392222928}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222928}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222928}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841035983, active_ts:1729061841035983, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841035982, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.037050] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222928} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222928}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.037064] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513902925729792},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222928} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.037075] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.037081] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513902925729792},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.037124] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.037148] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222928} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.037986] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f4633d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222929} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0e5abfe808 trans_version={val:1729061780821595001, v:0} scn={val:1729061780821595001, v:0} tx_id={txid:392222308} prev=0x2b0e5abfe6c8 next=(nil) modify_count=1 acc_checksum=3732697445 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1064}, snapshot_version={val:1729061841037376000, v:0}, *this={this=0x2b0e5abfe3c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abfe808 latest_compact_node=(nil) max_trans_version={val:1729061780821595001, v:0} max_trans_id=392222308 max_elr_trans_version={val:1729061780821595001, v:0} max_elr_trans_id=392222308 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780821595001, v:0} min_modify_scn={val:1729061780821595001, v:0}}) [2024-10-16 14:57:21.038015] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f4633d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222929} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841037376000, v:0}, tx_id:{txid:392222929}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222929}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222929}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841037376, active_ts:1729061841037376, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841037375, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.038070] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222929} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222929}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.038086] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654910318681116672},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222929} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.038099] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.038104] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318681116672},{"BIGINT":1729058400},{"VARCHAR":"AWQ"À", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.038141] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.038164] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222929} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.043011] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=12] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:21.053099] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061841052773000, start_id=1729061841052773000, end_id=1729061841052773001) [2024-10-16 14:57:21.061464] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.062331] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:468) [21535][T1_SSTableDefra][T1][Y0-0000000000000000-0-0] [lt=7] skip defragment task(ret=0, need_defragment=false) [2024-10-16 14:57:21.062345] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [21535][T1_SSTableDefra][T1][Y0-0000000000000000-0-0] [lt=12] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=15) [2024-10-16 14:57:21.062350] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [21535][T1_SSTableDefra][T1][Y0-0000000000000000-0-0] [lt=5] successfully defragment data blocks(rewrite_cnt=0) [2024-10-16 14:57:21.062355] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [21535][T1_SSTableDefra][T1][Y0-0000000000000000-0-0] [lt=3] tablet skip filter destructed(total_cnt=0, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105366175, total_skip_cnt=310289, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:57:21.062777] INFO [SQL.PC] need_check_schema_version (ob_plan_cache_value.cpp:2237) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9A11E7854D-0-0] [lt=17] need check schema(new_schema_version=1722495466719360, cached_tenant_schema_version=1722495466719360, is_contain_synonym()=false, contain_sys_name_table_=true, is_contain_tmp_tbl()=false, is_contain_sys_pl_object()=false, pc_ctx.sql_ctx_.session_info_->get_has_temp_table_flag()=false, need_check=true, constructed_sql_=SELECT * FROM __all_ls_meta_table WHERE tenant_id = ? ORDER BY tenant_id, ls_id, svr_ip, svr_port) [2024-10-16 14:57:21.063772] WDIAG [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1862) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-4012] tx rpc condition wakeup(ret=-4012, tx.tx_id_={txid:392222930}, waittime=10000, rpc_ret=0, expire_ts=1729062440767291, remain=[{ls_id:{id:1003}, exec_epoch:23042007176474301, transfer_epoch:0}], remain_cnt=1, retries=0, tx.state=4) [2024-10-16 14:57:21.068308] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] handle_sp_rollback_resp(tx_id={txid:392222930}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.068391] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] all savepoint rollback succeed(tx.tx_id={txid:392222930}, remain_cnt=1, waittime=20000, retries=1) [2024-10-16 14:57:21.068405] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222930}, start_ts=1729061841052947, retries=1, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222930}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:13278}, op_sn:3, alloc_ts:1729061841038439, active_ts:1729061841052773, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441052773, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007176474301, first_scn:{branch:0, seq:12219}, last_scn:{branch:0, seq:12219}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841038438, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.068533] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392222930}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.068540] WDIAG [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2347) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6][errcode=-4018] get trans_desc fail(ret=-4018, tx_id={txid:392222930}) [2024-10-16 14:57:21.068545] WDIAG [STORAGE.TRANS] process (ob_trans_rpc.cpp:146) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5][errcode=-4018] handle txn message fail(ret=-4018, ret="OB_ENTRY_NOT_EXIST", msg={txMsg:{type:67, cluster_version:17180066304, tenant_id:1002, tx_id:{txid:392222930}, receiver:{id:9223372036854775807}, sender:{id:1003}, sender_addr:"10.0.4.9:2890", epoch:23042007176474301, transfer_epoch:-1, request_id:1722494171415185, timestamp:1729061841036313, cluster_id:1717559948}, ret:0, orig_epoch:23042007176474301, downstream_parts:[]}) [2024-10-16 14:57:21.070761] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.075551] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392222931}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.075591] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=56] all savepoint rollback succeed(tx.tx_id={txid:392222931}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.075609] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222931}, start_ts=1729061841075332, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222931}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1014}, op_sn:3, alloc_ts:1729061841074087, active_ts:1729061841075100, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441075100, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332059828023, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841074086, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.078936] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392222932}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.078985] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=46] all savepoint rollback succeed(tx.tx_id={txid:392222932}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.078999] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222932}, start_ts=1729061841078752, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222932}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1056}, op_sn:3, alloc_ts:1729061841077225, active_ts:1729061841078286, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441078286, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332063465845, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841077230, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.081749] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:21.081937] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D41-0-0] [lt=13] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061841014488000, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838625138004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850721265000, latest_log_ts:{val:1729061834119131001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:21.082107] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D41-0-0] [lt=90] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061841014488000, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838625138004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:21.082244] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D41-0-0] [lt=62] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061840625007004, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838625138004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:21.082409] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D41-0-0] [lt=81] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061840626063003, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838625138004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:21.082471] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D41-0-0] [lt=62] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:21.086774] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:21.086787] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=152413380, tenant_id=1) [2024-10-16 14:57:21.086800] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:21.090711] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061841089900000, start_id=1729061841089900000, end_id=1729061841089900001) [2024-10-16 14:57:21.110018] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:21.110039] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:21.120083] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:21.120104] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:21.120154] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:21.120163] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:21.120470] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:21.123025] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392222933}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.123104] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392222933}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.123114] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222933}, start_ts=1729061841121391, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222933}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:40078}, op_sn:3, alloc_ts:1729061841080363, active_ts:1729061841120440, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441120440, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007215568493, first_scn:{branch:0, seq:9541}, last_scn:{branch:0, seq:9541}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841080362, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.123145] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:21.123237] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:21.124758] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=10] current monitor number(seq_=-1) [2024-10-16 14:57:21.125587] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B88-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061841124610000, start_id=1729061841124610000, end_id=1729061841124610001) [2024-10-16 14:57:21.125691] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B8C-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061841125665000, start_id=1729061841125665000, end_id=1729061841125665001) [2024-10-16 14:57:21.127486] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C93-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510308995879, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410511, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:21.127658] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=275, mem_used=2079744) [2024-10-16 14:57:21.127797] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B89-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061841127762000, start_id=1729061841127762000, end_id=1729061841127762001) [2024-10-16 14:57:21.131791] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10339, avg_group_log_size=574) [2024-10-16 14:57:21.131807] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10339) [2024-10-16 14:57:21.135982] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061841130910000, start_id=1729061841130910000, end_id=1729061841130910001) [2024-10-16 14:57:21.136906] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061841130910000, start_id=1729061841130910000, end_id=1729061841130910001) [2024-10-16 14:57:21.136977] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=570, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:21.136989] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9746) [2024-10-16 14:57:21.137000] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=310, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:21.137011] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10339) [2024-10-16 14:57:21.137101] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:21.137108] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=2164, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:21.137113] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=4, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=965, avg_log_submit_to_slide_time=2179) [2024-10-16 14:57:21.137120] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:21.137267] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B900-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061841136948000, start_id=1729061841136948000, end_id=1729061841136948001) [2024-10-16 14:57:21.138745] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061841136948000, start_id=1729061841136948001, end_id=1729061841136948002) [2024-10-16 14:57:21.141888] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:21.145024] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6dedb30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202766} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6981}, prev=this=0x2b0ec6d4aef8 trans_version={val:1729061779461413000, v:0} scn={val:1729061779461413000, v:0} tx_id={txid:392201735} prev=0x2b0ec6d4adb8 next=(nil) modify_count=1 acc_checksum=190482087 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6607}, snapshot_version={val:1729061841138840000, v:0}, *this={this=0x2b0ec6d4aab0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d4aef8 latest_compact_node=(nil) max_trans_version={val:1729061779461413000, v:0} max_trans_id=392201735 max_elr_trans_version={val:1729061779461413000, v:0} max_elr_trans_id=392201735 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779461413000, v:0} min_modify_scn={val:1729061779461413000, v:0}}) [2024-10-16 14:57:21.145058] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6dedb30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202766} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6981}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606567, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606567, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841138840000, v:0}, tx_id:{txid:392202766}, scn:{branch:0, seq:2953}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202766}, tx_desc:{this:0x2b0ec37bfbb0, tx_id:{txid:392202766}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841143929, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841136951, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:6981}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.145117] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202766} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202766}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:6981}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.145136] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":654830046107258880},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202766} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.145155] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.145162] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046107258880},{"BIGINT":1729058400},{"VARCHAR":"B¿aˆ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.145218] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.147128] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:9219}, to_seq_no={branch:0, seq:2954}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202766} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.149093] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=42] get number(ret=0, service_type_=0, range=1, base_id=1729061841148288000, start_id=1729061841148288000, end_id=1729061841148288001) [2024-10-16 14:57:21.160820] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:21.160846] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:21.161537] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.161626] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=290, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:21.161642] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1547) [2024-10-16 14:57:21.161653] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:21.161659] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1547) [2024-10-16 14:57:21.161669] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:21.161788] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3EE1-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:21.161796] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3EE1-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=942, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:21.163372] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061841162437000, start_id=1729061841162437000, end_id=1729061841162437001) [2024-10-16 14:57:21.164982] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5017-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:21.164997] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5017-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:21.169636] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=11] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:21.169654] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:21.169662] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:21.169665] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:21.169668] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:21.169671] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16721) [2024-10-16 14:57:21.170283] WDIAG [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1862) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=-4012] tx rpc condition wakeup(ret=-4012, tx.tx_id_={txid:392222934}, waittime=10000, rpc_ret=0, expire_ts=1729062440767291, remain=[{ls_id:{id:1003}, exec_epoch:23042007284164182, transfer_epoch:0}], remain_cnt=1, retries=0, tx.state=4) [2024-10-16 14:57:21.170819] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.172935] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392222934}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.172962] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] all savepoint rollback succeed(tx.tx_id={txid:392222934}, remain_cnt=1, waittime=20000, retries=1) [2024-10-16 14:57:21.172975] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222934}, start_ts=1729061841160190, retries=1, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222934}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:10987}, op_sn:3, alloc_ts:1729061841148288, active_ts:1729061841159274, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441159274, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007284164182, first_scn:{branch:0, seq:3212}, last_scn:{branch:0, seq:3212}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841148287, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.176361] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392222934}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.176378] WDIAG [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2347) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=-4018] get trans_desc fail(ret=-4018, tx_id={txid:392222934}) [2024-10-16 14:57:21.176384] WDIAG [STORAGE.TRANS] process (ob_trans_rpc.cpp:146) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5][errcode=-4018] handle txn message fail(ret=-4018, ret="OB_ENTRY_NOT_EXIST", msg={txMsg:{type:67, cluster_version:17180066304, tenant_id:1002, tx_id:{txid:392222934}, receiver:{id:9223372036854775807}, sender:{id:1003}, sender_addr:"10.0.4.9:2890", epoch:23042007284164182, transfer_epoch:-1, request_id:1722494171415189, timestamp:1729061841131241, cluster_id:1717559948}, ret:0, orig_epoch:23042007284164182, downstream_parts:[]}) [2024-10-16 14:57:21.177157] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061841176119000, start_id=1729061841176119000, end_id=1729061841176119001) [2024-10-16 14:57:21.185632] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061841184549000, start_id=1729061841184549000, end_id=1729061841184549001) [2024-10-16 14:57:21.189151] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=66572) [2024-10-16 14:57:21.195414] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=11] set tenant work area memory(tenant_id=1001, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:21.195474] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=17] set tenant work area memory(tenant_id=1002, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:21.195517] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7] set tenant work area memory(tenant_id=1003, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:21.195560] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6] set tenant work area memory(tenant_id=1004, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:21.195679] EDIAG [SHARE.CONFIG] check_500_tenant_hold (ob_server_config.cpp:438) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] the hold memory of tenant_500 is over the reserved memory(hold=1082318848, reserved=1073741824) BACKTRACE:0x1a5796b1 0x6847325 0x6a4dfba 0x6a4d9a4 0x6a4d8e0 0x6a4d707 0x1479497f 0x11dea49b 0x6809ea1 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d [2024-10-16 14:57:21.195698] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:32) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=18] === Run print tenant memory usage task === [2024-10-16 14:57:21.195730] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:102) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4] ====== tenants memory info ====== === TENANTS MEMORY INFO === divisive_memory_used= 220,549,120 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,082,318,848 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 19,554,304 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 17,457,152 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 1 now= 1,729,061,841,195,227 active_memstore_used= 247,463,936 total_memstore_used= 245,366,784 total_memstore_hold= 245,366,784 memstore_freeze_trigger_limit= 152,413,380 memstore_limit= 858,993,440 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 1,976,012,800 max_mem_memstore_can_get_now= 762,066,934 memstore_alloc_pos= 20,002,635,776 memstore_frozen_pos= 19,755,171,840 memstore_reclaimed_pos= 19,757,268,992 [TENANT_MEMORY] tenant_id= 1,001 now= 1,729,061,841,195,227 active_memstore_used= 48,234,496 total_memstore_used= 44,040,192 total_memstore_hold= 44,040,192 memstore_freeze_trigger_limit= 54,432,140 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 902,713,344 max_mem_memstore_can_get_now= 272,160,742 memstore_alloc_pos= 32,717,668,352 memstore_frozen_pos= 32,669,433,856 memstore_reclaimed_pos= 32,673,628,160 [TENANT_MEMORY] tenant_id= 1,002 now= 1,729,061,841,195,227 active_memstore_used= 41,943,040 total_memstore_used= 41,943,040 total_memstore_hold= 41,943,040 memstore_freeze_trigger_limit= 28,020,300 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 965,505,024 max_mem_memstore_can_get_now= 140,101,590 memstore_alloc_pos= 465,949,425,664 memstore_frozen_pos= 465,907,482,624 memstore_reclaimed_pos= 465,907,482,624 [TENANT_MEMORY] tenant_id= 1,003 now= 1,729,061,841,195,227 active_memstore_used= 60,817,408 total_memstore_used= 60,817,408 total_memstore_hold= 60,817,408 memstore_freeze_trigger_limit= 53,593,280 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 938,364,928 max_mem_memstore_can_get_now= 267,966,458 memstore_alloc_pos= 34,273,755,136 memstore_frozen_pos= 34,212,937,728 memstore_reclaimed_pos= 34,212,937,728 [TENANT_MEMORY] tenant_id= 1,004 now= 1,729,061,841,195,227 active_memstore_used= 39,845,888 total_memstore_used= 39,845,888 total_memstore_hold= 39,845,888 memstore_freeze_trigger_limit= 27,838,040 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 954,744,832 max_mem_memstore_can_get_now= 139,190,238 memstore_alloc_pos= 1,602,224,128 memstore_frozen_pos= 1,562,378,240 memstore_reclaimed_pos= 1,562,378,240 [2024-10-16 14:57:21.195800] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:114) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=10] [CHUNK_MGR] limit= 12,884,901,888 hold= 7,735,504,896 total_hold= 8,638,169,088 used= 6,856,671,232 freelists_hold= 878,833,664 total_maps= 388,955 total_unmaps= 385,790 large_maps= 301 large_unmaps= 0 huge_maps= 200 huge_unmaps= 187 memalign=0 resident_size= 8,382,111,744 virtual_memory_used= 9,719,750,656 [CHUNK_MGR] 2 MB_CACHE: hold= 150,994,944 free= 72 pushes= 6,196,507,563 pops= 6,196,507,491 maps= 388,454 unmaps= 385,603 [CHUNK_MGR] 4 MB_CACHE: hold= 154,865,664 free= 73 pushes= 6,821,101 pops= 6,821,028 maps= 172 unmaps= 0 [CHUNK_MGR] 6 MB_CACHE: hold= 193,880,064 free= 46 pushes= 6,920 pops= 6,874 maps= 59 unmaps= 0 [CHUNK_MGR] 8 MB_CACHE: hold= 25,247,744 free= 4 pushes= 16,248,783 pops= 16,248,779 maps= 11 unmaps= 0 [CHUNK_MGR] 10 MB_CACHE: hold= 109,318,144 free= 13 pushes= 123 pops= 110 maps= 30 unmaps= 0 [CHUNK_MGR] 12 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [CHUNK_MGR] 14 MB_CACHE: hold= 26,435,584 free= 2 pushes= 4 pops= 2 maps= 9 unmaps= 0 [CHUNK_MGR] 16 MB_CACHE: hold= 33,316,864 free= 2 pushes= 4 pops= 2 maps= 2 unmaps= 0 [CHUNK_MGR] 18 MB_CACHE: hold= 184,774,656 free= 11 pushes= 993 pops= 982 maps= 18 unmaps= 0 [CHUNK_MGR] 20 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [2024-10-16 14:57:21.195839] INFO print (ob_malloc_time_monitor.cpp:39) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7] [MALLOC_TIME_MONITOR] show the distribution of ob_malloc's cost_time [MALLOC_TIME_MONITOR] [ 0, 10): delta_total_cost_time= 573779813, delta_count= 2749363631, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 10, 100): delta_total_cost_time= 26279119, delta_count= 1677351, avg_cost_time= 15 [MALLOC_TIME_MONITOR] [ 100, 1000): delta_total_cost_time= 389582, delta_count= 2058, avg_cost_time= 189 [MALLOC_TIME_MONITOR] [ 1000, 10000): delta_total_cost_time= 274276, delta_count= 97, avg_cost_time= 2827 [MALLOC_TIME_MONITOR] [ 10000, 100000): delta_total_cost_time= 291366, delta_count= 15, avg_cost_time= 19424 [MALLOC_TIME_MONITOR] [ 100000, 1000000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 1000000, 9223372036854775807): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [2024-10-16 14:57:21.205654] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392222935}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.205700] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41] all savepoint rollback succeed(tx.tx_id={txid:392222935}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.205709] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222935}, start_ts=1729061841200180, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222935}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061841198385, active_ts:1729061841199445, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441199445, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007323741159, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841198384, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.213319] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=56955) [2024-10-16 14:57:21.213610] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21657][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3DDB-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510309081994, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410512, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:21.213895] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=56955) [2024-10-16 14:57:21.216403] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eddb38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202769} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2189}, prev=this=0x2b0f22e89220 trans_version={val:1729061779474766000, v:0} scn={val:1729061779474766000, v:0} tx_id={txid:392201738} prev=0x2b0f22e890e0 next=(nil) modify_count=1 acc_checksum=4054715666 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5333}, snapshot_version={val:1729061841215352000, v:0}, *this={this=0x2b0f22e88dd8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e89220 latest_compact_node=(nil) max_trans_version={val:1729061779474766000, v:0} max_trans_id=392201738 max_elr_trans_version={val:1729061779474766000, v:0} max_elr_trans_id=392201738 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779474766000, v:0} min_modify_scn={val:1729061779474766000, v:0}}) [2024-10-16 14:57:21.216441] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eddb38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202769} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2189}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606256, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606256, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841215352000, v:0}, tx_id:{txid:392202769}, scn:{branch:0, seq:2186}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202769}, tx_desc:{this:0x2b0ec37b5d90, tx_id:{txid:392202769}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841216371, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841213167, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2189}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.216498] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202769} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202769}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:2189}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.216516] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513897162756096},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202769} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.216530] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.216539] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513897162756096},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.216585] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.216825] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3205}, to_seq_no={branch:0, seq:2187}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202769} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.218682] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C3A-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:21.220513] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26] handle_sp_rollback_resp(tx_id={txid:392222936}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.220585] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392222936}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.220595] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222936}, start_ts=1729061841220031, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222936}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6392}, op_sn:3, alloc_ts:1729061841213164, active_ts:1729061841219555, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441219555, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007339172323, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841213163, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.221146] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=6] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:57:21.221159] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=10] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=82, cache_node_num=82) [2024-10-16 14:57:21.221164] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=82, cache_node_num=82) [2024-10-16 14:57:21.221169] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:21.222823] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=611, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:21.222837] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=709, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:21.222853] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:21.223494] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=82, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}]) [2024-10-16 14:57:21.223611] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=115] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:21.225092] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=22, replace_time=0, replace_start_pos=1100995, replace_num=31457) [2024-10-16 14:57:21.225103] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=22, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:21.225351] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFAC-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:21.225370] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFAC-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2152, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:21.225374] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFAC-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=1146, avg_log_submit_to_slide_time=2133) [2024-10-16 14:57:21.225412] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFAD-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:21.225416] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFAD-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2121, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:21.229086] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=10] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:57:21.229928] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061841228687000, start_id=1729061841228687000, end_id=1729061841228687001) [2024-10-16 14:57:21.231360] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E99801B4F11-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061841228687000, start_id=1729061841228687000, end_id=1729061841228687001) [2024-10-16 14:57:21.232639] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] worker thread began to shrink(tenant_->id()=1002, group_id_=1, token=0) [2024-10-16 14:57:21.232847] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106946][T1002_L0_G1][T1002][Y0-0000000000000000-0-0] [lt=4] worker thread exit(tenant_->id()=1002, workers_.get_size()=2) [2024-10-16 14:57:21.232858] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106946][T1002_L0_G1][T0][Y0-0000000000000000-0-0] [lt=8] tenant thread end_run(id_=1002, ret=0, thread_count_=190) [2024-10-16 14:57:21.233717] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3EE7-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=39, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:21.233734] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3EE7-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=39, stat_interval=1000000, avg (count/cost)=1555, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:21.235185] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get_max_applied_scn(scn={val:1729061841038439000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959667671}, palf_committed_end_scn_:{val:1729061841148288000, v:0}, last_check_scn_:{val:1729061841148288000, v:0}, max_applied_cb_scn_:{val:1729061841038439000, v:0}}) [2024-10-16 14:57:21.235207] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:21.235260] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=41] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:57:21.235268] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] get_min_unreplayed_log_info(lsn={lsn:213057794896}, scn={val:1729061841212106001, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061841214373, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213057794896}, next_to_submit_scn_:{val:1729061841212106001, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:1624, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1624, log_storage_:{IteratorStorage:{start_lsn:{lsn:213057793272}, end_lsn:{lsn:213057794896}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061841212106000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:1536, scn_:{val:1729061841212106000, v:0}, data_checksum:238351160, flag:1}}, init_mode_version:0, accumulate_checksum:3016219112, curr_entry_is_padding:0, padding_entry_size:1568, padding_entry_scn:{val:1729061841212106000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560863, miss_cnt:1, cache_read_size:152590234878, hit_ratio:"9.999999928346674505e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:21.241875] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AF-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:21.241898] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9AF-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061840626063004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061840626063004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:21.242050] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f463ab8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202770} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:12458}, prev=this=0x2b0e5abe95f0 trans_version={val:1729061779478588002, v:0} scn={val:1729061779478588002, v:0} tx_id={txid:392201739} prev=0x2b0e5abe94b0 next=(nil) modify_count=1 acc_checksum=3872389668 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3819}, snapshot_version={val:1729061841234698000, v:0}, *this={this=0x2b0e5abe91a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abe95f0 latest_compact_node=(nil) max_trans_version={val:1729061779478588002, v:0} max_trans_id=392201739 max_elr_trans_version={val:1729061779478588002, v:0} max_elr_trans_id=392201739 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779478588002, v:0} min_modify_scn={val:1729061779478588002, v:0}}) [2024-10-16 14:57:21.242087] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=35][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f463ab8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202770} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:12458}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437611707, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437611707, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841234698000, v:0}, tx_id:{txid:392202770}, scn:{branch:0, seq:6006}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202770}, tx_desc:{this:0x2b0ec37c7a30, tx_id:{txid:392202770}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841241150, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841228693, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:12458}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.242141] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202770} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202770}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:12458}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.242175] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=0] primary key duplication conflict(key={"BIGINT":654830046308585472},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202770} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.242192] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.242198] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046308585472},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.242262] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.245752] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22267][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:15832}, to_seq_no={branch:0, seq:6007}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202770} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.247548] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9AF-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061841245665000, start_id=1729061841245665000, end_id=1729061841245665001) [2024-10-16 14:57:21.251006] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD102C-0-0] [lt=9] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:21.258051] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9AF-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061841257108000, start_id=1729061841257108000, end_id=1729061841257108001) [2024-10-16 14:57:21.258921] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:21.261458] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1729061841260394000, start_id=1729061841260394001, end_id=1729061841260394002) [2024-10-16 14:57:21.261609] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.264993] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] handle_sp_rollback_resp(tx_id={txid:392222937}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.265054] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392222937}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.265073] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222937}, start_ts=1729061841262387, retries=0, savepoint={branch:0, seq:4}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222937}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1183}, op_sn:3, alloc_ts:1729061841260394, active_ts:1729061841261577, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441261577, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007386575986, first_scn:{branch:0, seq:1182}, last_scn:{branch:0, seq:1182}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841260397, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.268712] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E99801B4F11-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061841267888000, start_id=1729061841267888000, end_id=1729061841267888001) [2024-10-16 14:57:21.270893] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.279174] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061841278584000, start_id=1729061841278584000, end_id=1729061841278584001) [2024-10-16 14:57:21.280835] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ede218 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202772} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4204}, prev=this=0x2b0f22e89920 trans_version={val:1729061779484939000, v:0} scn={val:1729061779484939000, v:0} tx_id={txid:392201741} prev=0x2b0f22e897e0 next=(nil) modify_count=1 acc_checksum=1954429165 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2120}, snapshot_version={val:1729061841278584000, v:0}, *this={this=0x2b0f22e894d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e89920 latest_compact_node=(nil) max_trans_version={val:1729061779484939000, v:0} max_trans_id=392201741 max_elr_trans_version={val:1729061779484939000, v:0} max_elr_trans_id=392201741 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779484939000, v:0} min_modify_scn={val:1729061779484939000, v:0}}) [2024-10-16 14:57:21.280872] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ede218 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202772} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4204}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606251, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606251, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841278584000, v:0}, tx_id:{txid:392202772}, scn:{branch:0, seq:2115}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202772}, tx_desc:{this:0x2b0ec37b9cd0, tx_id:{txid:392202772}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841280673, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841276470, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4204}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.280931] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202772} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202772}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4204}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.280949] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830046333751296},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202772} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.280964] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.280969] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046333751296},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.281027] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.282411] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5265}, to_seq_no={branch:0, seq:2116}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202772} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.286224] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] [WRS] update tenant weak read server version(tenant_id=1001, server_version={version:{val:1729061841184549000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061841286188}, version_delta=101674) [2024-10-16 14:57:21.286463] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=8] LogLoopThread round_cost_time(us)(round_cost_time=9) [2024-10-16 14:57:21.288170] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:45338084276}, base_lsn={lsn:45161508864}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:21.288226] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=55] get_replay_process success(id={id:1}, replayed_log_size=176575412, unreplayed_log_size=0) [2024-10-16 14:57:21.288244] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1003, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=168, last_replayed_log_size(MB)=168, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:21.288384] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061841287606000, start_id=1729061841287606000, end_id=1729061841287606001) [2024-10-16 14:57:21.288628] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392222938}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.288695] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392222938}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.288711] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222938}, start_ts=1729061841286754, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222938}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3283}, op_sn:3, alloc_ts:1729061841283258, active_ts:1729061841286540, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441286540, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007409037967, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841283257, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.290818] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061841289805000, start_id=1729061841289805000, end_id=1729061841289805001) [2024-10-16 14:57:21.290888] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=3786, avg_group_log_size=315) [2024-10-16 14:57:21.291324] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:21.291432] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F4E-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=21, avg_log_gen_to_submit_time=49, avg_log_submit_to_flush_time=358, avg_log_submit_to_slide_time=952) [2024-10-16 14:57:21.292637] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f52c8d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202773} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3375}, prev=this=0x2b0f7f496640 trans_version={val:1729061779488126000, v:0} scn={val:1729061779488126000, v:0} tx_id={txid:392201742} prev=0x2b0f7f496500 next=(nil) modify_count=1 acc_checksum=2419988663 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2126}, snapshot_version={val:1729061841289805000, v:0}, *this={this=0x2b0f7f4961f8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f496640 latest_compact_node=(nil) max_trans_version={val:1729061779488126000, v:0} max_trans_id=392201742 max_elr_trans_version={val:1729061779488126000, v:0} max_elr_trans_id=392201742 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779488126000, v:0} min_modify_scn={val:1729061779488126000, v:0}}) [2024-10-16 14:57:21.292668] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f52c8d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202773} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3375}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437607058, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437607058, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841289805000, v:0}, tx_id:{txid:392202773}, scn:{branch:0, seq:2351}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202773}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392202773}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841292062, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841288688, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:3375}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.292740] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=65][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202773} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202773}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:3375}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.292755] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":675513897213087744},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202773} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.292771] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.292776] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513897213087744},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.292860] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=59][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.293149] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3379}, to_seq_no={branch:0, seq:2352}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202773} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.299684] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=47][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6deeb28 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222939} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d717a0 trans_version={val:1729061781062093000, v:0} scn={val:1729061781062093000, v:0} tx_id={txid:392222318} prev=0x2b0ec6d71660 next=(nil) modify_count=1 acc_checksum=3639088228 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:16576}, snapshot_version={val:1729061841299152000, v:0}, *this={this=0x2b0ec6d71358 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d717a0 latest_compact_node=(nil) max_trans_version={val:1729061781062093000, v:0} max_trans_id=392222318 max_elr_trans_version={val:1729061781062093000, v:0} max_elr_trans_id=392222318 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781062093000, v:0} min_modify_scn={val:1729061781062093000, v:0}}) [2024-10-16 14:57:21.299721] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6deeb28 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222939} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841299152000, v:0}, tx_id:{txid:392222939}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222939}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222939}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841299152, active_ts:1729061841299152, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841299151, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.299784] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=63][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222939} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222939}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.299802] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654910318794362880},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222939} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.299817] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.299826] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318794362880},{"BIGINT":1729058400},{"VARCHAR":"@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.299884] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.299932] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222939} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.301359] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f464198 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222940} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0e5abff950 trans_version={val:1729061781084057000, v:0} scn={val:1729061781084057000, v:0} tx_id={txid:392222319} prev=0x2b0e5abff810 next=(nil) modify_count=1 acc_checksum=273740126 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5843}, snapshot_version={val:1729061841300261001, v:0}, *this={this=0x2b0e5abff1c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abff950 latest_compact_node=(nil) max_trans_version={val:1729061781084057000, v:0} max_trans_id=392222319 max_elr_trans_version={val:1729061781084057000, v:0} max_elr_trans_id=392222319 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781084057000, v:0} min_modify_scn={val:1729061781084057000, v:0}}) [2024-10-16 14:57:21.301391] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f464198 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222940} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841300261001, v:0}, tx_id:{txid:392222940}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222940}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222940}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841300261, active_ts:1729061841301325, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841300262, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.301442] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=50][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222940} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222940}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.301458] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":675513903114473472},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222940} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.301471] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.301477] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513903114473472},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.301518] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.301543] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222940} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.305002] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392222941}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.305040] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392222941}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.305049] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222941}, start_ts=1729061841303578, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222941}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2120}, op_sn:3, alloc_ts:1729061841301325, active_ts:1729061841303447, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441303447, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007427293061, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841301327, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.309461] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5495, mem_used=10398720) [2024-10-16 14:57:21.311060] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=7] ====== traversal_flush timer task ====== [2024-10-16 14:57:21.311154] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA6-0-0] [lt=12] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061849029444000, latest_log_ts:{val:1729061832415320001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:21.311212] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA6-0-0] [lt=54] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:21.316589] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C3C-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:21.317523] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb1eae0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202777} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3181}, prev=this=0x2b0fadaa3240 trans_version={val:1729061779494495001, v:0} scn={val:1729061779494495001, v:0} tx_id={txid:392201744} prev=0x2b0fadaa3100 next=(nil) modify_count=1 acc_checksum=42982014 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3157}, snapshot_version={val:1729061841315066000, v:0}, *this={this=0x2b0fadaa2df8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaa3240 latest_compact_node=(nil) max_trans_version={val:1729061779494495001, v:0} max_trans_id=392201744 max_elr_trans_version={val:1729061779494495001, v:0} max_elr_trans_id=392201744 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779494495001, v:0} min_modify_scn={val:1729061779494495001, v:0}}) [2024-10-16 14:57:21.317555] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb1eae0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202777} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3181}, ctx={this:0x2b0e2b750080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606409, tablet_id:{id:209279}, table_iter:0x2b0e2b74f210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606409, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841315066000, v:0}, tx_id:{txid:392202777}, scn:{branch:0, seq:2118}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202777}, tx_desc:{this:0x2b0ede4e1af0, tx_id:{txid:392202777}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841316127, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841312949, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c246c50, mem_ctx:0x2b0e0c2481d0, tx_scn:{branch:0, seq:3181}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.317617] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202777} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202777}, data:0x2b0e2b74c438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:3181}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.317631] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513897238253568},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202777} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.317647] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.317653] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513897238253568},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.317699] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.320738] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=10] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:21.322305] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:21.322332] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:21.323918] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:21.331837] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=7] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:57:21.331850] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:57:21.331854] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:21.332481] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061849029444000, latest_log_ts:{val:1729061832415320001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:21.332660] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:18433}, to_seq_no={branch:0, seq:2119}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202777} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.332766] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=77] handle_sp_rollback_resp(tx_id={txid:392222942}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.332874] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392222942}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.332885] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222942}, start_ts=1729061841331036, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222942}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2125}, op_sn:3, alloc_ts:1729061841329256, active_ts:1729061841330317, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441330317, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007455363539, first_scn:{branch:0, seq:1066}, last_scn:{branch:0, seq:1066}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841328192, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.336131] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22235][T1002_TenantWea][T1001][YB4A0A000407-00061E9B3B0B75D1-0-0] [lt=22] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.162681007385253906e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=308, Throttle Trigger(MB)=184, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.933448127025233816e+03) [2024-10-16 14:57:21.336314] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061841335625000, start_id=1729061841335625000, end_id=1729061841335625001) [2024-10-16 14:57:21.337248] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061841335625000, start_id=1729061841335625000, end_id=1729061841335625001) [2024-10-16 14:57:21.338635] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=24] get_max_applied_scn(scn={val:1729061841228687000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204022703012}, palf_committed_end_scn_:{val:1729061841301325000, v:0}, last_check_scn_:{val:1729061841301325000, v:0}, max_applied_cb_scn_:{val:1729061841228687000, v:0}}) [2024-10-16 14:57:21.338663] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=23] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:21.338956] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061841337442000, start_id=1729061841337442000, end_id=1729061841337442001) [2024-10-16 14:57:21.341278] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E01-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061841340231000, start_id=1729061841340231000, end_id=1729061841340231001) [2024-10-16 14:57:21.343011] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb1f1c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222943} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac2ff0 trans_version={val:1729061781118635000, v:0} scn={val:1729061781118635000, v:0} tx_id={txid:392222322} prev=0x2b0fadac2eb0 next=(nil) modify_count=1 acc_checksum=318481296 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2463}, snapshot_version={val:1729061841341935001, v:0}, *this={this=0x2b0fadac2ba8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac2ff0 latest_compact_node=(nil) max_trans_version={val:1729061781118635000, v:0} max_trans_id=392222322 max_elr_trans_version={val:1729061781118635000, v:0} max_elr_trans_id=392222322 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781118635000, v:0} min_modify_scn={val:1729061781118635000, v:0}}) [2024-10-16 14:57:21.343047] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb1f1c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222943} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841341935001, v:0}, tx_id:{txid:392222943}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222943}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222943}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841341935, active_ts:1729061841341935, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841341936, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.343104] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222943} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222943}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.343123] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654910318844694528},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222943} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.343138] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.343144] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318844694528},{"BIGINT":1729058400},{"VARCHAR":"A“(°T", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.343196] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.343226] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222943} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.344002] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] rpc statistics(total_trans_msg_count=87, total_batch_msg_count=0) [2024-10-16 14:57:21.345676] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] handle_sp_rollback_resp(tx_id={txid:392222944}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.345739] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA3-0-0] [lt=15] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:21.345755] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA3-0-0] [lt=11] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=17) [2024-10-16 14:57:21.345814] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392222944}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.345828] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222944}, start_ts=1729061841345440, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222944}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1035}, op_sn:3, alloc_ts:1729061841344059, active_ts:1729061841345093, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441345093, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007469749165, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841344058, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.353040] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392222945}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.353101] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392222945}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.353118] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222945}, start_ts=1729061841352780, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222945}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1027}, op_sn:3, alloc_ts:1729061841351425, active_ts:1729061841352451, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441352451, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332337028260, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841351424, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.354171] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E99801B4F11-0-0] [lt=45] get number(ret=0, service_type_=0, range=1, base_id=1729061841353510000, start_id=1729061841353510000, end_id=1729061841353510001) [2024-10-16 14:57:21.354177] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=30, avg_log_batch_cnt=0, total_group_log_size=47213, avg_group_log_size=1573) [2024-10-16 14:57:21.355759] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6def510 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222946} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d73128 trans_version={val:1729061781137162002, v:0} scn={val:1729061781137162002, v:0} tx_id={txid:392222325} prev=0x2b0ec6d72fe8 next=(nil) modify_count=1 acc_checksum=66659785 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841355667000, v:0}, *this={this=0x2b0ec6d72ce0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d73128 latest_compact_node=(nil) max_trans_version={val:1729061781137162002, v:0} max_trans_id=392222325 max_elr_trans_version={val:1729061781137162002, v:0} max_elr_trans_id=392222325 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781137162002, v:0} min_modify_scn={val:1729061781137162002, v:0}}) [2024-10-16 14:57:21.355793] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6def510 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222946} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841355667000, v:0}, tx_id:{txid:392222946}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222946}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222946}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841354607, active_ts:1729061841355667, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841354612, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.355850] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222946} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222946}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.355867] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513903290634240},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222946} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.355885] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.355891] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513903290634240},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.355968] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.356002] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1056}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222946} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.358580] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25] handle_sp_rollback_resp(tx_id={txid:392222947}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.358617] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392222947}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.358624] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222947}, start_ts=1729061841358297, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222947}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1057}, op_sn:3, alloc_ts:1729061841356724, active_ts:1729061841357780, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441357780, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007482247830, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841356723, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.359876] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F87-0-0] [lt=22] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:21.359890] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F87-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9762, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:21.359888] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:21.359894] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F87-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=1126, avg_log_submit_to_slide_time=9756) [2024-10-16 14:57:21.359919] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:21.359926] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:21.361250] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f464878 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202782} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5306}, prev=this=0x2b0e5abe9cf0 trans_version={val:1729061779500848000, v:0} scn={val:1729061779500848000, v:0} tx_id={txid:392201746} prev=0x2b0e5abe9bb0 next=(nil) modify_count=1 acc_checksum=2964450260 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1067}, snapshot_version={val:1729061841358839000, v:0}, *this={this=0x2b0e5abe98a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abe9cf0 latest_compact_node=(nil) max_trans_version={val:1729061779500848000, v:0} max_trans_id=392201746 max_elr_trans_version={val:1729061779500848000, v:0} max_elr_trans_id=392201746 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779500848000, v:0} min_modify_scn={val:1729061779500848000, v:0}}) [2024-10-16 14:57:21.361292] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=41][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f464878 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202782} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5306}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606282, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606282, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841358839000, v:0}, tx_id:{txid:392202782}, scn:{branch:0, seq:3171}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202782}, tx_desc:{this:0x2b0ec37c99d0, tx_id:{txid:392202782}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841360975, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841355670, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:5306}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.361351] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202782} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202782}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:5306}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.361373] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=0] primary key duplication conflict(key={"BIGINT":675513897263419392},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202782} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.361388] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.361397] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513897263419392},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.361448] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.361670] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.365059] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8460}, to_seq_no={branch:0, seq:3172}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202782} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.368061] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061841367399000, start_id=1729061841367399000, end_id=1729061841367399001) [2024-10-16 14:57:21.368991] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30] handle_sp_rollback_resp(tx_id={txid:392222948}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.369019] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392222948}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.369027] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222948}, start_ts=1729061841368688, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222948}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061841367399, active_ts:1729061841368459, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441368459, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332353343240, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841367398, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.371085] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.374010] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392222949}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.374039] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392222949}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.374048] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222949}, start_ts=1729061841373359, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222949}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1171}, op_sn:3, alloc_ts:1729061841371613, active_ts:1729061841372787, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441372787, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332357945095, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841371616, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.378147] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392222950}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.378198] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392222950}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.378210] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222950}, start_ts=1729061841377808, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222950}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2684}, op_sn:3, alloc_ts:1729061841376534, active_ts:1729061841377565, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441377565, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007501790352, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841374881, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.382124] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061841381800000, start_id=1729061841381800000, end_id=1729061841381800001) [2024-10-16 14:57:21.383452] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E99801B4F11-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061841382888000, start_id=1729061841382888000, end_id=1729061841382888001) [2024-10-16 14:57:21.383674] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:21.384811] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ede8f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222951} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f22ea38d8 trans_version={val:1729061781188656001, v:0} scn={val:1729061781188656001, v:0} tx_id={txid:392222330} prev=0x2b0f22ea3798 next=(nil) modify_count=1 acc_checksum=2303794222 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841383917000, v:0}, *this={this=0x2b0f22ea3490 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea38d8 latest_compact_node=(nil) max_trans_version={val:1729061781188656001, v:0} max_trans_id=392222330 max_elr_trans_version={val:1729061781188656001, v:0} max_elr_trans_id=392222330 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781188656001, v:0} min_modify_scn={val:1729061781188656001, v:0}}) [2024-10-16 14:57:21.384845] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ede8f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222951} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209267}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841383917000, v:0}, tx_id:{txid:392222951}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222951}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222951}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841383917, active_ts:1729061841383917, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841383916, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.384906] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222951} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222951}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.384924] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654910318957940736},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222951} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.384939] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.384944] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318957940736},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.385002] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.385041] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222951} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.386054] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f464f58 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222952} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f404808 trans_version={val:1729061781190392002, v:0} scn={val:1729061781190392002, v:0} tx_id={txid:392222331} prev=0x2b0f3f4046c8 next=(nil) modify_count=1 acc_checksum=4032189954 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841384981001, v:0}, *this={this=0x2b0f3f4043c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f404808 latest_compact_node=(nil) max_trans_version={val:1729061781190392002, v:0} max_trans_id=392222331 max_elr_trans_version={val:1729061781190392002, v:0} max_elr_trans_id=392222331 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781190392002, v:0} min_modify_scn={val:1729061781190392002, v:0}}) [2024-10-16 14:57:21.386083] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f464f58 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222952} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841384981001, v:0}, tx_id:{txid:392222952}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222952}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222952}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841384981, active_ts:1729061841384981, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841384980, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.386132] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=48][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222952} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222952}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.386145] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513903366131712},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222952} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.386166] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.386171] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513903366131712},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.386211] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.386231] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1071}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222952} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.387121] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061841387110000, start_id=1729061841387110000, end_id=1729061841387110001) [2024-10-16 14:57:21.390677] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392222953}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.390711] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392222953}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.390724] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222953}, start_ts=1729061841390016, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222953}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2098}, op_sn:3, alloc_ts:1729061841387110, active_ts:1729061841389207, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441389207, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007513934395, first_scn:{branch:0, seq:1059}, last_scn:{branch:0, seq:1059}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841387109, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.394714] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3C-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:21.394738] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3C-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061841381800000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061841381800000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:21.395859] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=9] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:21.395982] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C3C-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061841395472000, start_id=1729061841395472000, end_id=1729061841395472001) [2024-10-16 14:57:21.398544] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=45] new mysql sessid created(conn.sessid_=3221737514, support_ssl=false) [2024-10-16 14:57:21.398581] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=12] sm conn init succ(conn.sessid_=3221737514, sess.client_addr_="127.0.0.1:33470") [2024-10-16 14:57:21.398589] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0e357fcdf0, session_id:3221737514, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.398611] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=18] new mysql sessid created(conn.sessid_=3221735693, support_ssl=false) [2024-10-16 14:57:21.398624] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221735693, sess.client_addr_="127.0.0.1:33474") [2024-10-16 14:57:21.398628] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdcc030, session_id:3221735693, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:214}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.398637] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221732014, support_ssl=false) [2024-10-16 14:57:21.398648] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221732014, sess.client_addr_="127.0.0.1:33488") [2024-10-16 14:57:21.398652] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdb8df0, session_id:3221732014, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:215}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.398674] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=9] new mysql sessid created(conn.sessid_=3221732588, support_ssl=false) [2024-10-16 14:57:21.398691] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221732588, sess.client_addr_="127.0.0.1:33472") [2024-10-16 14:57:21.398696] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f27414df0, session_id:3221732588, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:218}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.398706] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221737211, support_ssl=false) [2024-10-16 14:57:21.398716] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221737211, sess.client_addr_="127.0.0.1:33486") [2024-10-16 14:57:21.398720] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f2742c030, session_id:3221737211, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:221}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.398728] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221696746, support_ssl=false) [2024-10-16 14:57:21.398739] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221696746, sess.client_addr_="127.0.0.1:33492") [2024-10-16 14:57:21.398749] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=9] accept one succ(*s={this:0x2b0f2742cdf0, session_id:3221696746, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:223}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.398767] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.398779] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=4] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.398782] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:295) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=4] mysql login queue(mysql_queue->queue_.size()=1) [2024-10-16 14:57:21.398791] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.398797] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.398803] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.398809] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.398855] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=34] new mysql sessid created(conn.sessid_=3221735968, support_ssl=false) [2024-10-16 14:57:21.398880] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=11] sm conn init succ(conn.sessid_=3221735968, sess.client_addr_="127.0.0.1:33476") [2024-10-16 14:57:21.398880] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE11-0-0] [lt=13] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12276, add_timestamp:1729061841398872}) [2024-10-16 14:57:21.398887] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0e5cdfcdf0, session_id:3221735968, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:225}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.398901] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=11] new mysql sessid created(conn.sessid_=3221735943, support_ssl=false) [2024-10-16 14:57:21.398912] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221735943, sess.client_addr_="127.0.0.1:33478") [2024-10-16 14:57:21.398916] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdf8df0, session_id:3221735943, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:227}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.398924] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221736155, support_ssl=false) [2024-10-16 14:57:21.398935] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221736155, sess.client_addr_="127.0.0.1:33480") [2024-10-16 14:57:21.398939] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdbadf0, session_id:3221736155, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:229}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.399029] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB3-0-0] [lt=14] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.399153] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=10] new mysql sessid created(conn.sessid_=3221737462, support_ssl=false) [2024-10-16 14:57:21.399178] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] sm conn init succ(conn.sessid_=3221737462, sess.client_addr_="127.0.0.1:33482") [2024-10-16 14:57:21.399183] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] accept one succ(*s={this:0x2b0f27404030, session_id:3221737462, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:230}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.399193] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] new mysql sessid created(conn.sessid_=3221734381, support_ssl=false) [2024-10-16 14:57:21.399204] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221734381, sess.client_addr_="127.0.0.1:33484") [2024-10-16 14:57:21.399207] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f27404df0, session_id:3221734381, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:231}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.399216] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221737418, support_ssl=false) [2024-10-16 14:57:21.399227] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221737418, sess.client_addr_="127.0.0.1:33490") [2024-10-16 14:57:21.399230] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f27406030, session_id:3221737418, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:232}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.399239] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221704578, support_ssl=false) [2024-10-16 14:57:21.399249] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221704578, sess.client_addr_="127.0.0.1:33494") [2024-10-16 14:57:21.399253] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f27406df0, session_id:3221704578, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:234}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.399350] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A4-0-0] [lt=12] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.399458] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB3-0-0] [lt=5] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221732588, proxy_sessid=0, client_sessid=3221732588, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.399527] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB4-0-0] [lt=27] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.399777] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A4-0-0] [lt=5] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221696746, proxy_sessid=0, client_sessid=3221696746, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.399824] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E07-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12276, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:21.399952] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB4-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737211, proxy_sessid=0, client_sessid=3221737211, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.400006] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.400020] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=7] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.400027] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.400032] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.400038] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.400043] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.400085] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C95-0-0] [lt=20] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.400470] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C95-0-0] [lt=3] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221732014, proxy_sessid=0, client_sessid=3221732014, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.400485] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.400501] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=9] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.400514] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C96-0-0] [lt=12] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.400510] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.400542] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=11] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.400550] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.400554] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.400561] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.400565] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.400862] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C96-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221735693, proxy_sessid=0, client_sessid=3221735693, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.400878] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E07-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12276, locations=[{table_id:12276, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841400864, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841400864, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841400864, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.400900] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C97-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.400898] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E07-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:12276, add_timestamp:1729061841398872}, locations=[{table_id:12276, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841400864, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841400864, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841400864, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.401236] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C97-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737514, proxy_sessid=0, client_sessid=3221737514, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.401271] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C98-0-0] [lt=9] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.401636] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C98-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221704578, proxy_sessid=0, client_sessid=3221704578, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.401677] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C99-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.402011] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C99-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737418, proxy_sessid=0, client_sessid=3221737418, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.402044] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9A-0-0] [lt=9] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.402397] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C3C-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061841399942000, start_id=1729061841399942000, end_id=1729061841399942001) [2024-10-16 14:57:21.402405] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9A-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221734381, proxy_sessid=0, client_sessid=3221734381, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.402451] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9B-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.402803] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9B-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737462, proxy_sessid=0, client_sessid=3221737462, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.402927] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C95-0-0] [lt=17] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1729061841402920}) [2024-10-16 14:57:21.402978] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=14] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12358, add_timestamp:1729061841402975}) [2024-10-16 14:57:21.402993] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=8] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12396, add_timestamp:1729061841402992}) [2024-10-16 14:57:21.403829] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.403838] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.404511] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.404543] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.404550] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.404555] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.404559] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.404587] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:21.404590] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.404606] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.404609] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.404617] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:21.404620] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.404624] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.404627] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED80-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.405270] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061841405268}) [2024-10-16 14:57:21.405280] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=10][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061841405268}) [2024-10-16 14:57:21.405285] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=5][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12358, vtable_locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061826427676, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.405300] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=11][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061841405299}) [2024-10-16 14:57:21.405303] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061841405299}) [2024-10-16 14:57:21.405306] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12396, vtable_locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061826427694, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.405474] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=6] get_stable_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:21.405492] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21671][T1_L0_G0][T1001][YB4A0A000407-00061E99786CED81-0-0] [lt=5] get_stable_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:21.405502] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21671][T1_L0_G0][T1002][YB4A0A000407-00061E99786CED81-0-0] [lt=3] get_stable_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4384) [2024-10-16 14:57:21.405509] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21671][T1_L0_G0][T1003][YB4A0A000407-00061E99786CED81-0-0] [lt=3] get_stable_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:21.405525] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21671][T1_L0_G0][T1004][YB4A0A000407-00061E99786CED81-0-0] [lt=3] get_stable_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:21.406260] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.406269] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.406897] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.406916] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.406923] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.406927] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.406931] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.406965] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:21.406968] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.406977] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.406980] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.406989] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:21.406992] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.406996] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.406999] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21671][T1_L0_G0][T1][YB4A0A000407-00061E99786CED81-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.407712] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.407730] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=10] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.407740] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=4] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.407744] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.407750] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:21.407754] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:21.407905] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A5-0-0] [lt=14] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.408292] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A5-0-0] [lt=5] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221736155, proxy_sessid=0, client_sessid=3221736155, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.408338] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A6-0-0] [lt=13] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.408686] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A6-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221735943, proxy_sessid=0, client_sessid=3221735943, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.408724] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A7-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:21.409047] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A7-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221735968, proxy_sessid=0, client_sessid=3221735968, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:21.410013] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C95-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:21.410707] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E08-0-0] [lt=26] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:21.410968] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC91D-0-0] [lt=12] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12142, add_timestamp:1729061841410964}) [2024-10-16 14:57:21.410985] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC91D-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12217, add_timestamp:1729061841410984}) [2024-10-16 14:57:21.410991] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC91D-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12068, add_timestamp:1729061841410991}) [2024-10-16 14:57:21.410997] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC91D-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11035, add_timestamp:1729061841410996}) [2024-10-16 14:57:21.411002] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC91D-0-0] [lt=3] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12072, add_timestamp:1729061841411001}) [2024-10-16 14:57:21.411215] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=16] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1729061841411212}) [2024-10-16 14:57:21.411237] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=9][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061841411236}) [2024-10-16 14:57:21.411244] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=7][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061841411236}) [2024-10-16 14:57:21.411250] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=5][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12358, vtable_locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061826427676, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.411270] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=15][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061841411270}) [2024-10-16 14:57:21.411275] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=5][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061841411270}) [2024-10-16 14:57:21.411280] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=5][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12396, vtable_locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061826427694, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.412274] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.412286] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=11] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.412984] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.413012] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=13] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.413018] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.413023] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.413027] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.413067] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:21.413070] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.413079] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.413082] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.413090] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:21.413093] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.413097] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.413100] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED788-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.414794] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E08-0-0] [lt=16] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11017, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414772, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414772, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414772, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.414816] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E08-0-0] [lt=21] success to process renew task(task={tenant_id:1, table_id:11017, add_timestamp:1729061841402920}, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414772, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414772, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414772, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.414838] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E09-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12358, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414835, sql_renew_time:1729061841414835, is_mark_fail:false}]) [2024-10-16 14:57:21.414849] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E09-0-0] [lt=10] success to process renew task(task={tenant_id:1, table_id:12358, add_timestamp:1729061841402975}, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414835, sql_renew_time:1729061841414835, is_mark_fail:false}]) [2024-10-16 14:57:21.414858] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0A-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12396, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414856, sql_renew_time:1729061841414856, is_mark_fail:false}]) [2024-10-16 14:57:21.414864] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0A-0-0] [lt=6] success to process renew task(task={tenant_id:1, table_id:12396, add_timestamp:1729061841402992}, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414856, sql_renew_time:1729061841414856, is_mark_fail:false}]) [2024-10-16 14:57:21.414873] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0B-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12142, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414871, sql_renew_time:1729061841414871, is_mark_fail:false}]) [2024-10-16 14:57:21.414878] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0B-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12142, add_timestamp:1729061841410964}, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414871, sql_renew_time:1729061841414871, is_mark_fail:false}]) [2024-10-16 14:57:21.414887] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0C-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12217, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414885, sql_renew_time:1729061841414885, is_mark_fail:false}]) [2024-10-16 14:57:21.414893] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0C-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12217, add_timestamp:1729061841410984}, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414885, sql_renew_time:1729061841414885, is_mark_fail:false}]) [2024-10-16 14:57:21.414901] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0D-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12068, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414899, sql_renew_time:1729061841414899, is_mark_fail:false}]) [2024-10-16 14:57:21.414907] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0D-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12068, add_timestamp:1729061841410991}, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414899, sql_renew_time:1729061841414899, is_mark_fail:false}]) [2024-10-16 14:57:21.414916] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0E-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11035, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414913, sql_renew_time:1729061841414913, is_mark_fail:false}]) [2024-10-16 14:57:21.414922] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0E-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:11035, add_timestamp:1729061841410996}, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414913, sql_renew_time:1729061841414913, is_mark_fail:false}]) [2024-10-16 14:57:21.414930] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0F-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12072, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414928, sql_renew_time:1729061841414928, is_mark_fail:false}]) [2024-10-16 14:57:21.414936] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E0F-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12072, add_timestamp:1729061841411001}, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414928, sql_renew_time:1729061841414928, is_mark_fail:false}]) [2024-10-16 14:57:21.414943] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E10-0-0] [lt=6] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:21.415029] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E10-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10008, locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841415025, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841415025, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841415025, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.415040] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E10-0-0] [lt=11] success to process renew task(task={tenant_id:1, table_id:10008, add_timestamp:1729061841411212}, locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841415025, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841415025, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841415025, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.415227] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6defef8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222954} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d746d8 trans_version={val:1729061781196958000, v:0} scn={val:1729061781196958000, v:0} tx_id={txid:392222333} prev=0x2b0ec6d74598 next=(nil) modify_count=1 acc_checksum=1382599754 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841414438000, v:0}, *this={this=0x2b0ec6d74290 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d746d8 latest_compact_node=(nil) max_trans_version={val:1729061781196958000, v:0} max_trans_id=392222333 max_elr_trans_version={val:1729061781196958000, v:0} max_elr_trans_id=392222333 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781196958000, v:0} min_modify_scn={val:1729061781196958000, v:0}}) [2024-10-16 14:57:21.415253] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6defef8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222954} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841414438000, v:0}, tx_id:{txid:392222954}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222954}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222954}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841414438, active_ts:1729061841414438, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841414440, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.415304] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=50][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222954} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222954}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.415317] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513903416463360},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222954} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.415328] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.415333] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513903416463360},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.415404] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=47][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.415439] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222954} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.417282] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C3E-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:21.418672] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.418683] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=11] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.419324] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.419346] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.419352] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.419356] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.419361] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.419529] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:21.419534] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.419543] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.419546] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.419556] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:21.419560] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.419564] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.419567] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED789-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.419795] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED78A-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11021, add_timestamp:1729061841419788}) [2024-10-16 14:57:21.426041] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF95-0-0] [lt=4][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:21.426873] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85C4-0-0] [lt=2] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11008, add_timestamp:1729061841426869}) [2024-10-16 14:57:21.427703] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:290) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED78A-0-0] [lt=7][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:21.428205] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E11-0-0] [lt=10] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11021, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:21.429817] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e357fc030, session_id:3221728988, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:224}, err:5, last_decode_time:1729061841397548, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:1513, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.429841] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=19] kill and revert session(conn.sessid_=3221728988, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.429848] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e2917e030, session_id:3221724208, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061841397548, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:2152, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.429856] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=7] kill and revert session(conn.sessid_=3221724208, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.429869] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0e357fc030, session_id:3221728988, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:224}, err:5, last_decode_time:1729061841397548, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:1513, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.429882] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221728988, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.429887] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] connection close(sessid=3221728988, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9978BCFE12-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:37, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{receive:obp_mysql_header_{len_:144, rec_:144, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[8]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_mysql_header_{pkt_num_:1, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[10]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[11]:{send:obp_mysql_header_{pkt_num_:1, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[12]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[13]:{receive:obp_mysql_header_{len_:179, rec_:179, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[14]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[15]:{send:obp_mysql_header_{pkt_num_:2, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[16]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[17]:{send:obp_mysql_header_{pkt_num_:1, seq_:6}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[18]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[19]:{receive:obp_mysql_header_{len_:144, rec_:144, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[20]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[21]:{send:obp_mysql_header_{pkt_num_:1, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[22]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[23]:{send:obp_mysql_header_{pkt_num_:1, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[24]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[25]:{receive:obp_mysql_header_{len_:144, rec_:144, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[26]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[27]:{send:obp_mysql_header_{pkt_num_:1, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[28]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[29]:{send:obp_mysql_header_{pkt_num_:1, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[30]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[31]:{receive:obp_mysql_header_{len_:144, rec_:144, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[32]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[33]:{send:obp_mysql_header_{pkt_num_:634, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[34]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[35]:{send:obp_mysql_header_{pkt_num_:7, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[36]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.429942] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=53] sock ref clean, do destroy(*s={this:0x2b0e2917e030, session_id:3221724208, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061841397548, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:2152, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.429950] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221724208, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.429953] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221724208, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9978BCFE11-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:55, pkt_rec[23]:{send:obp_mysql_header_{pkt_num_:1, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[24]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[25]:{receive:obp_mysql_header_{len_:179, rec_:179, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[26]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[27]:{send:obp_mysql_header_{pkt_num_:2, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[28]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[29]:{send:obp_mysql_header_{pkt_num_:1, seq_:6}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[30]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[31]:{receive:obp_mysql_header_{len_:144, rec_:144, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[32]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[33]:{send:obp_mysql_header_{pkt_num_:689, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[34]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[35]:{send:obp_mysql_header_{pkt_num_:4, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[36]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[37]:{receive:obp_mysql_header_{len_:179, rec_:179, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[38]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[39]:{send:obp_mysql_header_{pkt_num_:146, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[40]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[41]:{send:obp_mysql_header_{pkt_num_:2, seq_:6}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[42]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[43]:{receive:obp_mysql_header_{len_:179, rec_:179, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[44]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[45]:{send:obp_mysql_header_{pkt_num_:181, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[46]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[47]:{send:obp_mysql_header_{pkt_num_:3, seq_:6}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[48]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[49]:{receive:obp_mysql_header_{len_:179, rec_:179, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[50]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[51]:{send:obp_mysql_header_{pkt_num_:146, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[52]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[53]:{send:obp_mysql_header_{pkt_num_:2, seq_:6}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[54]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.430094] INFO [SERVER] run (obmp_disconnect.cpp:82) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=8] free session successfully(sessid=3221728988) [2024-10-16 14:57:21.430101] INFO [SERVER] run (obmp_disconnect.cpp:89) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=6] mark session id unused(sessid=3221728988) [2024-10-16 14:57:21.430140] INFO [SERVER] run (obmp_disconnect.cpp:82) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221724208) [2024-10-16 14:57:21.430144] INFO [SERVER] run (obmp_disconnect.cpp:89) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] mark session id unused(sessid=3221724208) [2024-10-16 14:57:21.430167] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=26, replace_time=0, replace_start_pos=1132452, replace_num=31457) [2024-10-16 14:57:21.430180] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=26, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:21.431268] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.431279] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=11] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.431946] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:21.431968] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.431975] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.431980] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.431983] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.432025] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:21.432028] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.432037] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:21.432040] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=2] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.432047] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:21.432050] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.432055] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:21.432057] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21669][T1_L0_G0][T1][YB4A0A000407-00061E9978BCFE13-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:21.432423] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27406030, session_id:3221737418, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:232}, err:5, last_decode_time:1729061841402585, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:738, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.432442] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=15] kill and revert session(conn.sessid_=3221737418, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.432453] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] sock ref clean, do destroy(*s={this:0x2b0f27406030, session_id:3221737418, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:232}, err:5, last_decode_time:1729061841402585, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:738, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.432467] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] unlock session of tenant(conn.sessid_=3221737418, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.432472] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221737418, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E99786CED80-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:633, rec_:633, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.433709] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e357fcdf0, session_id:3221737514, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:231, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.433731] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=19] kill and revert session(conn.sessid_=3221737514, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.433737] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdb8df0, session_id:3221732014, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:215}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:466, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.433744] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221732014, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.433747] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdcc030, session_id:3221735693, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:214}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:239, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.433754] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] kill and revert session(conn.sessid_=3221735693, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.433757] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] sock ref clean, do destroy(*s={this:0x2b0e357fcdf0, session_id:3221737514, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:231, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.433768] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221737514, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.433775] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=7] connection close(sessid=3221737514, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B34FA85C4-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:126, rec_:126, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:64, seq_:70}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:71}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.433797] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=20] sock ref clean, do destroy(*s={this:0x2b0e5cdb8df0, session_id:3221732014, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:215}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:466, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.433807] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221732014, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.433810] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221732014, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED78C-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:361, rec_:361, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:7, seq_:9}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.433822] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0e5cdcc030, session_id:3221735693, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:214}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:239, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.433831] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221735693, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.433834] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221735693, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED78B-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:134, rec_:134, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:7}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:8}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.433893] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] free session successfully(sessid=3221737418) [2024-10-16 14:57:21.433899] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221737418) [2024-10-16 14:57:21.433925] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221737514) [2024-10-16 14:57:21.433928] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221737514) [2024-10-16 14:57:21.433948] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221732014) [2024-10-16 14:57:21.433950] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221732014) [2024-10-16 14:57:21.433972] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221735693) [2024-10-16 14:57:21.433974] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221735693) [2024-10-16 14:57:21.434049] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF851-0-0] [lt=2] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11003, add_timestamp:1729061841434045}) [2024-10-16 14:57:21.434404] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=39][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdbadf0, session_id:3221736155, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:229}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:534, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.434422] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=14] kill and revert session(conn.sessid_=3221736155, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.434428] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdf8df0, session_id:3221735943, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:227}, err:5, last_decode_time:1729061841410535, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.434435] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221735943, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.434438] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0e5cdbadf0, session_id:3221736155, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:229}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:534, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.434450] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221736155, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.434454] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221736155, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9978BCFE13-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:429, rec_:429, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.434474] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=19] sock ref clean, do destroy(*s={this:0x2b0e5cdf8df0, session_id:3221735943, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:227}, err:5, last_decode_time:1729061841410535, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.434485] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221735943, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.434488] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2] connection close(sessid=3221735943, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED788-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:452, rec_:452, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.434537] INFO [SERVER] run (obmp_disconnect.cpp:82) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221736155) [2024-10-16 14:57:21.434541] INFO [SERVER] run (obmp_disconnect.cpp:89) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] mark session id unused(sessid=3221736155) [2024-10-16 14:57:21.434566] INFO [SERVER] run (obmp_disconnect.cpp:82) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221735943) [2024-10-16 14:57:21.434568] INFO [SERVER] run (obmp_disconnect.cpp:89) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221735943) [2024-10-16 14:57:21.435012] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=22][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27404030, session_id:3221737462, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:230}, err:5, last_decode_time:1729061841430283, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:203, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.435029] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=14] kill and revert session(conn.sessid_=3221737462, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.435034] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] sock ref clean, do destroy(*s={this:0x2b0f27404030, session_id:3221737462, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:230}, err:5, last_decode_time:1729061841430283, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:203, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.435046] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221737462, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.435050] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221737462, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AECBAF851-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:98, rec_:98, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.435121] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=6] free session successfully(sessid=3221737462) [2024-10-16 14:57:21.435127] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221737462) [2024-10-16 14:57:21.435657] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E11-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11021, locations=[{table_id:11021, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841435645, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841435645, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841435645, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.435679] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E11-0-0] [lt=22] success to process renew task(task={tenant_id:1, table_id:11021, add_timestamp:1729061841419788}, locations=[{table_id:11021, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841435645, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841435645, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841435645, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.435698] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E12-0-0] [lt=11] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11008, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:21.437208] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27414df0, session_id:3221732588, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:218}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:793, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.437231] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=17] kill and revert session(conn.sessid_=3221732588, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.437236] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2742c030, session_id:3221737211, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:221}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:654, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.437245] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221737211, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.437248] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2742cdf0, session_id:3221696746, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:223}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.437259] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=10] kill and revert session(conn.sessid_=3221696746, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.437262] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0f27414df0, session_id:3221732588, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:218}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:793, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.437275] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=9] unlock session of tenant(conn.sessid_=3221732588, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.437280] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221732588, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED78A-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:688, rec_:688, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:370, seq_:120}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:121}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.437301] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=19] sock ref clean, do destroy(*s={this:0x2b0f2742c030, session_id:3221737211, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:221}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:654, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.437309] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221737211, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.437312] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] connection close(sessid=3221737211, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B2D49CF95-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:549, rec_:549, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:25, seq_:32}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:33}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.437318] INFO [SERVER] run (obmp_disconnect.cpp:82) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221732588) [2024-10-16 14:57:21.437326] INFO [SERVER] run (obmp_disconnect.cpp:89) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=8] mark session id unused(sessid=3221732588) [2024-10-16 14:57:21.437325] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0f2742cdf0, session_id:3221696746, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:223}, err:5, last_decode_time:1729061841417817, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.437333] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221696746, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.437336] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221696746, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED789-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:452, rec_:452, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.437349] INFO [SERVER] run (obmp_disconnect.cpp:82) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221737211) [2024-10-16 14:57:21.437352] INFO [SERVER] run (obmp_disconnect.cpp:89) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221737211) [2024-10-16 14:57:21.437379] INFO [SERVER] run (obmp_disconnect.cpp:82) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221696746) [2024-10-16 14:57:21.437382] INFO [SERVER] run (obmp_disconnect.cpp:89) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221696746) [2024-10-16 14:57:21.439032] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E12-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11008, locations=[{table_id:11008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841439021, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841439021, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841439021, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.439056] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E12-0-0] [lt=23] success to process renew task(task={tenant_id:1, table_id:11008, add_timestamp:1729061841426869}, locations=[{table_id:11008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841439021, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841439021, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841439021, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.439076] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E13-0-0] [lt=13] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11003, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:21.445159] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=46] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:21.445765] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:21.445809] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061838714540000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:21.445848] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=38] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061838714540000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1001, freeze_info:{latest_snapshot_gc_scn:{val:1729061838714540000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}}) [2024-10-16 14:57:21.447612] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [22268][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392222955}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.447743] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [22268][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f52d2c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202787} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:17157}, prev=this=0x2b0f7f497818 trans_version={val:1729061779512891000, v:0} scn={val:1729061779512891000, v:0} tx_id={txid:392201751} prev=0x2b0f7f4976d8 next=(nil) modify_count=1 acc_checksum=1082530753 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2183}, snapshot_version={val:1729061841437167000, v:0}, *this={this=0x2b0f7f4973d0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f497818 latest_compact_node=(nil) max_trans_version={val:1729061779512891000, v:0} max_trans_id=392201751 max_elr_trans_version={val:1729061779512891000, v:0} max_elr_trans_id=392201751 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779512891000, v:0} min_modify_scn={val:1729061779512891000, v:0}}) [2024-10-16 14:57:21.447768] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [22268][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f52d2c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202787} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:17157}, ctx={this:0x2b0e2b254080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437614860, tablet_id:{id:209285}, table_iter:0x2b0e2b253210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437614860, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841437167000, v:0}, tx_id:{txid:392202787}, scn:{branch:0, seq:6880}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202787}, tx_desc:{this:0x2b0dba0d1850, tx_id:{txid:392202787}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841447444, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841430288, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c278d50, mem_ctx:0x2b0e0c27a2d0, tx_scn:{branch:0, seq:17157}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.447842] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [22268][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=73][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202787} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202787}, data:0x2b0e2b250438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:17157}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.447859] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392222955}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.447858] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [22268][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830046698655744},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202787} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.447873] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [22268][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.447879] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [22268][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046698655744},{"BIGINT":1729058400},{"VARCHAR":"AWøu", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.447870] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222955}, start_ts=1729061841435210, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222955}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:10}, op_sn:3, alloc_ts:1729061841430283, active_ts:1729061841430283, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441430283, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007559424656, first_scn:{branch:0, seq:9}, last_scn:{branch:0, seq:9}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841430282, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.447924] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [22268][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.448886] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6B-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:21.448899] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6B-0-0] [lt=12] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:21.449142] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E13-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11003, locations=[{table_id:11003, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841449129, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841449129, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841449129, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.449163] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E13-0-0] [lt=21] success to process renew task(task={tenant_id:1, table_id:11003, add_timestamp:1729061841434045}, locations=[{table_id:11003, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841449129, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841449129, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841449129, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:21.454685] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:24071}, to_seq_no={branch:0, seq:6881}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202787} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.459235] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:57:21.459251] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:21.459259] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:21.459264] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:21.459499] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=36, avg_log_batch_cnt=0, total_group_log_size=58453, avg_group_log_size=1623) [2024-10-16 14:57:21.460520] INFO [STORAGE] normal_running_env_check_ (ob_tenant_snapshot_service.cpp:279) [22822][T1004_TSnapSvc][T1004][Y0-0000000000000000-0-0] [lt=10] normal_running_env_check_ succ(this={is_inited_:true, is_running_:true, running_mode_:3, meta_loaded_:true, tg_id_:758}) [2024-10-16 14:57:21.460534] INFO [STORAGE] try_gc_tenant_snapshot_ (ob_tenant_snapshot_service.cpp:650) [22822][T1004_TSnapSvc][T1004][Y0-0000000000000000-0-0] [lt=13] try_gc_tenant_snapshot finished(ret=0, ret="OB_SUCCESS", this={is_inited_:true, is_running_:true, running_mode_:3, meta_loaded_:true, tg_id_:758}) [2024-10-16 14:57:21.461215] INFO [STORAGE] calculate_idle_time_ (ob_tenant_snapshot_service.cpp:829) [22822][T1004_TSnapSvc][T1004][Y0-0000000000000000-0-0] [lt=6] ObTenantSnapshotService thread idle time(tg_id_=758, idle_time=60000, running_mode_=3) [2024-10-16 14:57:21.461452] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:57:21.461594] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=139] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:21.461686] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.466084] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=7, avg_log_batch_cnt=1, total_group_log_size=854, avg_group_log_size=122) [2024-10-16 14:57:21.466094] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=854) [2024-10-16 14:57:21.466292] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061841466271000, start_id=1729061841466271000, end_id=1729061841466271001) [2024-10-16 14:57:21.466537] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22edefd8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222956} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f22ea3fd8 trans_version={val:1729061781201171000, v:0} scn={val:1729061781201171000, v:0} tx_id={txid:392222335} prev=0x2b0f22ea3e98 next=(nil) modify_count=1 acc_checksum=4190498330 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841466271001, v:0}, *this={this=0x2b0f22ea3b90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea3fd8 latest_compact_node=(nil) max_trans_version={val:1729061781201171000, v:0} max_trans_id=392222335 max_elr_trans_version={val:1729061781201171000, v:0} max_elr_trans_id=392222335 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781201171000, v:0} min_modify_scn={val:1729061781201171000, v:0}}) [2024-10-16 14:57:21.466563] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22edefd8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222956} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209267}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841466271001, v:0}, tx_id:{txid:392222956}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222956}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222956}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841466271, active_ts:1729061841466271, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841466270, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.466620] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222956} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222956}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.466634] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513903454212096},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222956} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.466646] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.466651] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513903454212096},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.466693] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.466743] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222956} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.467975] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:21.469177] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] handle_sp_rollback_resp(tx_id={txid:392222957}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.469208] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392222957}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.469218] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222957}, start_ts=1729061841468705, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222957}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1684}, op_sn:3, alloc_ts:1729061841466271, active_ts:1729061841467961, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441467961, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007592964894, first_scn:{branch:0, seq:1683}, last_scn:{branch:0, seq:1683}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841466280, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.469696] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D0-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061841460913, keepalive_ts:-1}) [2024-10-16 14:57:21.469709] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D0-0-0] [lt=12] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061841460913, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:21.471156] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.471470] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=303, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:21.471489] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=58, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:21.472500] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1729061841472242000, start_id=1729061841472242000, end_id=1729061841472242001) [2024-10-16 14:57:21.474873] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] handle_sp_rollback_resp(tx_id={txid:392222958}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.474945] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392222958}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.474957] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222958}, start_ts=1729061841472973, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222958}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841472242, active_ts:1729061841472242, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441472242, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007597352968, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841472241, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.476458] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:21.476469] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=11] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:21.476480] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:21.476478] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A23-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:21.476485] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:21.476490] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:21.476489] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A23-0-0] [lt=9] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:21.476492] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:21.476495] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:21.476498] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:21.476500] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:21.476503] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:21.476506] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:21.477291] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6df0be8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202790} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1097}, prev=this=0x2b0ec6d4c930 trans_version={val:1729061779518151001, v:0} scn={val:1729061779518151001, v:0} tx_id={txid:392201754} prev=0x2b0ec6d4c7f0 next=(nil) modify_count=1 acc_checksum=4151851990 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2082}, snapshot_version={val:1729061841475418000, v:0}, *this={this=0x2b0ec6d4c4e8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d4c930 latest_compact_node=(nil) max_trans_version={val:1729061779518151001, v:0} max_trans_id=392201754 max_elr_trans_version={val:1729061779518151001, v:0} max_elr_trans_id=392201754 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779518151001, v:0} min_modify_scn={val:1729061779518151001, v:0}}) [2024-10-16 14:57:21.477326] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6df0be8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202790} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1097}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606494, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606494, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841475418000, v:0}, tx_id:{txid:392202790}, scn:{branch:0, seq:6}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202790}, tx_desc:{this:0x2b0ec37a2150, tx_id:{txid:392202790}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841476512, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841475418, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1097}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.477382] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202790} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202790}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:1097}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.477399] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513897376665600},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202790} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.477414] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.477420] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513897376665600},{"BIGINT":1729058400},{"VARCHAR":"@¹Æ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.477470] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.477707] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2128}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202790} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.479622] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6df0f10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222959} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d75588 trans_version={val:1729061781209208001, v:0} scn={val:1729061781209208001, v:0} tx_id={txid:392222338} prev=0x2b0ec6d75448 next=(nil) modify_count=1 acc_checksum=2438062653 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841478625000, v:0}, *this={this=0x2b0ec6d75140 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d75588 latest_compact_node=(nil) max_trans_version={val:1729061781209208001, v:0} max_trans_id=392222338 max_elr_trans_version={val:1729061781209208001, v:0} max_elr_trans_id=392222338 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781209208001, v:0} min_modify_scn={val:1729061781209208001, v:0}}) [2024-10-16 14:57:21.479649] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6df0f10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222959} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841478625000, v:0}, tx_id:{txid:392222959}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222959}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222959}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841478625, active_ts:1729061841478625, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841478624, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.479701] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=51][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222959} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222959}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.479715] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654910319046021120},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222959} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.479725] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.479730] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910319046021120},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.479767] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.479798] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222959} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.482056] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392222960}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.482111] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392222960}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.482126] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222960}, start_ts=1729061841481692, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222960}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1037}, op_sn:3, alloc_ts:1729061841480012, active_ts:1729061841481047, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441481047, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007605662562, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841480012, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.482643] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061841482097000, start_id=1729061841482097000, end_id=1729061841482097001) [2024-10-16 14:57:21.484435] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22edf6b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202792} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2082}, prev=this=0x2b0f22e8a020 trans_version={val:1729061779521765000, v:0} scn={val:1729061779521765000, v:0} tx_id={txid:392201756} prev=0x2b0f22e89ee0 next=(nil) modify_count=1 acc_checksum=1910298845 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2563}, snapshot_version={val:1729061841483156000, v:0}, *this={this=0x2b0f22e89bd8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e8a020 latest_compact_node=(nil) max_trans_version={val:1729061779521765000, v:0} max_trans_id=392201756 max_elr_trans_version={val:1729061779521765000, v:0} max_elr_trans_id=392201756 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779521765000, v:0} min_modify_scn={val:1729061779521765000, v:0}}) [2024-10-16 14:57:21.484466] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22edf6b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202792} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2082}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606250, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606250, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841483156000, v:0}, tx_id:{txid:392202792}, scn:{branch:0, seq:1058}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202792}, tx_desc:{this:0x2b0ec37bbc70, tx_id:{txid:392202792}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061841484180, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841482099, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2082}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.484517] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=50][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202792} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202792}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:2082}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.484533] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":675513897414414336},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202792} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.484548] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.484552] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513897414414336},{"BIGINT":1729058400},{"VARCHAR":"@»", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.484598] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.485235] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2085}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202792} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.488037] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=42][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f465638 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222961} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f404f08 trans_version={val:1729061781216803000, v:0} scn={val:1729061781216803000, v:0} tx_id={txid:392222340} prev=0x2b0f3f404dc8 next=(nil) modify_count=1 acc_checksum=2084767405 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1056}, snapshot_version={val:1729061841487395000, v:0}, *this={this=0x2b0f3f404ac0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f404f08 latest_compact_node=(nil) max_trans_version={val:1729061781216803000, v:0} max_trans_id=392222340 max_elr_trans_version={val:1729061781216803000, v:0} max_elr_trans_id=392222340 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781216803000, v:0} min_modify_scn={val:1729061781216803000, v:0}}) [2024-10-16 14:57:21.488064] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f465638 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222961} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841487395000, v:0}, tx_id:{txid:392222961}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222961}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222961}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841487395, active_ts:1729061841487395, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841487395, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.488118] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222961} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222961}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.488132] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=0] primary key duplication conflict(key={"BIGINT":654910319058604032},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222961} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.488142] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.488147] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910319058604032},{"BIGINT":1729058400},{"VARCHAR":"@²ô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.488184] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.488205] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222961} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.488483] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:57:21.488492] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:21.488503] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=70475264, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:21.490897] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061841489744000, start_id=1729061841489744000, end_id=1729061841489744001) [2024-10-16 14:57:21.496477] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdfcdf0, session_id:3221735968, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:225}, err:5, last_decode_time:1729061841410535, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:224, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.496505] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=22] kill and revert session(conn.sessid_=3221735968, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.496515] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=10] sock ref clean, do destroy(*s={this:0x2b0e5cdfcdf0, session_id:3221735968, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:225}, err:5, last_decode_time:1729061841410535, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:224, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.496533] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=9] unlock session of tenant(conn.sessid_=3221735968, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:21.496554] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=20] connection close(sessid=3221735968, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B196AC91D-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:119, rec_:119, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:2, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:10}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:11}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:21.496580] INFO [SERVER] run (obmp_disconnect.cpp:82) [21678][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=12] free session successfully(sessid=3221735968) [2024-10-16 14:57:21.496590] INFO [SERVER] run (obmp_disconnect.cpp:89) [21678][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=9] mark session id unused(sessid=3221735968) [2024-10-16 14:57:21.496894] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb1f8a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222962} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac36f0 trans_version={val:1729061781217868001, v:0} scn={val:1729061781217868001, v:0} tx_id={txid:392222341} prev=0x2b0fadac35b0 next=(nil) modify_count=1 acc_checksum=2551350714 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1064}, snapshot_version={val:1729061841496586000, v:0}, *this={this=0x2b0fadac32a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac36f0 latest_compact_node=(nil) max_trans_version={val:1729061781217868001, v:0} max_trans_id=392222341 max_elr_trans_version={val:1729061781217868001, v:0} max_elr_trans_id=392222341 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781217868001, v:0} min_modify_scn={val:1729061781217868001, v:0}}) [2024-10-16 14:57:21.496919] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb1f8a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222962} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841496586000, v:0}, tx_id:{txid:392222962}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222962}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222962}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841496586, active_ts:1729061841496586, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841496585, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.496968] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=49][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222962} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222962}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.496982] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513903529709568},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222962} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.496993] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.496998] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513903529709568},{"BIGINT":1729058400},{"VARCHAR":"@Q", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.497071] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.497097] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222962} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.498206] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22edfd98 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222963} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f22ea46d8 trans_version={val:1729061781218951000, v:0} scn={val:1729061781218951000, v:0} tx_id={txid:392222342} prev=0x2b0f22ea4598 next=(nil) modify_count=1 acc_checksum=4080234910 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:9}, snapshot_version={val:1729061841497612000, v:0}, *this={this=0x2b0f22ea4290 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea46d8 latest_compact_node=(nil) max_trans_version={val:1729061781218951000, v:0} max_trans_id=392222342 max_elr_trans_version={val:1729061781218951000, v:0} max_elr_trans_id=392222342 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781218951000, v:0} min_modify_scn={val:1729061781218951000, v:0}}) [2024-10-16 14:57:21.498269] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22edfd98 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222963} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209267}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841497612000, v:0}, tx_id:{txid:392222963}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222963}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222963}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841497612, active_ts:1729061841497612, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841497611, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.498332] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222963} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222963}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.498349] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654910319083769856},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222963} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.498363] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.498368] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910319083769856},{"BIGINT":1729058400},{"VARCHAR":"@õ=", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.498414] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.498441] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222963} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.501934] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392222964}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.501971] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392222964}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.501981] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222964}, start_ts=1729061841501594, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222964}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2117}, op_sn:3, alloc_ts:1729061841498669, active_ts:1729061841500785, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441500785, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007625766488, first_scn:{branch:0, seq:1062}, last_scn:{branch:0, seq:1062}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841498668, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.505974] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061831504951, end_time=1729061841505973, dag_status={}) [2024-10-16 14:57:21.505993] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=17] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:21.506001] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:21.506004] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:21.506008] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:21.506011] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:21.506014] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:21.506019] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:21.506024] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:21.506027] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:21.506034] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506044] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506050] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506054] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506059] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506066] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506076] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506080] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506087] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506091] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506097] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506101] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506108] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506112] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506119] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506123] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506127] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506134] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506137] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506144] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506148] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506154] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506158] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506165] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506169] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506173] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506180] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506183] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506190] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506194] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506201] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506205] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506211] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506215] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506224] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506228] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506236] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506243] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506253] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506258] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506266] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506273] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506283] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506287] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506294] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506297] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506302] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506308] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506312] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506319] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506322] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506332] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506337] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506341] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506348] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506352] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506356] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506362] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:21.506367] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:21.506373] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:21.506377] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:21.506381] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:21.506383] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:21.506388] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:21.506390] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:21.506394] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:21.506396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:21.506405] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:21.506408] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:21.508618] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392222965}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.508649] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392222965}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.508658] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222965}, start_ts=1729061841507517, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222965}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1038}, op_sn:3, alloc_ts:1729061841506423, active_ts:1729061841507460, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441507460, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007631534410, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841506422, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.512555] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392222966}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.512593] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392222966}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.512607] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222966}, start_ts=1729061841512349, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222966}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1057}, op_sn:3, alloc_ts:1729061841510633, active_ts:1729061841511692, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441511692, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332496959569, first_scn:{branch:0, seq:1056}, last_scn:{branch:0, seq:1056}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841510638, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.515412] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392222967}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.515464] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31] all savepoint rollback succeed(tx.tx_id={txid:392222967}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.515477] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222967}, start_ts=1729061841515215, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222967}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1018}, op_sn:3, alloc_ts:1729061841513758, active_ts:1729061841514779, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441514779, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332499922676, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841513761, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.516279] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061841515837000, start_id=1729061841515837000, end_id=1729061841515837001) [2024-10-16 14:57:21.516305] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061841516301], gts:1729061841515837000, latest_srr:[mts=1729061841516301]}) [2024-10-16 14:57:21.517741] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb1ff80 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222968} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac3df0 trans_version={val:1729061781232958002, v:0} scn={val:1729061781232958002, v:0} tx_id={txid:392222347} prev=0x2b0fadac3cb0 next=(nil) modify_count=1 acc_checksum=1131753190 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841516895000, v:0}, *this={this=0x2b0fadac39a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac3df0 latest_compact_node=(nil) max_trans_version={val:1729061781232958002, v:0} max_trans_id=392222347 max_elr_trans_version={val:1729061781232958002, v:0} max_elr_trans_id=392222347 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781232958002, v:0} min_modify_scn={val:1729061781232958002, v:0}}) [2024-10-16 14:57:21.517772] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb1ff80 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222968} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841516895000, v:0}, tx_id:{txid:392222968}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222968}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222968}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841516895, active_ts:1729061841516895, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841516896, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.517826] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222968} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222968}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.517844] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":675513903592624128},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222968} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.517858] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.517864] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513903592624128},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.517921] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.517946] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222968} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.518570] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3F11-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=44, avg_log_gen_to_freeze_time=192, avg_log_gen_to_submit_time=218, avg_log_submit_to_flush_time=714, avg_log_submit_to_slide_time=1604) [2024-10-16 14:57:21.519372] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392222969}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.519404] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392222969}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.519415] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222969}, start_ts=1729061841519168, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222969}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1012}, op_sn:3, alloc_ts:1729061841517957, active_ts:1729061841518968, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441518968, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332503861667, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841517956, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.521958] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392222970}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.522017] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=44] all savepoint rollback succeed(tx.tx_id={txid:392222970}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.522029] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222970}, start_ts=1729061841521796, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222970}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841521039, active_ts:1729061841521039, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441521039, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332506547088, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841521038, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.524263] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:21.526772] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFB5-0-0] [lt=20] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=1414, avg_log_submit_to_slide_time=3230) [2024-10-16 14:57:21.526848] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392222971}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.526883] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26] all savepoint rollback succeed(tx.tx_id={txid:392222971}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.526891] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222971}, start_ts=1729061841525673, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222971}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2182}, op_sn:3, alloc_ts:1729061841523156, active_ts:1729061841525341, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441525341, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007649878743, first_scn:{branch:0, seq:1158}, last_scn:{branch:0, seq:1158}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841523159, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.531006] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f52d9a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222972} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f7f4bd8a8 trans_version={val:1729061781247786001, v:0} scn={val:1729061781247786001, v:0} tx_id={txid:392222351} prev=0x2b0f7f4bd768 next=(nil) modify_count=1 acc_checksum=1265990423 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841530634000, v:0}, *this={this=0x2b0f7f4bd460 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4bd8a8 latest_compact_node=(nil) max_trans_version={val:1729061781247786001, v:0} max_trans_id=392222351 max_elr_trans_version={val:1729061781247786001, v:0} max_elr_trans_id=392222351 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781247786001, v:0} min_modify_scn={val:1729061781247786001, v:0}}) [2024-10-16 14:57:21.531040] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f52d9a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222972} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209285}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841530634000, v:0}, tx_id:{txid:392222972}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222972}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222972}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841530634, active_ts:1729061841530634, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841530633, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.531094] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222972} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392222972}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.531112] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":675513903630372864},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222972} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.531127] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.531135] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513903630372864},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.531190] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.531228] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222972} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.531983] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6df19a8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222973} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d76b38 trans_version={val:1729061781249908000, v:0} scn={val:1729061781249908000, v:0} tx_id={txid:392222352} prev=0x2b0ec6d769f8 next=(nil) modify_count=1 acc_checksum=3286749292 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1087}, snapshot_version={val:1729061841531692000, v:0}, *this={this=0x2b0ec6d766f0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d76b38 latest_compact_node=(nil) max_trans_version={val:1729061781249908000, v:0} max_trans_id=392222352 max_elr_trans_version={val:1729061781249908000, v:0} max_elr_trans_id=392222352 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781249908000, v:0} min_modify_scn={val:1729061781249908000, v:0}}) [2024-10-16 14:57:21.532007] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6df19a8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392222973} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841531692000, v:0}, tx_id:{txid:392222973}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392222973}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392222973}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841531692, active_ts:1729061841531692, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841531691, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.532051] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319171850240},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222973} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.532098] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222973} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.533556] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392222974}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.533583] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392222974}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.533591] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222974}, start_ts=1729061841533297, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222974}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1035}, op_sn:3, alloc_ts:1729061841531692, active_ts:1729061841532734, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441532734, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332517941181, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841531699, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.533855] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:21.533882] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513897452163072},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202794} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.534446] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4215}, to_seq_no={branch:0, seq:3152}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202794} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.536205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=44] get number(ret=0, service_type_=0, range=1, base_id=1729061841535916000, start_id=1729061841535916000, end_id=1729061841535916001) [2024-10-16 14:57:21.537271] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=49] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:21.537288] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:21.537307] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061841536974000, start_id=1729061841536974000, end_id=1729061841536974001) [2024-10-16 14:57:21.538188] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=902713344, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=171028480, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:21.538243] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=26] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:21.538255] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:21.538291] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=967602176, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=106139648, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:21.538327] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:21.538334] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:21.538360] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=942559232, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=131182592, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:21.538399] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:21.538407] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011853, drift_size_=0, pre_drift_size=1498290, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:21.538452] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=11] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=954744832, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=118996992, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:21.538502] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=13] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:21.538512] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:21.539053] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392222975}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.539109] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392222975}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.539118] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222975}, start_ts=1729061841538781, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222975}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3181}, op_sn:3, alloc_ts:1729061841534856, active_ts:1729061841538038, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441538038, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332521642282, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841534857, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.539708] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061841539079000, start_id=1729061841539079000, end_id=1729061841539079001) [2024-10-16 14:57:21.541292] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513903668121600},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222976} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.541368] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222976} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.543791] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392222977}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.543859] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392222977}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.543871] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222977}, start_ts=1729061841542555, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222977}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1063}, op_sn:3, alloc_ts:1729061841541198, active_ts:1729061841542261, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441542261, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007666887910, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841541198, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.551985] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:21.556753] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26] handle_sp_rollback_resp(tx_id={txid:392222978}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.556796] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31] all savepoint rollback succeed(tx.tx_id={txid:392222978}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.556818] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222978}, start_ts=1729061841556459, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222978}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1067}, op_sn:3, alloc_ts:1729061841555106, active_ts:1729061841556172, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441556172, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332541145764, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841555105, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.558819] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5070-0-0] [lt=39] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:21.558834] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5070-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=7723, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:21.558828] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=47215, hit_cnt=30, read_cnt=30, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:21.559740] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] handle_sp_rollback_resp(tx_id={txid:392222979}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.559805] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392222979}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.559817] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222979}, start_ts=1729061841559467, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222979}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1062}, op_sn:3, alloc_ts:1729061841558288, active_ts:1729061841559351, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441559351, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332544187349, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841558289, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.561745] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.562879] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23] handle_sp_rollback_resp(tx_id={txid:392222980}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.562922] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392222980}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.562939] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222980}, start_ts=1729061841562580, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222980}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1098}, op_sn:3, alloc_ts:1729061841561439, active_ts:1729061841562538, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441562538, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332547224942, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841561440, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.566149] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:21.566165] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:21.567881] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392222981}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.567938] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392222981}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.567951] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222981}, start_ts=1729061841566957, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222981}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841566212, active_ts:1729061841566212, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441566212, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007691304514, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841566212, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.571219] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=24] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:21.571236] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:21.571245] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.571415] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.571447] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.571471] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.571483] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.572851] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D1-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061841540054, keepalive_ts:-1}) [2024-10-16 14:57:21.572865] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D1-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061841540054, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:21.575204] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513897577992192},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202799} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.575407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1729061841573947000, start_id=1729061841573947000, end_id=1729061841573947001) [2024-10-16 14:57:21.576218] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5306}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202799} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.578460] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] handle_sp_rollback_resp(tx_id={txid:392222982}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.578507] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392222982}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.578523] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222982}, start_ts=1729061841576081, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222982}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2087}, op_sn:3, alloc_ts:1729061841573947, active_ts:1729061841576034, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441576034, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332560704598, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841573947, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.578933] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1002}, {id:1001}]) [2024-10-16 14:57:21.579281] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061841579216000, start_id=1729061841579216000, end_id=1729061841579216001) [2024-10-16 14:57:21.582016] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392222983}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.582062] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392222983}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.582076] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222983}, start_ts=1729061841581835, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222983}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1065}, op_sn:3, alloc_ts:1729061841580278, active_ts:1729061841581341, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441581341, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332566416111, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841580279, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.583417] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061841582398000, start_id=1729061841582398000, end_id=1729061841582398001) [2024-10-16 14:57:21.584240] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513903743619072},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222984} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.584326] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222984} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.585871] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392222985}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.585910] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392222985}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.585920] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222985}, start_ts=1729061841585650, retries=0, savepoint={branch:0, seq:5}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222985}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1066}, op_sn:3, alloc_ts:1729061841584533, active_ts:1729061841585598, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441585598, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332570310937, first_scn:{branch:0, seq:8}, last_scn:{branch:0, seq:8}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841584532, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.586363] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1001, min_version={val:1729061841340231000, v:0}, base_version={val:1729061841130910000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061841387110000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061841586351, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061841340231000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061841481451, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061841399942000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061841575701, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:21.587270] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061841585598000, start_id=1729061841585598000, end_id=1729061841585598001) [2024-10-16 14:57:21.589277] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513903781367808},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222986} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.589347] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222986} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.592107] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392222987}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.592161] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392222987}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.592177] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222987}, start_ts=1729061841591443, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222987}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1016}, op_sn:3, alloc_ts:1729061841589487, active_ts:1729061841590499, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441590499, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007715377275, first_scn:{branch:0, seq:1015}, last_scn:{branch:0, seq:1015}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841589486, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.596194] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=4559, mem_used=8318976) [2024-10-16 14:57:21.598427] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513903793950720},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222988} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.598506] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222988} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.599413] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319310262272},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222989} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.599467] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222989} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.600748] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D59-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:21.600777] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D59-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061841125665004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061841125665004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:21.601772] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392222990}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.601805] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392222990}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.601818] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222990}, start_ts=1729061841601441, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222990}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1073}, op_sn:3, alloc_ts:1729061841600019, active_ts:1729061841601089, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441601089, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007725718104, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841600018, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.602037] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D59-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061841601089000, start_id=1729061841601089000, end_id=1729061841601089001) [2024-10-16 14:57:21.604625] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D59-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061841604315000, start_id=1729061841604315000, end_id=1729061841604315001) [2024-10-16 14:57:21.608990] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513897703821312},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202804} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.609851] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4185}, to_seq_no={branch:0, seq:2072}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202804} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.610904] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319322845184},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222991} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.610985] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222991} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.614863] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27] handle_sp_rollback_resp(tx_id={txid:392222992}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.614904] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392222992}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.614933] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222992}, start_ts=1729061841614636, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222992}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2118}, op_sn:3, alloc_ts:1729061841611674, active_ts:1729061841613791, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441613791, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332597229410, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841611673, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.619195] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F8B-0-0] [lt=35] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:21.619211] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F8B-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13378, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:21.619216] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8F8B-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=8, avg_log_submit_to_flush_time=387, avg_log_submit_to_slide_time=13370) [2024-10-16 14:57:21.619244] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:21.620350] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392222993}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.620414] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392222993}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.620426] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222993}, start_ts=1729061841618759, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222993}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1050}, op_sn:3, alloc_ts:1729061841617024, active_ts:1729061841618073, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441618073, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007743030159, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841617023, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.623819] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=71447) [2024-10-16 14:57:21.623869] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=71447, hit_cnt=42, read_cnt=42, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:21.624631] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513903856865280},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222994} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.624691] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222994} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.626093] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B8D-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061841625527000, start_id=1729061841625527000, end_id=1729061841625527001) [2024-10-16 14:57:21.626189] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B91-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061841625527000, start_id=1729061841625527000, end_id=1729061841625527001) [2024-10-16 14:57:21.626290] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B8E-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061841625527000, start_id=1729061841625527000, end_id=1729061841625527001) [2024-10-16 14:57:21.626422] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392222995}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.626494] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392222995}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.626518] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222995}, start_ts=1729061841626082, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222995}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1063}, op_sn:3, alloc_ts:1729061841624460, active_ts:1729061841625527, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441625527, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332610704798, first_scn:{branch:0, seq:1062}, last_scn:{branch:0, seq:1062}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841624467, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.627111] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":675513897728987136},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202805} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.631749] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] Cache replace map node details(ret=0, replace_node_count=19, replace_time=0, replace_start_pos=1163909, replace_num=31457) [2024-10-16 14:57:21.631768] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=19, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:21.631901] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392222996}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.631925] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392222996}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.631942] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222996}, start_ts=1729061841629441, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222996}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1056}, op_sn:3, alloc_ts:1729061841627644, active_ts:1729061841628705, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441628705, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007753779598, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841627649, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.638414] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392222997}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.638468] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392222997}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.638483] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392222997}, start_ts=1729061841635200, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392222997}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061841634022, active_ts:1729061841635079, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441635079, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007759620422, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841634021, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.640144] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:14775}, to_seq_no={branch:0, seq:1067}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202805} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.641855] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513903907196928},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222998} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.641922] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222998} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.642720] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319423508480},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392222999} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.642769] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392222999} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.643987] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] handle_sp_rollback_resp(tx_id={txid:392223000}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.644017] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392223000}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.644031] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223000}, start_ts=1729061841643783, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223000}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1055}, op_sn:3, alloc_ts:1729061841642422, active_ts:1729061841643485, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441643485, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332628587300, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841642430, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.647103] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223001}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.647397] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41] all savepoint rollback succeed(tx.tx_id={txid:392223001}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.647410] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223001}, start_ts=1729061841646881, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223001}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061841645611, active_ts:1729061841646670, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441646670, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007771222305, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841645612, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.650901] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513897766735872},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202807} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.651054] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223002}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.651083] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223002}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.651092] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223002}, start_ts=1729061841650582, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223002}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841649856, active_ts:1729061841649856, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441649856, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332635411768, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841649855, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.651206] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2235}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202807} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.651581] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:21.653224] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319473840128},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223003} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.653283] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223003} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.654749] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223004}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.654780] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392223004}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.654788] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223004}, start_ts=1729061841654510, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223004}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1057}, op_sn:3, alloc_ts:1729061841653161, active_ts:1729061841654219, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441654219, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332639341560, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841653162, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.660410] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223005}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.660438] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392223005}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.660450] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223005}, start_ts=1729061841660215, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223005}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1146}, op_sn:3, alloc_ts:1729061841658636, active_ts:1729061841659785, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441659785, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332645012916, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841658639, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.661822] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.663352] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26147][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB18CC6EA-0-0] [lt=24] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:21.664946] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223006}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.665002] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223006}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.665018] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223006}, start_ts=1729061841664305, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223006}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2120}, op_sn:3, alloc_ts:1729061841661905, active_ts:1729061841664024, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441664024, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007788518284, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841661904, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.666161] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:57:21.666173] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:57:21.666176] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:21.666801] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838772614000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850098662000, latest_log_ts:{val:1729061833508923001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:21.667506] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=47] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838772614000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:21.668157] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=37] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838772614000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:21.668800] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=30] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061838772614000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:21.669046] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [21605][T1_CompactionRe][T1][YB4A0A000407-00061E9973ED0D6E-0-0] [lt=18] InfoPoolResizeTask(cost_ts=8) [2024-10-16 14:57:21.671265] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.681285] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3D48-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=49, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:21.681522] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=48, stat_interval=1000000, avg (count/cost)=344, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:21.681537] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=49, stat_interval=1000000, avg (count/cost)=133, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:21.681563] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=49, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:21.681948] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] get number(ret=0, service_type_=0, range=1, base_id=1729061841680941000, start_id=1729061841680941001, end_id=1729061841680941002) [2024-10-16 14:57:21.686634] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223007}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.686677] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392223007}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.686689] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223007}, start_ts=1729061841682477, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223007}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061841680941, active_ts:1729061841682003, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441682003, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007806805702, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841680942, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.687876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061841687320000, start_id=1729061841687320000, end_id=1729061841687320001) [2024-10-16 14:57:21.691029] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061841690862000, start_id=1729061841690862000, end_id=1729061841690862001) [2024-10-16 14:57:21.691102] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:21.691115] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3545) [2024-10-16 14:57:21.691123] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:21.695530] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:21.700496] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392223008}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.700536] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223008}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.700546] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223008}, start_ts=1729061841695758, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223008}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061841694006, active_ts:1729061841695066, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441695066, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007820096928, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841694005, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.706153] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319536754688},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223009} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.706242] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223009} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.707820] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392223010}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.707903] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] all savepoint rollback succeed(tx.tx_id={txid:392223010}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.707916] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223010}, start_ts=1729061841707548, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223010}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841706756, active_ts:1729061841706756, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441706756, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332692297606, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841706755, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.710271] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319549337600},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223011} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.710367] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223011} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.711466] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904020443136},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223012} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.711554] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223012} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.712973] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223013}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.713016] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23] all savepoint rollback succeed(tx.tx_id={txid:392223013}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.713030] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223013}, start_ts=1729061841712761, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223013}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841712074, active_ts:1729061841712074, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441712074, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332697575964, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841712073, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.715315] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904033026048},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223014} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.715387] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223014} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.717058] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=355, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:21.717073] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:21.717084] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:21.717108] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:21.717232] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223015}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.717260] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392223015}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.717268] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223015}, start_ts=1729061841716865, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223015}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1062}, op_sn:3, alloc_ts:1729061841715257, active_ts:1729061841716318, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441716318, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007841184217, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841715258, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.721607] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223016}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.721649] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=8] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:21.721690] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=43] all savepoint rollback succeed(tx.tx_id={txid:392223016}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.721700] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223016}, start_ts=1729061841721186, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223016}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:8}, op_sn:3, alloc_ts:1729061841720582, active_ts:1729061841720582, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441720582, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007845581209, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841720581, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.724551] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=32] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:21.726449] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:21.726466] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:21.736189] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=21] BlockGCTimerTask success(ret=0, cost_time_us=12, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1632, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:21.736291] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061841735381000, start_id=1729061841735381000, end_id=1729061841735381001) [2024-10-16 14:57:21.737462] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061841736420000, start_id=1729061841736420000, end_id=1729061841736420001) [2024-10-16 14:57:21.737560] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392223017}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.737926] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223017}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.737948] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223017}, start_ts=1729061841736536, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223017}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:8538}, op_sn:3, alloc_ts:1729061841727881, active_ts:1729061841736420, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441736420, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007859396838, first_scn:{branch:0, seq:2242}, last_scn:{branch:0, seq:2242}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841727882, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.739884] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061841738590000, start_id=1729061841738590000, end_id=1729061841738590001) [2024-10-16 14:57:21.747502] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223018}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.747531] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223018}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.747543] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223018}, start_ts=1729061841745419, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223018}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2078}, op_sn:3, alloc_ts:1729061841743239, active_ts:1729061841745322, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441745322, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007869545643, first_scn:{branch:0, seq:1058}, last_scn:{branch:0, seq:1058}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841743244, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.748821] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:21.748846] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=14] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:21.748862] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:21.751945] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319612252160},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223019} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.752020] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223019} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.760017] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=17] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:21.761324] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223020}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.761399] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=3] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061837748308, last_first_register_time_us=1722494157071428, register_time_us=1729061837748308, curr_time_us=1729061841761397) [2024-10-16 14:57:21.761427] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=14] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061841761413, last_submit_register_req_time_us=1729061841761413, register_parent_reason=3) [2024-10-16 14:57:21.761428] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392223020}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.761435] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:21.761447] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223020}, start_ts=1729061841760240, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223020}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:8369}, op_sn:3, alloc_ts:1729061841752672, active_ts:1729061841759986, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441759986, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007878055297, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841751617, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.761881] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.762572] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=49] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}) [2024-10-16 14:57:21.762623] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513898056142848},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202821} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.764171] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2118}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202821} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.768479] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545341893758976},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223021} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.768548] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223021} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.768784] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:21.771312] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.772741] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061841772686000, start_id=1729061841772686000, end_id=1729061841772686001) [2024-10-16 14:57:21.774342] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223022}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.774395] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392223022}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.774414] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223022}, start_ts=1729061841773682, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223022}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:4272}, op_sn:3, alloc_ts:1729061841768418, active_ts:1729061841772686, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441772686, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007896786777, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841768418, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.776077] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:21.778163] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513898093891584},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202823} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.778531] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3197}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202823} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.780566] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=12] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:21.782011] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061841781157000, start_id=1729061841781157000, end_id=1729061841781157001) [2024-10-16 14:57:21.784480] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392223023}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.784524] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223023}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.784544] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223023}, start_ts=1729061841784048, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223023}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2209}, op_sn:3, alloc_ts:1729061841781157, active_ts:1729061841783362, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441783362, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007908368208, first_scn:{branch:0, seq:2208}, last_scn:{branch:0, seq:2208}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841781156, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.787364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061841786539000, start_id=1729061841786539000, end_id=1729061841786539001) [2024-10-16 14:57:21.787431] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:21.787446] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10789, avg_group_log_size=567) [2024-10-16 14:57:21.787455] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10789) [2024-10-16 14:57:21.787472] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:21.787687] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=348, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:21.787700] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10789) [2024-10-16 14:57:21.787709] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=66, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:21.787730] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10789) [2024-10-16 14:57:21.787742] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:21.787880] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F937E-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:21.787886] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F937E-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=1621, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:21.787890] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F937E-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=604, avg_log_submit_to_slide_time=1591) [2024-10-16 14:57:21.787991] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33705, start_service_tstamp_=1722494263684720, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:21.788492] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.788766] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789650] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789668] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789681] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789692] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789707] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789723] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789738] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789751] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789766] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:21.789815] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061841789745000, start_id=1729061841789745000, end_id=1729061841789745001) [2024-10-16 14:57:21.792723] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545341994422272},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223024} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.792803] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223024} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.793645] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545342032171008},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223025} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.793704] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223025} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.795151] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223026}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.795187] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223026}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.795195] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223026}, start_ts=1729061841794913, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223026}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841794121, active_ts:1729061841794121, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441794121, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332779719022, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841794120, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.797711] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223027}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.797755] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223027}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.797769] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223027}, start_ts=1729061841797474, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223027}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1057}, op_sn:3, alloc_ts:1729061841796242, active_ts:1729061841797302, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441797302, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332782300089, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841796245, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.799835] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545342191554561},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223028} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.799904] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223028} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.801545] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223029}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.801577] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392223029}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.801588] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223029}, start_ts=1729061841801350, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223029}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841800485, active_ts:1729061841800485, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441800485, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332786182324, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841800484, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.804457] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=17] new mysql sessid created(conn.sessid_=3221735823, support_ssl=false) [2024-10-16 14:57:21.804489] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12] sm conn init succ(conn.sessid_=3221735823, sess.client_addr_="10.0.4.7:53344") [2024-10-16 14:57:21.804497] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0f27406030, session_id:3221735823, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.809032] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27406030, session_id:3221735823, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.809054] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=19] kill and revert session(conn.sessid_=3221735823, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:21.809084] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] sock ref clean, do destroy(*s={this:0x2b0f27406030, session_id:3221735823, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:21.809094] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:223) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=8] mark session id unused(conn.sessid_=3221735823) [2024-10-16 14:57:21.809098] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221735823, proxy_sessid=0, tenant_id=0, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=false, ret=0, trace_id=Y0-0000000000000000-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:0], disconnect_state=0) [2024-10-16 14:57:21.810182] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223030}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.810229] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223030}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.810247] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223030}, start_ts=1729061841809150, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223030}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:5185}, op_sn:3, alloc_ts:1729061841803784, active_ts:1729061841808968, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441808968, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007930689464, first_scn:{branch:0, seq:8}, last_scn:{branch:0, seq:8}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841803783, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.815134] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=11][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:21.817744] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1003}, {id:1002}]) [2024-10-16 14:57:21.818776] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656752, miss_cnt:17, cache_read_size:8050140474, hit_ratio:"9.999997286805516472e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:21.821408] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223033}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.821442] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=42] all savepoint rollback succeed(tx.tx_id={txid:392223033}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.821451] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223033}, start_ts=1729061841820008, retries=0, savepoint={branch:0, seq:1079}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223033}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1082}, op_sn:3, alloc_ts:1729061841819506, active_ts:1729061841819506, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441819506, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007944415166, first_scn:{branch:0, seq:1081}, last_scn:{branch:0, seq:1081}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841818429, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.826727] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=225, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:21.826756] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=393, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:21.827272] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:21.827286] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:21.827301] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52208170, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:21.833425] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=17, replace_time=0, replace_start_pos=1195366, replace_num=31457) [2024-10-16 14:57:21.833443] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=17, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:21.836159] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223034}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.836194] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223034}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.836204] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223034}, start_ts=1729061841834436, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223034}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841833504, active_ts:1729061841833504, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441833504, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007958793483, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841833503, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.843929] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223035}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.843986] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41] all savepoint rollback succeed(tx.tx_id={txid:392223035}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.844001] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223035}, start_ts=1729061841843749, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223035}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841843013, active_ts:1729061841843013, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441843013, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332828456145, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841843012, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.846518] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392223036}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.846568] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223036}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.846581] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223036}, start_ts=1729061841846348, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223036}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1067}, op_sn:3, alloc_ts:1729061841845131, active_ts:1729061841846199, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441846199, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332831143660, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841845132, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.847927] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=56, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:21.849054] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223037}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.849101] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223037}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.849114] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223037}, start_ts=1729061841848876, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223037}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841848294, active_ts:1729061841848294, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441848294, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332833744744, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841848293, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.849826] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3195, mem_used=6239232) [2024-10-16 14:57:21.850758] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=25] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139560911, miss_cnt:1, cache_read_size:152590314445, hit_ratio:"9.999999928346698930e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:21.852789] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513898232303616},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202829} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.852923] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223038}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.852944] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223038}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.852951] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223038}, start_ts=1729061841851951, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223038}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061841850377, active_ts:1729061841851435, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441851435, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007976292399, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841850376, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.853119] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202829} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.856956] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=40][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:21.856989] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545342472572928},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223039} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.857040] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223039} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.857235] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=64, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:21.857253] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=64, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:21.857488] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=64, stat_interval=1000000, avg (count/cost)=246, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:21.857503] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=100914) [2024-10-16 14:57:21.857514] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=64, stat_interval=1000000, avg (count/cost)=323, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:21.857527] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=100914) [2024-10-16 14:57:21.857539] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=64, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:21.857790] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=10] [APPLY STAT CB APPEND COST TIME](cur_stat_count=176, stat_interval=5000000, avg (count/cost)=37, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:57:21.857801] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=176, stat_interval=5000000, avg (count/cost)=887, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:57:21.857806] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=176, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:57:21.857809] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=176, stat_interval=5000000, avg (count/cost)=-587, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:57:21.857813] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=176, stat_interval=5000000, avg (count/cost)=337, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:57:21.857895] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545342497738752},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223040} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.857943] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223040} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.858709] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545342548070400},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223041} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.858757] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223041} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.859327] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545342585819136},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223042} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.859373] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1077}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223042} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.859794] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AC7-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:21.859919] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=233, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:21.859934] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:21.859963] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:21.860176] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545342610984960},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223043} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.860230] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223043} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.861614] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223044}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.861651] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] all savepoint rollback succeed(tx.tx_id={txid:392223044}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.861662] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223044}, start_ts=1729061841861331, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223044}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841860267, active_ts:1729061841860267, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441860267, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042007985690652, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841860270, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.861941] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.865965] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598498, miss_cnt:9, cache_read_size:149777554303, hit_ratio:"9.999999345923135108e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:21.866527] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223045}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.866548] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392223045}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.866560] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223045}, start_ts=1729061841866102, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223045}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841865573, active_ts:1729061841865573, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441865573, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332850936621, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841865572, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.866715] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3F60-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=8, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=435, avg_log_submit_to_slide_time=1064) [2024-10-16 14:57:21.868958] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513898320384000},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202831} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.869374] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601308, offset=9603054}, end_cursor:ObLogCursor{file_id=192, log_id=47601309, offset=9603338}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a44050, len:2066, log_data_len:284, seq:47601308, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:21.869800] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2121}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202831} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.871669] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.871813] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA490-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061841870990000, start_id=1729061841870990000, end_id=1729061841870990001) [2024-10-16 14:57:21.872299] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3545) [2024-10-16 14:57:21.872327] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3545) [2024-10-16 14:57:21.872845] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26143][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB26CC6B7-0-0] [lt=25] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:57:21.872956] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] [APPLY STAT CB APPEND COST TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=40, this=0x2b0e07718280, extra_info="") [2024-10-16 14:57:21.872969] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=14] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=1405, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:57:21.872997] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=27] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:57:21.873001] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=-408, this=0x2b0e07718910, extra_info="") [2024-10-16 14:57:21.873004] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=1037, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:57:21.873089] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257674, offset=39008193}, end_cursor:ObLogCursor{file_id=197, log_id=49257675, offset=39008476}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29e4050, len:2111, log_data_len:283, seq:49257674, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:21.873309] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25] handle_sp_rollback_resp(tx_id={txid:392223046}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.873375] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31] all savepoint rollback succeed(tx.tx_id={txid:392223046}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.873386] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223046}, start_ts=1729061841872404, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223046}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2158}, op_sn:3, alloc_ts:1729061841868837, active_ts:1729061841870990, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441870990, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332854885702, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841868836, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.873557] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601309, offset=9603338}, end_cursor:ObLogCursor{file_id=192, log_id=47601310, offset=9603612}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c1c050, len:1782, log_data_len:274, seq:47601309, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:21.873629] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B1C-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:21.873637] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B1C-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1274, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:21.876097] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB35B-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061841875964000, start_id=1729061841875964000, end_id=1729061841875964001) [2024-10-16 14:57:21.876308] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061841875964000, start_id=1729061841875964000, end_id=1729061841875964001) [2024-10-16 14:57:21.877979] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86C-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:21.877998] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86C-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061841864503000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061841864503000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:21.878033] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257675, offset=39008476}, end_cursor:ObLogCursor{file_id=197, log_id=49257676, offset=39008758}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cdc050, len:1828, log_data_len:282, seq:49257675, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:21.878652] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601310, offset=9603612}, end_cursor:ObLogCursor{file_id=192, log_id=47601311, offset=9603886}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a1c050, len:1508, log_data_len:274, seq:47601310, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:21.878939] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601311, offset=9603886}, end_cursor:ObLogCursor{file_id=192, log_id=47601312, offset=9604160}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2964050, len:1234, log_data_len:274, seq:47601311, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:21.880560] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545342774562816},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223047} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.880649] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223047} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.881020] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA86C-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061841879979000, start_id=1729061841879979000, end_id=1729061841879979001) [2024-10-16 14:57:21.881526] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545342824894464},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223048} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.881592] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223048} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.881991] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257676, offset=39008758}, end_cursor:ObLogCursor{file_id=197, log_id=49257677, offset=39009040}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd295c050, len:1546, log_data_len:282, seq:49257676, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:21.882641] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257677, offset=39009040}, end_cursor:ObLogCursor{file_id=197, log_id=49257678, offset=39009322}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd29f0050, len:1264, log_data_len:282, seq:49257677, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:21.882707] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650019, miss_cnt:32, cache_read_size:8050393337, hit_ratio:"9.999994892262736856e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:21.882850] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061841882138000, start_id=1729061841882138000, end_id=1729061841882138001) [2024-10-16 14:57:21.883289] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:21.883431] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223049}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.883466] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] all savepoint rollback succeed(tx.tx_id={txid:392223049}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.883478] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223049}, start_ts=1729061841883247, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223049}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061841882138, active_ts:1729061841883197, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441883197, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332868039700, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841882138, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.886313] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223050}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.886346] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223050}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.886354] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223050}, start_ts=1729061841886051, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223050}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841885317, active_ts:1729061841885317, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441885317, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332870878949, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841885316, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.889923] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392223051}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.889984] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA86C-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061841889563000, start_id=1729061841889563000, end_id=1729061841889563001) [2024-10-16 14:57:21.890150] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41] all savepoint rollback succeed(tx.tx_id={txid:392223051}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.890159] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223051}, start_ts=1729061841889735, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223051}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1055}, op_sn:3, alloc_ts:1729061841888505, active_ts:1729061841889563, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441889563, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332874570521, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841888508, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.891494] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061841890611000, start_id=1729061841890611000, end_id=1729061841890611001) [2024-10-16 14:57:21.891666] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=2] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:21.891682] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061841890611000, start_id=1729061841890611000, end_id=1729061841890611001) [2024-10-16 14:57:21.892377] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513898345549824},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202832} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.893015] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545343013638144},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223052} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.893071] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223052} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.893835] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545343051386880},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223053} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.893901] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223053} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.895152] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:14126}, to_seq_no={branch:0, seq:9584}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202832} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.895305] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28] handle_sp_rollback_resp(tx_id={txid:392223054}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.895343] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223054}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.895350] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223054}, start_ts=1729061841895081, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223054}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841894105, active_ts:1729061841894105, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441894105, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332879897405, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841894104, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.900951] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 223.50; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 12743.11, iops: 8.98, rt: 4194.56; ) [2024-10-16 14:57:21.900977] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:21.901015] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 716.00; ) [2024-10-16 14:57:21.901024] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:21.901035] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:21.902948] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392223055}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.902998] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223055}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.903010] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223055}, start_ts=1729061841901298, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223055}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3182}, op_sn:3, alloc_ts:1729061841897217, active_ts:1729061841900396, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441900396, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008022213188, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841897216, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.908323] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513898421047296},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202835} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.909155] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3175}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202835} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.909419] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223056}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.909452] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223056}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.909460] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223056}, start_ts=1729061841909159, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223056}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1057}, op_sn:3, alloc_ts:1729061841907830, active_ts:1729061841908888, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441908888, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332893966363, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841907831, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.911948] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545343252713472},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223057} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.912023] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223057} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.917447] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223058}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.917523] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223058}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.917543] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223058}, start_ts=1729061841917168, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223058}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:4434}, op_sn:3, alloc_ts:1729061841911977, active_ts:1729061841916408, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441916408, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008041517942, first_scn:{branch:0, seq:2129}, last_scn:{branch:0, seq:2129}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841911976, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.919812] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:21.920002] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:21.920020] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=209) [2024-10-16 14:57:21.920884] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DD5-0-0] [lt=11] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:57:21.921232] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061841921231, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007618, add_bytes: 0 [2024-10-16 14:57:21.924851] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:21.932925] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223059}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.932986] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=53] all savepoint rollback succeed(tx.tx_id={txid:392223059}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.933000] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223059}, start_ts=1729061841932458, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223059}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:8435}, op_sn:3, alloc_ts:1729061841923174, active_ts:1729061841931611, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441931611, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008056742914, first_scn:{branch:0, seq:7379}, last_scn:{branch:0, seq:7379}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841923176, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.936434] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061841935843000, start_id=1729061841935843000, end_id=1729061841935843001) [2024-10-16 14:57:21.937658] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=29] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=1816, mem_used=4159488) [2024-10-16 14:57:21.937836] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061841936899000, start_id=1729061841936899000, end_id=1729061841936899001) [2024-10-16 14:57:21.938076] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061841561439001, v:0}, new_max_version={val:1729061841561439001, v:0}, last_min_version={val:1729061841396535000, v:0}, last_max_version={val:1729061841396535000, v:0}, total_time_us=2491, delta_us=376636) [2024-10-16 14:57:21.938091] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061841561439001, v:0}, min_version={val:1729061841561439001, v:0}, max_version={val:1729061841561439001, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:57:21.938905] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545343365959680},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223060} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.938986] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223060} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.940063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061841939222000, start_id=1729061841939222000, end_id=1729061841939222001) [2024-10-16 14:57:21.941118] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26] handle_sp_rollback_resp(tx_id={txid:392223061}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.941148] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223061}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.941156] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223061}, start_ts=1729061841940899, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223061}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841940245, active_ts:1729061841940245, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441940245, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332925652894, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841940244, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.942655] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=65, stat_interval=1000000, avg (count/cost)=202, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:21.942673] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=65, stat_interval=1000000, avg (count/cost)=67, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:21.942700] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=65, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:21.943124] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545343441457152},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223062} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.943193] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223062} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.944710] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223063}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.944752] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392223063}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.944764] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223063}, start_ts=1729061841944535, retries=0, savepoint={branch:0, seq:4}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223063}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:7}, op_sn:3, alloc_ts:1729061841943455, active_ts:1729061841944517, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441944517, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332929360934, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841943455, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.946953] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545343504371712},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223064} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.947016] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223064} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:21.948225] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223065}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.948274] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223065}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.948288] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223065}, start_ts=1729061841948014, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223065}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1066}, op_sn:3, alloc_ts:1729061841946691, active_ts:1729061841947764, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441947764, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332932838717, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841946698, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.954589] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:21.955383] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223066}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.955433] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223066}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.955448] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223066}, start_ts=1729061841955090, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223066}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1535}, op_sn:3, alloc_ts:1729061841954563, active_ts:1729061841954563, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441954563, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332939937187, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841953028, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.959685] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AC9-0-0] [lt=43] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:21.959714] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AC9-0-0] [lt=22] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:21.959799] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223067}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.959852] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223067}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.959865] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223067}, start_ts=1729061841959478, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223067}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1433}, op_sn:3, alloc_ts:1729061841957893, active_ts:1729061841959329, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441959329, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332944290870, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841957896, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.959925] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061841959925, bytes: 4446802296126, bw: 1.520542 MB/s, add_ts: 1000845, add_bytes: 1595751 [2024-10-16 14:57:21.962006] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.962447] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223068}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.962475] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223068}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.962484] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223068}, start_ts=1729061841962213, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223068}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841961468, active_ts:1729061841961468, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441961468, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008086523935, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841961467, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.964656] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=13][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5024, dropped:262, tid:12153}]) [2024-10-16 14:57:21.965462] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:21.971735] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:21.973519] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223069}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.973562] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223069}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.973614] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223069}, start_ts=1729061841968104, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223069}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061841967033, active_ts:1729061841967033, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441967033, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008092422197, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841967032, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.979424] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223070}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.980612] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=51] all savepoint rollback succeed(tx.tx_id={txid:392223070}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.980626] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223070}, start_ts=1729061841979215, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223070}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1062}, op_sn:3, alloc_ts:1729061841977748, active_ts:1729061841978809, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441978809, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008103472419, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841977747, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.983569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061841983043000, start_id=1729061841983043000, end_id=1729061841983043001) [2024-10-16 14:57:21.984985] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223071}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.985020] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392223071}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.985028] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223071}, start_ts=1729061841984754, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223071}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:8}, op_sn:3, alloc_ts:1729061841983043, active_ts:1729061841983043, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441983043, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332969442778, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841983042, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.985164] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106743][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D34-0-0] [lt=15] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:21.985177] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106743][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D34-0-0] [lt=9] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:21.988518] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061841983043000, start_id=1729061841983043000, end_id=1729061841983043001) [2024-10-16 14:57:21.991119] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [106984][][T1001][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1001, ret=0, thread_count_=172) [2024-10-16 14:57:21.991140] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [106984][][T1001][Y0-0000000000000000-0-0] [lt=19] Init thread local success [2024-10-16 14:57:21.991161] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [106984][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9383-0-0] [lt=4] init point thread id with(&point=0x55e3435c2880, idx_=488, point=[thread id=106984, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=106984) [2024-10-16 14:57:21.991366] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061841991088000, start_id=1729061841991088000, end_id=1729061841991088001) [2024-10-16 14:57:21.992742] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15CA-0-0] [lt=4] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:21.992912] INFO [CLOG] do_thread_task_ (ob_remote_fetch_log_worker.cpp:249) [21544][T1_RFLWorker][T1][YB4A0A000407-00061E996BC960C1-0-0] [lt=14] ObRemoteFetchWorker is running(thread_index=0) [2024-10-16 14:57:21.994158] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223072}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:21.994203] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392223072}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:21.994211] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223072}, start_ts=1729061841992811, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223072}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1670}, op_sn:3, alloc_ts:1729061841991088, active_ts:1729061841992760, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062441992760, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008116987203, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841991090, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:21.994440] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] no need to update min/max version(new_version={val:1729061841515837000, v:0}, min_version={val:1729061841515837000, v:0}, max_version={val:1729061841515837000, v:0}) [2024-10-16 14:57:21.994691] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=10] worker thread created(tenant_->id()=1001, group_id_=1, token=1) [2024-10-16 14:57:21.999748] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb23d60 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223073} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac75f0 trans_version={val:1729061781561821000, v:0} scn={val:1729061781561821000, v:0} tx_id={txid:392222452} prev=0x2b0fadac74b0 next=(nil) modify_count=1 acc_checksum=4154425104 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061841999027000, v:0}, *this={this=0x2b0fadac71a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac75f0 latest_compact_node=(nil) max_trans_version={val:1729061781561821000, v:0} max_trans_id=392222452 max_elr_trans_version={val:1729061781561821000, v:0} max_elr_trans_id=392222452 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781561821000, v:0} min_modify_scn={val:1729061781561821000, v:0}}) [2024-10-16 14:57:21.999787] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb23d60 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223073} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061841999027000, v:0}, tx_id:{txid:392223073}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223073}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223073}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061841999027, active_ts:1729061841999027, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061841999026, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:21.999844] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223073} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223073}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:21.999859] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":684545343856693248},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223073} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:21.999877] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:21.999883] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343856693248},{"BIGINT":1729058400},{"VARCHAR":"@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:21.999934] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:21.999973] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223073} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.000201] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=79, avg_log_batch_cnt=1, total_group_log_size=125271, avg_group_log_size=1585) [2024-10-16 14:57:22.000213] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=125271) [2024-10-16 14:57:22.000884] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ee3498 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223074} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f22ea69d8 trans_version={val:1729061781562879000, v:0} scn={val:1729061781562879000, v:0} tx_id={txid:392222453} prev=0x2b0f22ea6898 next=(nil) modify_count=1 acc_checksum=2036588164 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842000087001, v:0}, *this={this=0x2b0f22ea6590 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea69d8 latest_compact_node=(nil) max_trans_version={val:1729061781562879000, v:0} max_trans_id=392222453 max_elr_trans_version={val:1729061781562879000, v:0} max_elr_trans_id=392222453 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781562879000, v:0} min_modify_scn={val:1729061781562879000, v:0}}) [2024-10-16 14:57:22.000910] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ee3498 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223074} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209267}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842000087001, v:0}, tx_id:{txid:392223074}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223074}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223074}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842000087, active_ts:1729061842000087, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842000086, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.000958] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=47][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223074} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223074}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.000973] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":684545343907024896},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223074} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.000987] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.000991] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343907024896},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.001040] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.001063] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223074} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.002055] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46af98 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223075} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5}, prev=this=0x2b0f3f408e20 trans_version={val:1729061781563900000, v:0} scn={val:1729061781563900000, v:0} tx_id={txid:392222454} prev=0x2b0f3f408ce0 next=(nil) modify_count=1 acc_checksum=4010878372 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842001144001, v:0}, *this={this=0x2b0f3f4089d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f408e20 latest_compact_node=(nil) max_trans_version={val:1729061781563900000, v:0} max_trans_id=392222454 max_elr_trans_version={val:1729061781563900000, v:0} max_elr_trans_id=392222454 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781563900000, v:0} min_modify_scn={val:1729061781563900000, v:0}}) [2024-10-16 14:57:22.002083] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46af98 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223075} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842001144001, v:0}, tx_id:{txid:392223075}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223075}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223075}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842001144, active_ts:1729061842001144, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842001146, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.002124] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223075} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223075}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:5}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.002141] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":684545343944773632},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223075} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.002160] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.002165] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545343944773632},{"BIGINT":1729058400},{"VARCHAR":"@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.002202] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.002223] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223075} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.003415] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223076}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.003445] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223076}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.003460] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223076}, start_ts=1729061842003211, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223076}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842002202, active_ts:1729061842002202, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442002202, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332988017708, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842002204, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.005925] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223077}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.005969] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29] all savepoint rollback succeed(tx.tx_id={txid:392223077}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.005978] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223077}, start_ts=1729061842005696, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223077}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061842004322, active_ts:1729061842005385, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442005385, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012332990516683, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842004327, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.011183] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=151, stat_interval=1000000, avg (count/cost)=30, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:22.011201] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=151, stat_interval=1000000, avg (count/cost)=89, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:22.015954] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D50D9-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=78, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:22.015969] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D50D9-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=78, stat_interval=1000000, avg (count/cost)=4276, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:22.016624] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061842015915000, start_id=1729061842015915000, end_id=1729061842015915001) [2024-10-16 14:57:22.016646] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:22.016663] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061842016660], gts:1729061842015915001, latest_srr:[mts=1729061842016660]}) [2024-10-16 14:57:22.017445] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223078}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.017499] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29] all savepoint rollback succeed(tx.tx_id={txid:392223078}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.017508] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223078}, start_ts=1729061842017089, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223078}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:9446}, op_sn:3, alloc_ts:1729061842007504, active_ts:1729061842016949, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442016949, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008132801843, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842007503, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.021005] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223079}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.021052] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223079}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.021064] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223079}, start_ts=1729061842020836, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223079}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842020124, active_ts:1729061842020124, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442020124, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333005541606, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842020123, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.026423] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223080}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.026528] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223080}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.026540] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223080}, start_ts=1729061842025245, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223080}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1131}, op_sn:3, alloc_ts:1729061842023323, active_ts:1729061842024453, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442024453, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333009732093, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842023322, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.026635] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:22.026653] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:22.026975] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:22.027070] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1914) [2024-10-16 14:57:22.033802] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ee3b78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202846} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5989}, prev=this=0x2b0f22e8c320 trans_version={val:1729061779676329001, v:0} scn={val:1729061779676329001, v:0} tx_id={txid:392201810} prev=0x2b0f22e8c1e0 next=(nil) modify_count=1 acc_checksum=945749684 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:10130}, snapshot_version={val:1729061842032663000, v:0}, *this={this=0x2b0f22e8bed8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e8c320 latest_compact_node=(nil) max_trans_version={val:1729061779676329001, v:0} max_trans_id=392201810 max_elr_trans_version={val:1729061779676329001, v:0} max_elr_trans_id=392201810 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779676329001, v:0} min_modify_scn={val:1729061779676329001, v:0}}) [2024-10-16 14:57:22.033831] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ee3b78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202846} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5989}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606260, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606260, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842032663000, v:0}, tx_id:{txid:392202846}, scn:{branch:0, seq:5986}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202846}, tx_desc:{this:0x2b0ec37a6090, tx_id:{txid:392202846}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061842032663, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842026682, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:5989}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.033889] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202846} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202846}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:5989}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.033903] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513898672705536},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202846} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.033915] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.033920] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513898672705536},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.033965] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.035027] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] Cache replace map node details(ret=0, replace_node_count=25, replace_time=0, replace_start_pos=1226823, replace_num=31457) [2024-10-16 14:57:22.035044] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=25, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:22.035269] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8489}, to_seq_no={branch:0, seq:5987}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202846} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.039410] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061842038897000, start_id=1729061842038897000, end_id=1729061842038897001) [2024-10-16 14:57:22.039579] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223081}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.039629] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223081}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.039638] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223081}, start_ts=1729061842039190, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223081}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061842037839, active_ts:1729061842038897, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442038897, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008163453393, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842037839, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.040896] INFO [ARCHIVE] load_archive_round_attr (ob_archive_persist_mgr.cpp:323) [22246][T1002_ArcSrv][T1002][YB4A0A000407-00061E996F8960C1-0-0] [lt=10] no archive dest exist, just skip(ret=-4018, tenant_id_=1002) [2024-10-16 14:57:22.045697] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223082}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.045730] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=49] all savepoint rollback succeed(tx.tx_id={txid:392223082}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.045739] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223082}, start_ts=1729061842045448, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223082}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1018}, op_sn:3, alloc_ts:1729061842044213, active_ts:1729061842045230, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442045230, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008169332813, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842044212, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.050076] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46b678 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223083} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f409520 trans_version={val:1729061781581955000, v:0} scn={val:1729061781581955000, v:0} tx_id={txid:392222462} prev=0x2b0f3f4093e0 next=(nil) modify_count=1 acc_checksum=2356725117 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842049483001, v:0}, *this={this=0x2b0f3f4090d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f409520 latest_compact_node=(nil) max_trans_version={val:1729061781581955000, v:0} max_trans_id=392222462 max_elr_trans_version={val:1729061781581955000, v:0} max_elr_trans_id=392222462 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781581955000, v:0} min_modify_scn={val:1729061781581955000, v:0}}) [2024-10-16 14:57:22.050113] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46b678 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223083} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842049483001, v:0}, tx_id:{txid:392223083}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223083}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223083}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842049483, active_ts:1729061842049483, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842049482, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.050170] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223083} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223083}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.050188] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":684545344448090112},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223083} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.050202] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.050215] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344448090112},{"BIGINT":1729058400},{"VARCHAR":"Ag¦, ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.050267] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.050299] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223083} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.051488] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6df9088 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202849} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2119}, prev=this=0x2b0ec6d4ff68 trans_version={val:1729061779685738000, v:0} scn={val:1729061779685738000, v:0} tx_id={txid:392201813} prev=0x2b0ec6d4fe28 next=(nil) modify_count=1 acc_checksum=2724318507 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3168}, snapshot_version={val:1729061842049483000, v:0}, *this={this=0x2b0ec6d4fb20 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d4ff68 latest_compact_node=(nil) max_trans_version={val:1729061779685738000, v:0} max_trans_id=392201813 max_elr_trans_version={val:1729061779685738000, v:0} max_elr_trans_id=392201813 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779685738000, v:0} min_modify_scn={val:1729061779685738000, v:0}}) [2024-10-16 14:57:22.051523] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6df9088 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202849} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2119}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606566, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606566, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842049483000, v:0}, tx_id:{txid:392202849}, scn:{branch:0, seq:1061}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202849}, tx_desc:{this:0x2b0ec37a40f0, tx_id:{txid:392202849}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061842050545, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842048429, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2119}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.051586] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202849} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202849}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:2119}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.051607] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=0] primary key duplication conflict(key={"BIGINT":675513898760785920},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202849} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.051619] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.051624] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513898760785920},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.051667] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.052927] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] handle_sp_rollback_resp(tx_id={txid:392223084}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.052965] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223084}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.052975] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223084}, start_ts=1729061842052624, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223084}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1191}, op_sn:3, alloc_ts:1729061842050545, active_ts:1729061842051735, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442051735, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008176771509, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842050544, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.053932] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5425}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202849} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.056817] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25] handle_sp_rollback_resp(tx_id={txid:392223085}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.056950] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223085}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.056962] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223085}, start_ts=1729061842056499, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223085}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1065}, op_sn:3, alloc_ts:1729061842054911, active_ts:1729061842055975, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442055975, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008180844687, first_scn:{branch:0, seq:1064}, last_scn:{branch:0, seq:1064}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842054914, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.059727] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2ACB-0-0] [lt=8][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178260, log_task={header:{begin_lsn:{lsn:14538464252}, end_lsn:{lsn:14538464374}, log_id:115178260, min_scn:{val:1729061841625527003, v:0}, max_scn:{val:1729061841625527003, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538464130}, prev_proposal_id:23, committed_end_lsn:{lsn:14538464252}, data_checksum:1715074492, accum_checksum:2318110569, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061841959681, freeze_ts:1729061841959681, submit_ts:1729061841959709, flushed_ts:1729061841959901, gen_to_freeze cost time:0, gen_to_submit cost time:28, submit_to_flush cost time:192}, fs_cb_begin_ts=1729061842059720, log_life_time=100039) [2024-10-16 14:57:22.062029] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223086}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.062065] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.062078] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392223086}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.062088] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223086}, start_ts=1729061842061793, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223086}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1054}, op_sn:3, alloc_ts:1729061842059123, active_ts:1729061842061235, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442061235, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333045213360, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842059124, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.064286] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ecd3f5ab8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223087} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779088644 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ecd3b40b8 trans_version={val:1729061781595881000, v:0} scn={val:1729061781595881000, v:0} tx_id={txid:392222466} prev=0x2b0ecd3b3f78 next=(nil) modify_count=1 acc_checksum=2083663429 version=1729061779088644 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842063357002, v:0}, *this={this=0x2b0ecd3b3c70 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ecd3b40b8 latest_compact_node=(nil) max_trans_version={val:1729061781595881000, v:0} max_trans_id=392222466 max_elr_trans_version={val:1729061781595881000, v:0} max_elr_trans_id=392222466 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781595881000, v:0} min_modify_scn={val:1729061781595881000, v:0}}) [2024-10-16 14:57:22.064320] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ecd3f5ab8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223087} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779088644 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209270}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842063357002, v:0}, tx_id:{txid:392223087}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223087}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223087}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842063357, active_ts:1729061842063357, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842063360, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.064374] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223087} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223087}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779088644, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.064391] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":684545344695554049},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223087} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.064403] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.064419] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344695554049},{"BIGINT":1729058400},{"VARCHAR":"A¡! ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.064469] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.064496] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223087} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.066748] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223088}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.066785] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392223088}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.066792] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223088}, start_ts=1729061842066241, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223088}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1168}, op_sn:3, alloc_ts:1729061842064416, active_ts:1729061842065583, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442065583, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008190324721, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842064415, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.071775] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.074216] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223089}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.074258] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29] all savepoint rollback succeed(tx.tx_id={txid:392223089}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.074279] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223089}, start_ts=1729061842073347, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223089}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1068}, op_sn:3, alloc_ts:1729061842071910, active_ts:1729061842072977, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442072977, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008197599646, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842071909, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.077970] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=47][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46bd58 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223090} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779396338 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f409c20 trans_version={val:1729061781602205000, v:0} scn={val:1729061781602205000, v:0} tx_id={txid:392222469} prev=0x2b0f3f409ae0 next=(nil) modify_count=1 acc_checksum=3197255433 version=1729061779396338 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842077211000, v:0}, *this={this=0x2b0f3f4097d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f409c20 latest_compact_node=(nil) max_trans_version={val:1729061781602205000, v:0} max_trans_id=392222469 max_elr_trans_version={val:1729061781602205000, v:0} max_elr_trans_id=392222469 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781602205000, v:0} min_modify_scn={val:1729061781602205000, v:0}}) [2024-10-16 14:57:22.078008] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46bd58 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223090} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779396338 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209282}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842077211000, v:0}, tx_id:{txid:392223090}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223090}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223090}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842077211, active_ts:1729061842077211, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842077210, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.078074] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=65][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223090} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223090}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779396338, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.078091] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":684545344833966081},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223090} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.078108] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.078117] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344833966081},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.078192] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=45][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.078222] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223090} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.079248] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0e885b7eb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223091} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0e885720d8 trans_version={val:1729061781603374000, v:0} scn={val:1729061781603374000, v:0} tx_id={txid:392222470} prev=0x2b0e88571f98 next=(nil) modify_count=1 acc_checksum=1793081675 version=1729061779336102 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842078270001, v:0}, *this={this=0x2b0e88571c90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e885720d8 latest_compact_node=(nil) max_trans_version={val:1729061781603374000, v:0} max_trans_id=392222470 max_elr_trans_version={val:1729061781603374000, v:0} max_elr_trans_id=392222470 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781603374000, v:0} min_modify_scn={val:1729061781603374000, v:0}}) [2024-10-16 14:57:22.079282] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0e885b7eb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223091} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209276}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842078270001, v:0}, tx_id:{txid:392223091}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223091}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223091}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842078270, active_ts:1729061842078270, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842078269, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.079340] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223091} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223091}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779336102, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.079356] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":684545344859131905},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223091} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.079370] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.079375] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344859131905},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.079418] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.079442] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223091} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.081078] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223092}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.081110] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223092}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.081118] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223092}, start_ts=1729061842080840, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223092}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1073}, op_sn:3, alloc_ts:1729061842079330, active_ts:1729061842080401, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442080401, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333065375135, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842079329, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.083577] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1729061842083553000, start_id=1729061842083553000, end_id=1729061842083553001) [2024-10-16 14:57:22.084233] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223093}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.084260] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392223093}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.084269] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223093}, start_ts=1729061842084015, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223093}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1029}, op_sn:3, alloc_ts:1729061842082521, active_ts:1729061842083553, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442083553, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333068757004, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842082524, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.086253] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46c438 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223094} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f40a320 trans_version={val:1729061781609951001, v:0} scn={val:1729061781609951001, v:0} tx_id={txid:392222473} prev=0x2b0f3f40a1e0 next=(nil) modify_count=1 acc_checksum=505988221 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842085762000, v:0}, *this={this=0x2b0f3f409ed8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f40a320 latest_compact_node=(nil) max_trans_version={val:1729061781609951001, v:0} max_trans_id=392222473 max_elr_trans_version={val:1729061781609951001, v:0} max_elr_trans_id=392222473 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781609951001, v:0} min_modify_scn={val:1729061781609951001, v:0}}) [2024-10-16 14:57:22.086292] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46c438 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223094} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842085762000, v:0}, tx_id:{txid:392223094}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223094}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223094}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842085762, active_ts:1729061842085762, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842085761, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.086352] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223094} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223094}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.086367] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":684545344951406592},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223094} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.086378] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.086384] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344951406592},{"BIGINT":1729058400},{"VARCHAR":"@¼Ã", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.086429] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.086453] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223094} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.087617] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0eeda3fc58 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223095} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778320400 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f65df9c10 trans_version={val:1729061781612075000, v:0} scn={val:1729061781612075000, v:0} tx_id={txid:392222474} prev=0x2b0f65df9ad0 next=(nil) modify_count=1 acc_checksum=2492395580 version=1729061778320400 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842086820000, v:0}, *this={this=0x2b0f65df97c8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f65df9c10 latest_compact_node=(nil) max_trans_version={val:1729061781612075000, v:0} max_trans_id=392222474 max_elr_trans_version={val:1729061781612075000, v:0} max_elr_trans_id=392222474 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781612075000, v:0} min_modify_scn={val:1729061781612075000, v:0}}) [2024-10-16 14:57:22.087644] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0eeda3fc58 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223095} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778320400 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209264}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842086820000, v:0}, tx_id:{txid:392223095}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223095}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223095}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842086820, active_ts:1729061842086820, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842086819, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.087699] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223095} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223095}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778320400, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.087716] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":684545344972378113},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223095} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.087729] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.087734] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545344972378113},{"BIGINT":1729058400},{"VARCHAR":"@QÀ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.087778] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.087798] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223095} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.089428] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223096}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.089476] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223096}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.089490] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223096}, start_ts=1729061842089166, retries=0, savepoint={branch:0, seq:5}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223096}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842087876, active_ts:1729061842088934, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442088934, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008213501561, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842087875, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.092983] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223097}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.093042] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29] all savepoint rollback succeed(tx.tx_id={txid:392223097}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.093062] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223097}, start_ts=1729061842092632, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223097}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1154}, op_sn:3, alloc_ts:1729061842091019, active_ts:1729061842092180, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442092180, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008216941844, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842091026, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.093327] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061842093223000, start_id=1729061842093223000, end_id=1729061842093223001) [2024-10-16 14:57:22.095374] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061842094284000, start_id=1729061842094284000, end_id=1729061842094284001) [2024-10-16 14:57:22.098324] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223098}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.098387] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392223098}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.098402] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223098}, start_ts=1729061842098116, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223098}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842097282, active_ts:1729061842097282, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442097282, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333082790253, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842097281, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.104213] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] handle_sp_rollback_resp(tx_id={txid:392223099}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.104244] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=40] all savepoint rollback succeed(tx.tx_id={txid:392223099}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.104253] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223099}, start_ts=1729061842103977, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223099}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1063}, op_sn:3, alloc_ts:1729061842102490, active_ts:1729061842103552, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442103552, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008228286557, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842102489, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.108933] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223100}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.110105] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:22.110131] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=25][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:22.110275] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223100}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.110290] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223100}, start_ts=1729061842108161, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223100}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1071}, op_sn:3, alloc_ts:1729061842106731, active_ts:1729061842107801, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442107801, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008232514396, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842106730, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.111963] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb24440 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202857} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1065}, prev=this=0x2b0fadaa5540 trans_version={val:1729061779709255000, v:0} scn={val:1729061779709255000, v:0} tx_id={txid:392201821} prev=0x2b0fadaa5400 next=(nil) modify_count=1 acc_checksum=334922740 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2124}, snapshot_version={val:1729061842111243000, v:0}, *this={this=0x2b0fadaa50f8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaa5540 latest_compact_node=(nil) max_trans_version={val:1729061779709255000, v:0} max_trans_id=392201821 max_elr_trans_version={val:1729061779709255000, v:0} max_elr_trans_id=392201821 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779709255000, v:0} min_modify_scn={val:1729061779709255000, v:0}}) [2024-10-16 14:57:22.111995] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb24440 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202857} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1065}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606247, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606247, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842111243000, v:0}, tx_id:{txid:392202857}, scn:{branch:0, seq:1062}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202857}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392202857}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061842111243, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842110183, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1065}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.112063] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=67][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202857} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202857}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1065}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.112086] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=0] primary key duplication conflict(key={"BIGINT":675513898936946688},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202857} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.112098] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.112107] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513898936946688},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.112159] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.115620] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5325}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202857} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.116906] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] handle_sp_rollback_resp(tx_id={txid:392223101}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.116969] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392223101}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.116981] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223101}, start_ts=1729061842116690, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223101}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1063}, op_sn:3, alloc_ts:1729061842115507, active_ts:1729061842116570, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442116570, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333101277026, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842115507, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.123133] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223102}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.123197] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=40] all savepoint rollback succeed(tx.tx_id={txid:392223102}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.123208] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223102}, start_ts=1729061842120353, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223102}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1057}, op_sn:3, alloc_ts:1729061842118695, active_ts:1729061842119751, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442119751, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008244538449, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842118694, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.123346] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:22.125149] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1204781056, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:22.126367] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B92-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061842126108000, start_id=1729061842126108000, end_id=1729061842126108001) [2024-10-16 14:57:22.126462] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0B95-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061842126108000, start_id=1729061842126108000, end_id=1729061842126108001) [2024-10-16 14:57:22.126675] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1914, avg_group_log_size=191) [2024-10-16 14:57:22.126687] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1914) [2024-10-16 14:57:22.126720] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:22.126724] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:22.126788] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B93-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061842126108000, start_id=1729061842126108000, end_id=1729061842126108001) [2024-10-16 14:57:22.126903] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B96-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061842126108000, start_id=1729061842126108000, end_id=1729061842126108001) [2024-10-16 14:57:22.127124] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:22.127741] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B907-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061842127159000, start_id=1729061842127159000, end_id=1729061842127159001) [2024-10-16 14:57:22.127755] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=363, mem_used=2079744) [2024-10-16 14:57:22.130146] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27] handle_sp_rollback_resp(tx_id={txid:392223103}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.130201] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223103}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.130210] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223103}, start_ts=1729061842129885, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223103}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1060}, op_sn:3, alloc_ts:1729061842128222, active_ts:1729061842129281, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442129281, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333113754565, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842128221, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.130998] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C96-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510309999390, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410513, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:22.135580] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223104}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.135618] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223104}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.135628] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223104}, start_ts=1729061842135270, retries=0, savepoint={branch:0, seq:1057}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223104}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3172}, op_sn:3, alloc_ts:1729061842131398, active_ts:1729061842134575, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442134575, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008259570500, first_scn:{branch:0, seq:1059}, last_scn:{branch:0, seq:1059}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842131403, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.136539] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061842135615000, start_id=1729061842135615000, end_id=1729061842135615001) [2024-10-16 14:57:22.137642] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10327, avg_group_log_size=543) [2024-10-16 14:57:22.137655] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10327) [2024-10-16 14:57:22.137783] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061842136673000, start_id=1729061842136673000, end_id=1729061842136673001) [2024-10-16 14:57:22.137983] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=403, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:22.137997] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10327) [2024-10-16 14:57:22.138005] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=62, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:22.138031] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10205) [2024-10-16 14:57:22.138066] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:22.138611] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCFE8-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:22.138620] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCFE8-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=912, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:22.138624] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CCFE8-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=30, avg_log_gen_to_submit_time=58, avg_log_submit_to_flush_time=537, avg_log_submit_to_slide_time=853) [2024-10-16 14:57:22.140104] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061842139047000, start_id=1729061842139047000, end_id=1729061842139047001) [2024-10-16 14:57:22.141735] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061842141166000, start_id=1729061842141166001, end_id=1729061842141166002) [2024-10-16 14:57:22.142647] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] handle_sp_rollback_resp(tx_id={txid:392223105}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.142677] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223105}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.142686] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223105}, start_ts=1729061842142427, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223105}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061842141166, active_ts:1729061842142226, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442142226, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333127043483, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842141165, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.145322] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223106}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.145350] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223106}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.145358] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223106}, start_ts=1729061842145115, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223106}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842144349, active_ts:1729061842144349, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442144349, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333129824092, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842144348, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.147892] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223107}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.147945] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223107}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.147953] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223107}, start_ts=1729061842147678, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223107}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842146471, active_ts:1729061842147531, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442147531, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333132394866, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842146472, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.151616] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223108}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.151647] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30] all savepoint rollback succeed(tx.tx_id={txid:392223108}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.151656] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223108}, start_ts=1729061842150947, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223108}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842149672, active_ts:1729061842150730, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442150730, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008275297855, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842149671, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.156242] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb24b20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223109} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac7cf0 trans_version={val:1729061781654456000, v:0} scn={val:1729061781654456000, v:0} tx_id={txid:392222488} prev=0x2b0fadac7bb0 next=(nil) modify_count=1 acc_checksum=3180664313 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2139}, snapshot_version={val:1729061842156039000, v:0}, *this={this=0x2b0fadac78a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac7cf0 latest_compact_node=(nil) max_trans_version={val:1729061781654456000, v:0} max_trans_id=392222488 max_elr_trans_version={val:1729061781654456000, v:0} max_elr_trans_id=392222488 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781654456000, v:0} min_modify_scn={val:1729061781654456000, v:0}}) [2024-10-16 14:57:22.156276] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb24b20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223109} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842156039000, v:0}, tx_id:{txid:392223109}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223109}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223109}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842156039, active_ts:1729061842156039, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842156038, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.156332] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223109} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223109}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.156349] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":684618578354884608},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223109} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.156364] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.156370] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618578354884608},{"BIGINT":1729058400},{"VARCHAR":"A·v¸ø", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.156426] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.156453] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223109} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.157446] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6dfa388 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223110} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d7e280 trans_version={val:1729061781655515000, v:0} scn={val:1729061781655515000, v:0} tx_id={txid:392222489} prev=0x2b0ec6d7e140 next=(nil) modify_count=1 acc_checksum=3194750917 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842157068000, v:0}, *this={this=0x2b0ec6d7de38 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d7e280 latest_compact_node=(nil) max_trans_version={val:1729061781655515000, v:0} max_trans_id=392222489 max_elr_trans_version={val:1729061781655515000, v:0} max_elr_trans_id=392222489 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781655515000, v:0} min_modify_scn={val:1729061781655515000, v:0}}) [2024-10-16 14:57:22.157478] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6dfa388 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223110} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842157068000, v:0}, tx_id:{txid:392223110}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223110}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223110}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842157068, active_ts:1729061842157068, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842157067, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.157566] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=88][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223110} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223110}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.157584] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":684618578682040320},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223110} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.157611] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.157617] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618578682040320},{"BIGINT":1729058400},{"VARCHAR":"AÙ™™", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.157660] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.157684] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223110} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.158430] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f533f80 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223111} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f7f4cbe00 trans_version={val:1729061781656578001, v:0} scn={val:1729061781656578001, v:0} tx_id={txid:392222490} prev=0x2b0f7f4cbcc0 next=(nil) modify_count=1 acc_checksum=2565126027 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842158109000, v:0}, *this={this=0x2b0f7f4cb9b8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4cbe00 latest_compact_node=(nil) max_trans_version={val:1729061781656578001, v:0} max_trans_id=392222490 max_elr_trans_version={val:1729061781656578001, v:0} max_elr_trans_id=392222490 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781656578001, v:0} min_modify_scn={val:1729061781656578001, v:0}}) [2024-10-16 14:57:22.158487] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f533f80 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223111} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209285}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842158109000, v:0}, tx_id:{txid:392223111}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223111}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223111}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842158109, active_ts:1729061842158109, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842158108, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.158532] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=44][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223111} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223111}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.158549] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":684618578895949824},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223111} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.158558] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.158563] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618578895949824},{"BIGINT":1729058400},{"VARCHAR":"Að", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.158609] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.158632] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223111} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.160318] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223112}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.160379] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392223112}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.160387] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223112}, start_ts=1729061842159876, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223112}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842159174, active_ts:1729061842159174, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442159174, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333144525361, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842159173, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.162089] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.164439] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392223113}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.164478] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27] all savepoint rollback succeed(tx.tx_id={txid:392223113}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.164492] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223113}, start_ts=1729061842164222, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223113}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842163470, active_ts:1729061842163470, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442163470, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333148889385, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842163469, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.166076] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5018-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:22.166097] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5018-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:22.167107] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFFD-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061842157729, keepalive_ts:-1}) [2024-10-16 14:57:22.167128] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFFD-0-0] [lt=20] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061842157729, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:22.167144] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFFC-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061842157684, keepalive_ts:-1}) [2024-10-16 14:57:22.167148] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFFC-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061842157684, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:22.167841] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=42][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb25200 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223114} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac83f0 trans_version={val:1729061781661876001, v:0} scn={val:1729061781661876001, v:0} tx_id={txid:392222493} prev=0x2b0fadac82b0 next=(nil) modify_count=1 acc_checksum=1217394550 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842167743000, v:0}, *this={this=0x2b0fadac7fa8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac83f0 latest_compact_node=(nil) max_trans_version={val:1729061781661876001, v:0} max_trans_id=392222493 max_elr_trans_version={val:1729061781661876001, v:0} max_elr_trans_id=392222493 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781661876001, v:0} min_modify_scn={val:1729061781661876001, v:0}}) [2024-10-16 14:57:22.167872] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb25200 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223114} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842167743000, v:0}, tx_id:{txid:392223114}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223114}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223114}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842166685, active_ts:1729061842167743, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842166688, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.167931] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223114} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223114}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.167949] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":684618579424432128},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223114} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.167963] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.167969] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618579424432128},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.168021] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.168055] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223114} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.171841] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.174785] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] handle_sp_rollback_resp(tx_id={txid:392223115}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.174843] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392223115}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.174857] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223115}, start_ts=1729061842172376, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223115}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:4203}, op_sn:3, alloc_ts:1729061842167743, active_ts:1729061842171945, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442171945, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008293693849, first_scn:{branch:0, seq:1061}, last_scn:{branch:0, seq:1061}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842167742, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.179666] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] handle_sp_rollback_resp(tx_id={txid:392223116}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.179697] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223116}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.179706] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223116}, start_ts=1729061842179457, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223116}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061842178297, active_ts:1729061842179354, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442179354, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333164088349, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842178296, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.181941] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6dfaa68 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223117} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d7e980 trans_version={val:1729061781668240000, v:0} scn={val:1729061781668240000, v:0} tx_id={txid:392222496} prev=0x2b0ec6d7e840 next=(nil) modify_count=1 acc_checksum=3993039107 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842181476000, v:0}, *this={this=0x2b0ec6d7e538 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d7e980 latest_compact_node=(nil) max_trans_version={val:1729061781668240000, v:0} max_trans_id=392222496 max_elr_trans_version={val:1729061781668240000, v:0} max_elr_trans_id=392222496 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781668240000, v:0} min_modify_scn={val:1729061781668240000, v:0}}) [2024-10-16 14:57:22.181973] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6dfaa68 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223117} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842181476000, v:0}, tx_id:{txid:392223117}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223117}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223117}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842181476, active_ts:1729061842181476, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842181475, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.182028] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223117} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223117}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.182047] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":684618580066160640},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223117} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.182062] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.182068] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618580066160640},{"BIGINT":1729058400},{"VARCHAR":"A¦²t", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.182120] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.182149] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223117} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.183168] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb258e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223118} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6}, prev=this=0x2b0fadac8af0 trans_version={val:1729061781668240002, v:0} scn={val:1729061781668240002, v:0} tx_id={txid:392222497} prev=0x2b0fadac89b0 next=(nil) modify_count=1 acc_checksum=3892665093 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842182531000, v:0}, *this={this=0x2b0fadac86a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac8af0 latest_compact_node=(nil) max_trans_version={val:1729061781668240002, v:0} max_trans_id=392222497 max_elr_trans_version={val:1729061781668240002, v:0} max_elr_trans_id=392222497 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781668240002, v:0} min_modify_scn={val:1729061781668240002, v:0}}) [2024-10-16 14:57:22.183196] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb258e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223118} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842182531000, v:0}, tx_id:{txid:392223118}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223118}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223118}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842182531, active_ts:1729061842182531, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842182530, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:6}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.183242] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=45][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223118} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223118}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:6}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.183258] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":684618580179406848},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223118} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.183270] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.183275] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618580179406848},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.183319] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.183340] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223118} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.183991] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061842183640000, start_id=1729061842183640000, end_id=1729061842183640001) [2024-10-16 14:57:22.185455] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223119}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.185500] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223119}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.185513] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223119}, start_ts=1729061842185230, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223119}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1060}, op_sn:3, alloc_ts:1729061842183640, active_ts:1729061842184699, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442184699, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008309347065, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842183639, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.188767] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061842187867000, start_id=1729061842187867000, end_id=1729061842187867001) [2024-10-16 14:57:22.190614] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223120}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.190660] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30] all savepoint rollback succeed(tx.tx_id={txid:392223120}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.190674] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223120}, start_ts=1729061842190409, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223120}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1049}, op_sn:3, alloc_ts:1729061842188934, active_ts:1729061842189986, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442189986, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008314626535, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842188937, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.192791] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=154268) [2024-10-16 14:57:22.194247] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223121}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.194282] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223121}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.194290] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223121}, start_ts=1729061842194030, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223121}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842193164, active_ts:1729061842193164, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442193164, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333178694298, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842193163, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.194722] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb25fc0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202860} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2118}, prev=this=0x2b0fadaa5c40 trans_version={val:1729061779716684001, v:0} scn={val:1729061779716684001, v:0} tx_id={txid:392201824} prev=0x2b0fadaa5b00 next=(nil) modify_count=1 acc_checksum=729386628 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2100}, snapshot_version={val:1729061842193164001, v:0}, *this={this=0x2b0fadaa57f8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaa5c40 latest_compact_node=(nil) max_trans_version={val:1729061779716684001, v:0} max_trans_id=392201824 max_elr_trans_version={val:1729061779716684001, v:0} max_elr_trans_id=392201824 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779716684001, v:0} min_modify_scn={val:1729061779716684001, v:0}}) [2024-10-16 14:57:22.194749] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb25fc0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202860} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2118}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606303, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606303, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842193164001, v:0}, tx_id:{txid:392202860}, scn:{branch:0, seq:1060}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202860}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392202860}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061842194224, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842192107, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2118}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.194797] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=47][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202860} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202860}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:2118}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.194815] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513898999861248},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202860} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.194829] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.194834] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513898999861248},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.194889] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.195533] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3176}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202860} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.196300] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:22.196424] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D81-0-0] [lt=12] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061857008855000, latest_log_ts:{val:1729061840488170001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:22.196479] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D81-0-0] [lt=51] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:22.196768] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061842196339000, start_id=1729061842196339000, end_id=1729061842196339001) [2024-10-16 14:57:22.197748] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46cb18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223122} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f40aa20 trans_version={val:1729061781677984000, v:0} scn={val:1729061781677984000, v:0} tx_id={txid:392222501} prev=0x2b0f3f40a8e0 next=(nil) modify_count=1 acc_checksum=3478581803 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842197418000, v:0}, *this={this=0x2b0f3f40a5d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f40aa20 latest_compact_node=(nil) max_trans_version={val:1729061781677984000, v:0} max_trans_id=392222501 max_elr_trans_version={val:1729061781677984000, v:0} max_elr_trans_id=392222501 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781677984000, v:0} min_modify_scn={val:1729061781677984000, v:0}}) [2024-10-16 14:57:22.197780] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46cb18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223122} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842197418000, v:0}, tx_id:{txid:392223122}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223122}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223122}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842197418, active_ts:1729061842197418, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842197417, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.197835] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223122} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223122}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.197852] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":684618580833718272},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223122} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.197866] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.197871] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618580833718272},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.197946] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.197970] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223122} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.200804] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223123}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.200868] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392223123}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.200879] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223123}, start_ts=1729061842200613, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223123}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842198478, active_ts:1729061842199536, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442199536, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008324780843, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842198477, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.204687] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46d1f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202861} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2186}, prev=this=0x2b0e5abee9f0 trans_version={val:1729061779720452000, v:0} scn={val:1729061779720452000, v:0} tx_id={txid:392201825} prev=0x2b0e5abee8b0 next=(nil) modify_count=1 acc_checksum=460878498 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1070}, snapshot_version={val:1729061842203786000, v:0}, *this={this=0x2b0e5abee5a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abee9f0 latest_compact_node=(nil) max_trans_version={val:1729061779720452000, v:0} max_trans_id=392201825 max_elr_trans_version={val:1729061779720452000, v:0} max_elr_trans_id=392201825 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779720452000, v:0} min_modify_scn={val:1729061779720452000, v:0}}) [2024-10-16 14:57:22.204717] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46d1f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202861} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2186}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606281, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606281, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842203786000, v:0}, tx_id:{txid:392202861}, scn:{branch:0, seq:2183}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202861}, tx_desc:{this:0x2b0ec37cf8b0, tx_id:{txid:392202861}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061842203786, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842201605, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2186}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.204766] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=48][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202861} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202861}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:2186}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.204781] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513899025027072},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202861} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.204792] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.204797] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513899025027072},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.204845] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.207116] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5403}, to_seq_no={branch:0, seq:2184}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202861} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.209397] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] handle_sp_rollback_resp(tx_id={txid:392223124}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.209447] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29] all savepoint rollback succeed(tx.tx_id={txid:392223124}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.209455] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223124}, start_ts=1729061842208755, retries=0, savepoint={branch:0, seq:1119}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223124}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1122}, op_sn:3, alloc_ts:1729061842207007, active_ts:1729061842208128, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442208128, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008332974167, first_scn:{branch:0, seq:1121}, last_scn:{branch:0, seq:1121}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842207011, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.210973] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5873-0-0] [lt=16] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:22.211066] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5873-0-0] [lt=12][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:22.211198] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=3] Garbage Collector is running(seq_=656697, gc_interval=10000000) [2024-10-16 14:57:22.211217] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=8] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:22.211227] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=22) [2024-10-16 14:57:22.211233] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:22.211952] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:22.211969] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=13] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:22.211983] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=14] execute_gc cost time(ret=0, time_us=15) [2024-10-16 14:57:22.211987] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:22.216063] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223125}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.216099] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29] all savepoint rollback succeed(tx.tx_id={txid:392223125}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.216128] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223125}, start_ts=1729061842215748, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223125}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1064}, op_sn:3, alloc_ts:1729061842214483, active_ts:1729061842215548, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442215548, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333200381114, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842214484, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.217479] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=151011) [2024-10-16 14:57:22.217772] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ee4258 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202862} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1064}, prev=this=0x2b0f22e8ca20 trans_version={val:1729061779722634001, v:0} scn={val:1729061779722634001, v:0} tx_id={txid:392201826} prev=0x2b0f22e8c8e0 next=(nil) modify_count=1 acc_checksum=2311696491 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1114}, snapshot_version={val:1729061842216607000, v:0}, *this={this=0x2b0f22e8c5d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22e8ca20 latest_compact_node=(nil) max_trans_version={val:1729061779722634001, v:0} max_trans_id=392201826 max_elr_trans_version={val:1729061779722634001, v:0} max_elr_trans_id=392201826 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779722634001, v:0} min_modify_scn={val:1729061779722634001, v:0}}) [2024-10-16 14:57:22.217804] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ee4258 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202862} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1064}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606525, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606525, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842216607000, v:0}, tx_id:{txid:392202862}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202862}, tx_desc:{this:0x2b0ec37b7d30, tx_id:{txid:392202862}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061842217670, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842216607, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1064}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.217861] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202862} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202862}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:1064}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.217876] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513899050192896},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202862} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.217889] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.217895] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513899050192896},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.217946] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.218024] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=151010) [2024-10-16 14:57:22.219965] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3184}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202862} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.221729] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30] handle_sp_rollback_resp(tx_id={txid:392223126}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.221767] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=42] all savepoint rollback succeed(tx.tx_id={txid:392223126}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.221777] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223126}, start_ts=1729061842220607, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223126}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2122}, op_sn:3, alloc_ts:1729061842217670, active_ts:1729061842219790, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442219790, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008344858097, first_scn:{branch:0, seq:2121}, last_scn:{branch:0, seq:2121}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842217672, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.226949] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=253, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:22.226964] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=49, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:22.226968] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:22.227218] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFCA-0-0] [lt=26] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:22.227225] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFCA-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2478, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:22.227282] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFC9-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:22.227286] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFC9-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2407, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:22.227290] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFC9-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=359, avg_log_submit_to_slide_time=2392) [2024-10-16 14:57:22.230515] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223127}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.230566] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392223127}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.230581] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223127}, start_ts=1729061842230287, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223127}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1056}, op_sn:3, alloc_ts:1729061842228532, active_ts:1729061842229591, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442229591, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333214828753, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842228535, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.233591] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223128}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.236487] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=17, replace_time=0, replace_start_pos=1258280, replace_num=31457) [2024-10-16 14:57:22.236499] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=17, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:22.238638] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=42] all savepoint rollback succeed(tx.tx_id={txid:392223128}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.238652] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223128}, start_ts=1729061842233329, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223128}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1172}, op_sn:3, alloc_ts:1729061842231715, active_ts:1729061842232892, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442232892, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333217987356, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842231722, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.240838] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb266a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223129} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac91f0 trans_version={val:1729061781697685001, v:0} scn={val:1729061781697685001, v:0} tx_id={txid:392222508} prev=0x2b0fadac90b0 next=(nil) modify_count=1 acc_checksum=2686765233 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842239746001, v:0}, *this={this=0x2b0fadac8da8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac91f0 latest_compact_node=(nil) max_trans_version={val:1729061781697685001, v:0} max_trans_id=392222508 max_elr_trans_version={val:1729061781697685001, v:0} max_elr_trans_id=392222508 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781697685001, v:0} min_modify_scn={val:1729061781697685001, v:0}}) [2024-10-16 14:57:22.240868] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb266a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223129} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842239746001, v:0}, tx_id:{txid:392223129}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223129}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223129}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842239746, active_ts:1729061842239746, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842239749, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.240946] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=77][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223129} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223129}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.240969] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22][errcode=0] primary key duplication conflict(key={"BIGINT":684618582318501888},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223129} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.240984] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.240990] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618582318501888},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.241041] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.241071] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1133}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223129} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.241677] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3FAA-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=90, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:22.241687] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3FAA-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=90, stat_interval=1000000, avg (count/cost)=787, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:22.244104] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223130}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.244147] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223130}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.244157] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223130}, start_ts=1729061842242227, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223130}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1056}, op_sn:3, alloc_ts:1729061842240881, active_ts:1729061842241941, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442241941, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008366591488, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842240885, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.250167] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46d8d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223131} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f40b120 trans_version={val:1729061781700929001, v:0} scn={val:1729061781700929001, v:0} tx_id={txid:392222510} prev=0x2b0f3f40afe0 next=(nil) modify_count=1 acc_checksum=2657801922 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842249290000, v:0}, *this={this=0x2b0f3f40acd8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f40b120 latest_compact_node=(nil) max_trans_version={val:1729061781700929001, v:0} max_trans_id=392222510 max_elr_trans_version={val:1729061781700929001, v:0} max_elr_trans_id=392222510 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781700929001, v:0} min_modify_scn={val:1729061781700929001, v:0}}) [2024-10-16 14:57:22.250204] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46d8d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223131} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842249290000, v:0}, tx_id:{txid:392223131}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223131}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223131}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842249290, active_ts:1729061842249290, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842249291, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.250258] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223131} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223131}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.250276] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":684618582658240512},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223131} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.250290] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.250296] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618582658240512},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.250343] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.250368] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223131} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.251542] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD04-0-0] [lt=5][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:22.252757] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24] handle_sp_rollback_resp(tx_id={txid:392223132}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.252788] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392223132}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.252800] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223132}, start_ts=1729061842252542, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223132}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:8}, op_sn:3, alloc_ts:1729061842251632, active_ts:1729061842251632, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442251632, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333237132228, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842251631, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.256445] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46dfb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223133} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f40b820 trans_version={val:1729061781705216000, v:0} scn={val:1729061781705216000, v:0} tx_id={txid:392222512} prev=0x2b0f3f40b6e0 next=(nil) modify_count=1 acc_checksum=2086068994 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842256322000, v:0}, *this={this=0x2b0f3f40b3d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f40b820 latest_compact_node=(nil) max_trans_version={val:1729061781705216000, v:0} max_trans_id=392222512 max_elr_trans_version={val:1729061781705216000, v:0} max_elr_trans_id=392222512 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781705216000, v:0} min_modify_scn={val:1729061781705216000, v:0}}) [2024-10-16 14:57:22.256488] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46dfb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223133} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842256322000, v:0}, tx_id:{txid:392223133}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223133}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223133}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842255264, active_ts:1729061842256322, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842255265, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.256540] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=52][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223133} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223133}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.256557] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":684618582909898752},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223133} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.256572] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.256585] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618582909898752},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.256641] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.256675] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223133} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.258805] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223134}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.258856] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392223134}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.258869] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223134}, start_ts=1729061842258608, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223134}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1022}, op_sn:3, alloc_ts:1729061842257388, active_ts:1729061842258409, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442258409, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333243134510, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842257387, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.261464] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392223135}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.261522] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223135}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.261531] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223135}, start_ts=1729061842261225, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223135}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842260507, active_ts:1729061842260507, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442260507, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333245871772, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842260506, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.262149] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.264441] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223136}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.264484] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29] all savepoint rollback succeed(tx.tx_id={txid:392223136}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.264492] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223136}, start_ts=1729061842264228, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223136}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061842262614, active_ts:1729061842263676, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442263676, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333248474491, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842262615, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.266573] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=28] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:22.266591] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:22.266770] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=332, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:22.266781] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=845) [2024-10-16 14:57:22.266790] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:22.266798] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=845) [2024-10-16 14:57:22.266810] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:22.267013] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3FAE-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:22.267021] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3FAE-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=917, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:22.268896] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223137}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.268956] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223137}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.268970] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223137}, start_ts=1729061842268558, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223137}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2117}, op_sn:3, alloc_ts:1729061842265797, active_ts:1729061842267913, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442267913, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008391609008, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842265796, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.270557] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B0-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:22.270581] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B0-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061842127159000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061842127159000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:22.271886] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9B0-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061842271045000, start_id=1729061842271045000, end_id=1729061842271045001) [2024-10-16 14:57:22.271907] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.275057] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f534f78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202866} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3860}, prev=this=0x2b0f7f4a8f08 trans_version={val:1729061779746022000, v:0} scn={val:1729061779746022000, v:0} tx_id={txid:392201830} prev=0x2b0f7f4a8dc8 next=(nil) modify_count=1 acc_checksum=602728125 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:9229}, snapshot_version={val:1729061842273851001, v:0}, *this={this=0x2b0f7f4a8ac0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4a8f08 latest_compact_node=(nil) max_trans_version={val:1729061779746022000, v:0} max_trans_id=392201830 max_elr_trans_version={val:1729061779746022000, v:0} max_elr_trans_id=392201830 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779746022000, v:0} min_modify_scn={val:1729061779746022000, v:0}}) [2024-10-16 14:57:22.275099] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=42][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f534f78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202866} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3860}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606314, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606314, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842273851001, v:0}, tx_id:{txid:392202866}, scn:{branch:0, seq:2801}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202866}, tx_desc:{this:0x2b0ec37abf70, tx_id:{txid:392202866}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061842274912, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842271053, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:3860}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.275155] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202866} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202866}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:3860}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.275170] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513899100524544},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202866} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.275182] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.275188] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513899100524544},{"BIGINT":1729058400},{"VARCHAR":"ArWõÐ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.275237] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.275565] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3863}, to_seq_no={branch:0, seq:2802}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202866} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.276977] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB92-0-0] [lt=13] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12239, add_timestamp:1729061842276964}) [2024-10-16 14:57:22.277011] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E14-0-0] [lt=17] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12239, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061842276993, sql_renew_time:1729061842276993, is_mark_fail:false}]) [2024-10-16 14:57:22.277030] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E14-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:12239, add_timestamp:1729061842276964}, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061842276993, sql_renew_time:1729061842276993, is_mark_fail:false}]) [2024-10-16 14:57:22.277305] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D33-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061842268522, keepalive_ts:-1}) [2024-10-16 14:57:22.277315] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D33-0-0] [lt=9] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061842268522, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:22.277367] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D32-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061842268477, keepalive_ts:-1}) [2024-10-16 14:57:22.277372] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D32-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061842268477, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:22.282478] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21668][T1_L0_G0][T1][YB4A0A000407-00061E9B3769AB92-0-0] [lt=10][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:22.283408] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9B0-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061842282739000, start_id=1729061842282739000, end_id=1729061842282739001) [2024-10-16 14:57:22.283951] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223138}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.284235] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392223138}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.284252] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223138}, start_ts=1729061842283663, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223138}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6770}, op_sn:3, alloc_ts:1729061842275971, active_ts:1729061842282739, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442282739, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333268218980, first_scn:{branch:0, seq:8}, last_scn:{branch:0, seq:8}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842275971, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.284449] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=12] ls blacklist refresh finish(cost_time=8086) [2024-10-16 14:57:22.284540] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:22.285335] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061842284896000, start_id=1729061842284896000, end_id=1729061842284896001) [2024-10-16 14:57:22.286921] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=45] get number(ret=0, service_type_=0, range=1, base_id=1729061842285994000, start_id=1729061842285994001, end_id=1729061842285994002) [2024-10-16 14:57:22.287093] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106820][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D37-0-0] [lt=15] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:22.287117] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106820][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D37-0-0] [lt=20] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:22.287713] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223139}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.287747] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392223139}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.287757] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223139}, start_ts=1729061842287395, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223139}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1072}, op_sn:3, alloc_ts:1729061842285994, active_ts:1729061842287070, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442287070, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333272169173, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842285998, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.291314] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183785692, max_lsn:114618321994, max_scn:{val:1729061842187867000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45685691230}, reuse_lsn:{lsn:114618321994}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114576378954}}, last_submit_lsn:{lsn:114618321402}, last_submit_end_lsn:{lsn:114618321994}, last_submit_log_id:183785692, last_submit_log_pid:33705, max_flushed_lsn:{lsn:114618321402}, max_flushed_end_lsn:{lsn:114618321994}, max_flushed_log_pid:33705, committed_end_lsn:{lsn:114618321994}, last_slide_log_id:183785692, last_slide_scn:{val:1729061842187867000, v:0}, last_slide_lsn:{lsn:114618321402}, last_slide_end_lsn:{lsn:114618321994}, last_slide_log_pid:33705, last_slide_log_accum_checksum:1420139520, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0df99e2530}, [StateMgr]={this:0x2b0df99e6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, leader:"10.0.4.7:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183785693, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261592457, palf_role_change_cb:0x2b0df99ed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, reconfig_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, persistent_config_version:{proposal_id:33705, config_seq:14}, ms_ack_list:0, resend_config_version:{proposal_id:33705, config_seq:14}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258289572, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258289572, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0df99e3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261792660, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729029056668344001, v:0}, min_block_id:1702, min_block_min_scn:{val:1729022447183332001, v:0}, base_lsn_for_block_gc:{lsn:114547838976}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, log_config_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, log_snapshot_meta:{version:1, base_lsn:{lsn:114547838976}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:233218048}, readable_log_tail:{lsn:233218048}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114682048512}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/meta", dir_fd:146, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:35201024, block_header_serialize_buf_:0x2b0df99e89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114618321994}, readable_log_tail:{lsn:114618321994}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114614943744}, min_scn:{val:1729061506448191001, v:0}, curr_block_id:1708, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/log", dir_fd:148, min_block_id:1702, max_block_id:1709, curr_writable_block_id:1708}, logical_block_size_:67104768, curr_block_writable_size_:63726518, block_header_serialize_buf_:0x2b0df99ea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261592575, this:0x2b0df99e6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33705, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33704, majority_max_lsn:{lsn:45686897415}, saved_end_lsn:{lsn:45686897415}, last_submit_prepare_req_time_us:1722494261592686, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0df99e69b0}) [2024-10-16 14:57:22.291529] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=211] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061842189060, last_flushed_end_lsn:{lsn:114618321994}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061842189246, last_flushed_end_lsn:{lsn:114618321994}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061842189864, last_flushed_end_lsn:{lsn:114618321994}}]) [2024-10-16 14:57:22.291552] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=20] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183785693, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061842189060, last_flushed_end_lsn:{lsn:114618321994}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061842189246, last_flushed_end_lsn:{lsn:114618321994}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061842189864, last_flushed_end_lsn:{lsn:114618321994}}]) [2024-10-16 14:57:22.295873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061842294817000, start_id=1729061842294817000, end_id=1729061842294817001) [2024-10-16 14:57:22.295975] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=35] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3545, avg_group_log_size=354) [2024-10-16 14:57:22.296058] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223140}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.296141] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=42] all savepoint rollback succeed(tx.tx_id={txid:392223140}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.296163] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223140}, start_ts=1729061842295686, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223140}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2121}, op_sn:3, alloc_ts:1729061842293757, active_ts:1729061842294817, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442294817, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008419599570, first_scn:{branch:0, seq:1060}, last_scn:{branch:0, seq:1060}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842292699, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.296450] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:22.296568] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F58-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=636, avg_log_submit_to_slide_time=948) [2024-10-16 14:57:22.300302] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061842300115000, start_id=1729061842300115000, end_id=1729061842300115001) [2024-10-16 14:57:22.302813] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223141}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.302861] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223141}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.302870] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223141}, start_ts=1729061842302522, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223141}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1081}, op_sn:3, alloc_ts:1729061842301175, active_ts:1729061842302257, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442302257, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333287311983, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842301176, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.309393] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392223142}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.309437] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41] all savepoint rollback succeed(tx.tx_id={txid:392223142}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.309457] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223142}, start_ts=1729061842308949, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223142}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842307555, active_ts:1729061842308613, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442308613, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008432996863, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842307554, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.309698] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=5558, mem_used=10398720) [2024-10-16 14:57:22.310908] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:22.310927] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:22.310947] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373511087, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:22.311279] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:22.311302] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=17] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53593280, tenant_id=1003) [2024-10-16 14:57:22.311321] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=14] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:22.316905] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=83][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46e698 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223143} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f40bf20 trans_version={val:1729061781730635001, v:0} scn={val:1729061781730635001, v:0} tx_id={txid:392222522} prev=0x2b0f3f40bde0 next=(nil) modify_count=1 acc_checksum=2629458911 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842316115000, v:0}, *this={this=0x2b0f3f40bad8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f40bf20 latest_compact_node=(nil) max_trans_version={val:1729061781730635001, v:0} max_trans_id=392222522 max_elr_trans_version={val:1729061781730635001, v:0} max_elr_trans_id=392222522 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781730635001, v:0} min_modify_scn={val:1729061781730635001, v:0}}) [2024-10-16 14:57:22.316943] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46e698 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223143} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842316115000, v:0}, tx_id:{txid:392223143}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223143}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223143}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842316115, active_ts:1729061842316115, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842316114, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.317018] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=74][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223143} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223143}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.317042] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22][errcode=0] primary key duplication conflict(key={"BIGINT":684618585363566592},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223143} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.317062] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.317071] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618585363566592},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.317151] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.317210] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223143} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.317875] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C4E-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:22.317909] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C4E-0-0] [lt=25] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:22.318107] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4888-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061842285705, keepalive_ts:-1}) [2024-10-16 14:57:22.318126] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4888-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061842285705, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:22.318812] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0e885b94c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223144} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5}, prev=this=0x2b0e88573360 trans_version={val:1729061781732815001, v:0} scn={val:1729061781732815001, v:0} tx_id={txid:392222523} prev=0x2b0e88573220 next=(nil) modify_count=1 acc_checksum=2471239483 version=1729061779336102 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842318216001, v:0}, *this={this=0x2b0e88572f18 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e88573360 latest_compact_node=(nil) max_trans_version={val:1729061781732815001, v:0} max_trans_id=392222523 max_elr_trans_version={val:1729061781732815001, v:0} max_elr_trans_id=392222523 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781732815001, v:0} min_modify_scn={val:1729061781732815001, v:0}}) [2024-10-16 14:57:22.318853] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=40][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0e885b94c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223144} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209276}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842318216001, v:0}, tx_id:{txid:392223144}, scn:{branch:0, seq:3}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223144}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223144}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842318216, active_ts:1729061842318216, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842318215, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.318929] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=75][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223144} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223144}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779336102, seq_no:{branch:0, seq:5}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.318950] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19][errcode=0] primary key duplication conflict(key={"BIGINT":684618585522950145},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223144} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.318967] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.318975] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618585522950145},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.319043] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.319104] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223144} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.320280] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb26d80 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223145} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadac98f0 trans_version={val:1729061781733874001, v:0} scn={val:1729061781733874001, v:0} tx_id={txid:392222524} prev=0x2b0fadac97b0 next=(nil) modify_count=1 acc_checksum=2242374786 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1056}, snapshot_version={val:1729061842319289001, v:0}, *this={this=0x2b0fadac94a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac98f0 latest_compact_node=(nil) max_trans_version={val:1729061781733874001, v:0} max_trans_id=392222524 max_elr_trans_version={val:1729061781733874001, v:0} max_elr_trans_id=392222524 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781733874001, v:0} min_modify_scn={val:1729061781733874001, v:0}}) [2024-10-16 14:57:22.320368] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=87][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb26d80 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223145} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842319289001, v:0}, tx_id:{txid:392223145}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223145}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223145}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842319289, active_ts:1729061842319289, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842319290, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.320430] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223145} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223145}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.320452] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=0] primary key duplication conflict(key={"BIGINT":684618585652973568},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223145} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.320464] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.320470] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618585652973568},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.320518] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.320558] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223145} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.320820] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=16] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:22.322787] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25] handle_sp_rollback_resp(tx_id={txid:392223146}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.322951] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223146}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.322965] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223146}, start_ts=1729061842322377, retries=0, savepoint={branch:0, seq:4}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223146}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1074}, op_sn:3, alloc_ts:1729061842320860, active_ts:1729061842321935, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442321935, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333307077826, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842320861, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.324334] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f535658 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202870} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2134}, prev=this=0x2b0f7f4b1fa0 trans_version={val:1729061779764152000, v:0} scn={val:1729061779764152000, v:0} tx_id={txid:392201834} prev=0x2b0f7f4b1e60 next=(nil) modify_count=1 acc_checksum=2451120618 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:8489}, snapshot_version={val:1729061842321935000, v:0}, *this={this=0x2b0f7f4a91c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4b1fa0 latest_compact_node=(nil) max_trans_version={val:1729061779764152000, v:0} max_trans_id=392201834 max_elr_trans_version={val:1729061779764152000, v:0} max_elr_trans_id=392201834 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779764152000, v:0} min_modify_scn={val:1729061779764152000, v:0}}) [2024-10-16 14:57:22.324369] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f535658 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202870} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2134}, ctx={this:0x2b0e2b750080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606314, tablet_id:{id:209285}, table_iter:0x2b0e2b74f210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606314, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842321935000, v:0}, tx_id:{txid:392202870}, scn:{branch:0, seq:1073}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202870}, tx_desc:{this:0x2b0ede4d3d90, tx_id:{txid:392202870}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061842324058, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842320863, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c246c50, mem_ctx:0x2b0e0c2481d0, tx_scn:{branch:0, seq:2134}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.324429] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202870} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202870}, data:0x2b0e2b74c438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:2134}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.324444] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513899163439104},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202870} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.324456] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.324462] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513899163439104},{"BIGINT":1729058400},{"VARCHAR":"Ah¾€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.324511] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.325480] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:22.325739] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4255}, to_seq_no={branch:0, seq:1074}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202870} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.327355] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:22.327361] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1914) [2024-10-16 14:57:22.330234] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24] handle_sp_rollback_resp(tx_id={txid:392223147}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.330325] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=47] all savepoint rollback succeed(tx.tx_id={txid:392223147}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.330356] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223147}, start_ts=1729061842329743, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223147}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3176}, op_sn:3, alloc_ts:1729061842326182, active_ts:1729061842329357, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442329357, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008453843667, first_scn:{branch:0, seq:2118}, last_scn:{branch:0, seq:2118}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842326181, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.337632] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22235][T1002_TenantWea][T1001][YB4A0A000407-00061E9B3B0B75E0-0-0] [lt=11] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.162681007385253906e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=308, Throttle Trigger(MB)=184, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.933448127025233816e+03) [2024-10-16 14:57:22.337857] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=46] get number(ret=0, service_type_=0, range=1, base_id=1729061842336827000, start_id=1729061842336827000, end_id=1729061842336827001) [2024-10-16 14:57:22.339792] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061842339374000, start_id=1729061842339374000, end_id=1729061842339374001) [2024-10-16 14:57:22.340342] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061842339374000, start_id=1729061842339374001, end_id=1729061842339374002) [2024-10-16 14:57:22.340708] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27] handle_sp_rollback_resp(tx_id={txid:392223148}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.341481] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41] all savepoint rollback succeed(tx.tx_id={txid:392223148}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.341502] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223148}, start_ts=1729061842338425, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223148}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3595}, op_sn:3, alloc_ts:1729061842334722, active_ts:1729061842338316, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442338316, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008462055816, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842334721, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.349194] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=43] rpc statistics(total_trans_msg_count=319, total_batch_msg_count=0) [2024-10-16 14:57:22.351131] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:22.351169] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=20] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:22.351185] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=14] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:22.351192] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=7] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:22.351227] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=35] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:22.351232] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17111) [2024-10-16 14:57:22.352114] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E6-0-0] [lt=25] [LS_LOCATION]dump tenant ls location caches(tenant_id=1, tenant_ls_locations=[{cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061839976770, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:22.352151] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E6-0-0] [lt=35] [LS_LOCATION]dump tenant ls location caches(tenant_id=1001, tenant_ls_locations=[{cache_key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061839977506, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:22.352167] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E6-0-0] [lt=15] [LS_LOCATION]dump tenant ls location caches(tenant_id=1002, tenant_ls_locations=[{cache_key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061839978239, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061839978237, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061839978240, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}]}, {cache_key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061839978235, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:22.352208] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E6-0-0] [lt=41] [LS_LOCATION]dump tenant ls location caches(tenant_id=1003, tenant_ls_locations=[{cache_key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061839978864, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:22.352228] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E6-0-0] [lt=12] [LS_LOCATION]dump tenant ls location caches(tenant_id=1004, tenant_ls_locations=[{cache_key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061839979542, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061839979545, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061839979546, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}]}, {cache_key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061839979543, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:22.352285] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6dfb758 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223149} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d7f830 trans_version={val:1729061781742362000, v:0} scn={val:1729061781742362000, v:0} tx_id={txid:392222528} prev=0x2b0ec6d7f6f0 next=(nil) modify_count=1 acc_checksum=241055568 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842351329000, v:0}, *this={this=0x2b0ec6d7f3e8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d7f830 latest_compact_node=(nil) max_trans_version={val:1729061781742362000, v:0} max_trans_id=392222528 max_elr_trans_version={val:1729061781742362000, v:0} max_elr_trans_id=392222528 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781742362000, v:0} min_modify_scn={val:1729061781742362000, v:0}}) [2024-10-16 14:57:22.352327] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6dfb758 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223149} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842351329000, v:0}, tx_id:{txid:392223149}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223149}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223149}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842351329, active_ts:1729061842351329, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842351328, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.352395] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=68][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223149} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223149}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.352410] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":684618586420531200},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223149} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.352422] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.352429] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618586420531200},{"BIGINT":1729058400},{"VARCHAR":"@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.352487] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.352520] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223149} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.352925] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B78EBC43F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061842352428000, start_id=1729061842352428000, end_id=1729061842352428001) [2024-10-16 14:57:22.360363] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ec6dfbe38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202871} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4300}, prev=this=0x2b0ec6d67070 trans_version={val:1729061779768402000, v:0} scn={val:1729061779768402000, v:0} tx_id={txid:392201835} prev=0x2b0ec6d66f30 next=(nil) modify_count=1 acc_checksum=53173695 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4252}, snapshot_version={val:1729061842354616000, v:0}, *this={this=0x2b0ec6d66c28 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d67070 latest_compact_node=(nil) max_trans_version={val:1729061779768402000, v:0} max_trans_id=392201835 max_elr_trans_version={val:1729061779768402000, v:0} max_elr_trans_id=392201835 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061779768402000, v:0} min_modify_scn={val:1729061779768402000, v:0}}) [2024-10-16 14:57:22.360393] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ec6dfbe38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202871} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4300}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437607380, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437607380, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842354616000, v:0}, tx_id:{txid:392202871}, scn:{branch:0, seq:3238}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202871}, tx_desc:{this:0x2b0ec37afeb0, tx_id:{txid:392202871}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061842357793, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842353494, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4300}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.360446] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=52][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202871} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202871}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4300}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.360461] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830046862233600},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202871} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.360473] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.360478] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046862233600},{"BIGINT":1729058400},{"VARCHAR":"BprN‰]p", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.360526] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.360581] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26170][T1_L0_G5][T1][YB4A0A000408-00061E9B12AF7BD7-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061842357793000, start_id=1729061842357793000, end_id=1729061842357793001) [2024-10-16 14:57:22.361722] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:22.362212] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.362287] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8598}, to_seq_no={branch:0, seq:3239}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202871} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.364641] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FC2D-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:22.364655] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FC2D-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19933, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:22.364660] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FC2D-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=283, avg_log_submit_to_slide_time=19927) [2024-10-16 14:57:22.364688] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:22.364721] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:22.364727] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:22.367264] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] handle_sp_rollback_resp(tx_id={txid:392223150}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.367345] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392223150}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.367354] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223150}, start_ts=1729061842360850, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223150}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3180}, op_sn:3, alloc_ts:1729061842355679, active_ts:1729061842357793, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442357793, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333343446691, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842354615, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.367499] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=15] blacklist_loop exec finished(cost_time=15, is_enabled=true, send_cnt=0) [2024-10-16 14:57:22.371972] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.374039] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=105, avg_log_batch_cnt=0, total_group_log_size=170494, avg_group_log_size=1623) [2024-10-16 14:57:22.375689] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46ed78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223151} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f40c620 trans_version={val:1729061781745557001, v:0} scn={val:1729061781745557001, v:0} tx_id={txid:392222530} prev=0x2b0f3f40c4e0 next=(nil) modify_count=1 acc_checksum=2008619548 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842375056000, v:0}, *this={this=0x2b0f3f40c1d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f40c620 latest_compact_node=(nil) max_trans_version={val:1729061781745557001, v:0} max_trans_id=392222530 max_elr_trans_version={val:1729061781745557001, v:0} max_elr_trans_id=392222530 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781745557001, v:0} min_modify_scn={val:1729061781745557001, v:0}}) [2024-10-16 14:57:22.375735] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=45][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46ed78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223151} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842375056000, v:0}, tx_id:{txid:392223151}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223151}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223151}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842375056, active_ts:1729061842375056, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842375055, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.375813] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=77][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223151} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223151}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:22.375850] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=0] primary key duplication conflict(key={"BIGINT":684618586999345152},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223151} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.375868] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:22.375882] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618586999345152},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:22.375974] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=45][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:22.376033] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223151} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.378867] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223152}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.378901] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] all savepoint rollback succeed(tx.tx_id={txid:392223152}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.378910] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223152}, start_ts=1729061842378566, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223152}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1616}, op_sn:3, alloc_ts:1729061842377735, active_ts:1729061842377735, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442377735, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333363337302, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842376119, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.383509] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223153}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.383558] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=43] all savepoint rollback succeed(tx.tx_id={txid:392223153}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.383573] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223153}, start_ts=1729061842383273, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223153}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2056}, op_sn:3, alloc_ts:1729061842380933, active_ts:1729061842382990, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442382990, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333367081985, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842380934, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.383715] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:22.388803] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061842388292000, start_id=1729061842388292000, end_id=1729061842388292001) [2024-10-16 14:57:22.390052] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23] handle_sp_rollback_resp(tx_id={txid:392223154}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.390099] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=53] all savepoint rollback succeed(tx.tx_id={txid:392223154}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.390150] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=46] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223154}, start_ts=1729061842387926, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223154}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2116}, op_sn:3, alloc_ts:1729061842385112, active_ts:1729061842387230, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442387230, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008512235196, first_scn:{branch:0, seq:1059}, last_scn:{branch:0, seq:1059}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842385114, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.395340] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=59] get number(ret=0, service_type_=0, range=1, base_id=1729061842394679000, start_id=1729061842394679000, end_id=1729061842394679001) [2024-10-16 14:57:22.396376] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:22.396397] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=15] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=8388608, ctx.total_memstore_used_=41943040, ctx.total_memstore_hold_=41943040, memstore_freeze_trigger_limit_=27810600, tenant_id=1002) [2024-10-16 14:57:22.396414] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:22.396541] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392223155}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.396624] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] all savepoint rollback succeed(tx.tx_id={txid:392223155}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.396634] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223155}, start_ts=1729061842396105, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223155}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061842394679, active_ts:1729061842395739, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442395739, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008520283711, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842394678, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.402408] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061842402042000, start_id=1729061842402042000, end_id=1729061842402042001) [2024-10-16 14:57:22.405270] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=60][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f46f458 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223156} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f40cd20 trans_version={val:1729061781759757000, v:0} scn={val:1729061781759757000, v:0} tx_id={txid:392222535} prev=0x2b0f3f40cbe0 next=(nil) modify_count=1 acc_checksum=2211142225 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061842404159000, v:0}, *this={this=0x2b0f3f40c8d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f40cd20 latest_compact_node=(nil) max_trans_version={val:1729061781759757000, v:0} max_trans_id=392222535 max_elr_trans_version={val:1729061781759757000, v:0} max_elr_trans_id=392222535 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781759757000, v:0} min_modify_scn={val:1729061781759757000, v:0}}) [2024-10-16 14:57:22.405308] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f46f458 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223156} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061842404159000, v:0}, tx_id:{txid:392223156}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223156}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223156}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061842404159, active_ts:1729061842405216, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842404158, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:22.405371] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618587943063552},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223156} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.405444] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223156} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.408426] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223157}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.408472] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223157}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.408490] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223157}, start_ts=1729061842407004, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223157}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1036}, op_sn:3, alloc_ts:1729061842406251, active_ts:1729061842406251, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442406251, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008531236831, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842405215, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.417825] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C50-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=9, avg_log_batch_cnt=0, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:22.419513] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3D-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:22.419542] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3D-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061842416882000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061842416882000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:22.421121] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C3D-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061842420066000, start_id=1729061842420066000, end_id=1729061842420066001) [2024-10-16 14:57:22.423491] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223158}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.424134] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=40] all savepoint rollback succeed(tx.tx_id={txid:392223158}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.424151] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223158}, start_ts=1729061842423171, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223158}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842422185, active_ts:1729061842422185, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442422185, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333407899819, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842422185, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.424374] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C3D-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061842424292000, start_id=1729061842424292000, end_id=1729061842424292001) [2024-10-16 14:57:22.428993] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223159}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.429032] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=46] all savepoint rollback succeed(tx.tx_id={txid:392223159}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.429043] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223159}, start_ts=1729061842427954, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223159}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1144}, op_sn:3, alloc_ts:1729061842426412, active_ts:1729061842427553, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442427553, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008552265055, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842426411, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.430136] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513899264102400},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202878} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.435773] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6355}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202878} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.438054] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=18, replace_time=0, replace_start_pos=1289737, replace_num=31457) [2024-10-16 14:57:22.438073] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=18, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:22.439284] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] handle_sp_rollback_resp(tx_id={txid:392223160}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.439317] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223160}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.439326] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223160}, start_ts=1729061842437705, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223160}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842436030, active_ts:1729061842436030, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442436030, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333422435816, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842436031, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.442815] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223161}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.442849] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223161}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.442858] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223161}, start_ts=1729061842442534, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223161}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1062}, op_sn:3, alloc_ts:1729061842441105, active_ts:1729061842442166, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442442166, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008566846441, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842441104, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.443759] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061842443235000, start_id=1729061842443235000, end_id=1729061842443235001) [2024-10-16 14:57:22.446777] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047139057664},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202879} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.447317] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3190}, to_seq_no={branch:0, seq:2122}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202879} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.449822] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6C-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:22.449843] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6C-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:22.451758] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223162}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.451794] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=42] all savepoint rollback succeed(tx.tx_id={txid:392223162}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.451822] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223162}, start_ts=1729061842451491, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223162}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842450676, active_ts:1729061842450676, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442450676, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333436202812, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842450675, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.455690] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392223163}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.456530] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223163}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.456546] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223163}, start_ts=1729061842455486, retries=0, savepoint={branch:0, seq:1215}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223163}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1218}, op_sn:3, alloc_ts:1729061842453846, active_ts:1729061842455060, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442455060, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333440302569, first_scn:{branch:0, seq:1217}, last_scn:{branch:0, seq:1217}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842453847, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.460359] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223164}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.460409] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41] all savepoint rollback succeed(tx.tx_id={txid:392223164}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.460421] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223164}, start_ts=1729061842459731, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223164}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1085}, op_sn:3, alloc_ts:1729061842458239, active_ts:1729061842459324, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442459324, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008584072068, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842458241, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.462272] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.467201] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=6, avg_log_batch_cnt=1, total_group_log_size=723, avg_group_log_size=120) [2024-10-16 14:57:22.467226] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=723) [2024-10-16 14:57:22.469293] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=23] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=101, avg_log_batch_cnt=0, total_group_log_size=164013, avg_group_log_size=1623) [2024-10-16 14:57:22.471593] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30] handle_sp_rollback_resp(tx_id={txid:392223165}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.471636] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223165}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.471645] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223165}, start_ts=1729061842471336, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223165}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1060}, op_sn:3, alloc_ts:1729061842469968, active_ts:1729061842471027, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442471027, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333455464924, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842469967, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.472048] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.473045] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=201, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:22.473060] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=216, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:22.476533] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=5] LogLoopThread round_cost_time(us)(round_cost_time=22) [2024-10-16 14:57:22.481517] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] update id meta(ret=0, service_type=0, limited_id=1729061859029444000, latest_scn={val:1729061842479541001, v:0}, *this={tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061859029444000, latest_log_ts:{val:1729061842479541001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:22.481569] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=52] handle submit callback(service_type_=0, success=true, used_ts=1037, log_ts={val:1729061842479541001, v:0}, limited_id=1729061859029444000) [2024-10-16 14:57:22.481577] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:22.481581] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:22.482624] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23] handle_sp_rollback_resp(tx_id={txid:392223166}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.482742] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=56] all savepoint rollback succeed(tx.tx_id={txid:392223166}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.482751] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223166}, start_ts=1729061842480423, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223166}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2120}, op_sn:3, alloc_ts:1729061842477420, active_ts:1729061842479541, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442479541, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008603986259, first_scn:{branch:0, seq:1064}, last_scn:{branch:0, seq:1064}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842477421, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.487793] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618589855666176},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223167} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.487863] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223167} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.489841] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223168}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.489872] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25] all savepoint rollback succeed(tx.tx_id={txid:392223168}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.489881] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223168}, start_ts=1729061842489028, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223168}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842488001, active_ts:1729061842488001, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442488001, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008613427833, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842488000, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.495772] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061842495415000, start_id=1729061842495415000, end_id=1729061842495415001) [2024-10-16 14:57:22.496247] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=51] get number(ret=0, service_type_=0, range=1, base_id=1729061842495415000, start_id=1729061842495415001, end_id=1729061842495415002) [2024-10-16 14:57:22.497035] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223169}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.497067] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24] all savepoint rollback succeed(tx.tx_id={txid:392223169}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.497076] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223169}, start_ts=1729061842496747, retries=0, savepoint={branch:0, seq:5}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223169}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1065}, op_sn:3, alloc_ts:1729061842495415, active_ts:1729061842496479, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442496479, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333481473934, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842495414, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.500848] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047290052608},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202883} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.501105] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223170}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.501229] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223170}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.501244] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223170}, start_ts=1729061842500746, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223170}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842499673, active_ts:1729061842500731, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442500731, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333485585000, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842499672, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.502113] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3155}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202883} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.504407] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] handle_sp_rollback_resp(tx_id={txid:392223171}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.504488] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223171}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.504498] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223171}, start_ts=1729061842504030, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223171}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1057}, op_sn:3, alloc_ts:1729061842502787, active_ts:1729061842503847, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442503847, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333488862164, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842502790, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.505773] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061842504908000, start_id=1729061842504908000, end_id=1729061842504908001) [2024-10-16 14:57:22.508834] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223172}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.508891] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223172}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.508907] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223172}, start_ts=1729061842508468, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223172}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2116}, op_sn:3, alloc_ts:1729061842506095, active_ts:1729061842508210, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442508210, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008632689240, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842506094, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.516964] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061842516796000, start_id=1729061842516796000, end_id=1729061842516796001) [2024-10-16 14:57:22.517131] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061842517125], gts:1729061842516796000, latest_srr:[mts=1729061842517125]}) [2024-10-16 14:57:22.517176] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=32] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a81850, tx_id:{txid:199663929}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061842516796, active_ts:1729061842516796, commit_ts:-1, finish_ts:1729061842516796, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1729061872516790, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842514676, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:22.518109] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=35] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a6bc70, tx_id:{txid:199663930}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061842516796, active_ts:1729061842516796, commit_ts:-1, finish_ts:1729061842517885, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061872516791, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842516795, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:22.524318] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223173}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.524379] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223173}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.524395] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223173}, start_ts=1729061842524063, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223173}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1417}, op_sn:3, alloc_ts:1729061842523501, active_ts:1729061842523501, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442523501, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333508884770, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842522084, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.525782] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:22.526935] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047336189953},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202885} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.527225] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4791}, to_seq_no={branch:0, seq:3581}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202885} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.527542] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFD2-0-0] [lt=20] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=421, avg_log_submit_to_slide_time=1340) [2024-10-16 14:57:22.529858] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24] handle_sp_rollback_resp(tx_id={txid:392223174}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.529932] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223174}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.529943] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223174}, start_ts=1729061842529120, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223174}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3328}, op_sn:3, alloc_ts:1729061842525663, active_ts:1729061842528994, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442528994, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008652060696, first_scn:{branch:0, seq:1212}, last_scn:{branch:0, seq:1212}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842525666, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.531783] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:22.531802] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4384) [2024-10-16 14:57:22.531818] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1443756197, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:22.532709] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C3FCF-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=85, avg_log_gen_to_freeze_time=20, avg_log_gen_to_submit_time=48, avg_log_submit_to_flush_time=318, avg_log_submit_to_slide_time=588) [2024-10-16 14:57:22.535099] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904196603904},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223175} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.535164] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223175} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.537576] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223176}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.537746] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223176}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.537758] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223176}, start_ts=1729061842537044, retries=0, savepoint={branch:0, seq:1067}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223176}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1070}, op_sn:3, alloc_ts:1729061842535426, active_ts:1729061842536492, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442536492, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333521749756, first_scn:{branch:0, seq:1069}, last_scn:{branch:0, seq:1069}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842535427, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.539050] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061842537550000, start_id=1729061842537550000, end_id=1729061842537550001) [2024-10-16 14:57:22.539991] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061842537550000, start_id=1729061842537550000, end_id=1729061842537550001) [2024-10-16 14:57:22.540037] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:22.540050] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:22.540530] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061842540229000, start_id=1729061842540229000, end_id=1729061842540229001) [2024-10-16 14:57:22.542893] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=54][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:22.542929] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047378132992},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202887} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.543765] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223177}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.543794] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=56] all savepoint rollback succeed(tx.tx_id={txid:392223177}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.543804] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223177}, start_ts=1729061842543550, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223177}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1044}, op_sn:3, alloc_ts:1729061842542293, active_ts:1729061842543338, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442543338, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333528360796, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842542294, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.546364] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223178}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.546396] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=41] all savepoint rollback succeed(tx.tx_id={txid:392223178}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.546411] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223178}, start_ts=1729061842546145, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223178}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842545417, active_ts:1729061842545417, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442545417, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333530992510, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842545416, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.547007] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8944}, to_seq_no={branch:0, seq:2683}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202887} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.549100] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23] handle_sp_rollback_resp(tx_id={txid:392223179}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.549133] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223179}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.549145] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223179}, start_ts=1729061842548883, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223179}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1087}, op_sn:3, alloc_ts:1729061842547538, active_ts:1729061842548626, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442548626, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333533718190, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842547539, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.553028] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061842552994000, start_id=1729061842552994000, end_id=1729061842552994001) [2024-10-16 14:57:22.554162] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223180}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.554246] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223180}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.554259] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223180}, start_ts=1729061842551625, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223180}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842550740, active_ts:1729061842550740, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442550740, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008675783560, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842550739, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.559282] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830049932464128},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223181} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.559394] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223181} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.560653] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:22.562186] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] handle_sp_rollback_resp(tx_id={txid:392223182}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.562234] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] all savepoint rollback succeed(tx.tx_id={txid:392223182}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.562249] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223182}, start_ts=1729061842561648, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223182}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2168}, op_sn:3, alloc_ts:1729061842559325, active_ts:1729061842561492, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442561492, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008685631031, first_scn:{branch:0, seq:1042}, last_scn:{branch:0, seq:1042}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842559324, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.562284] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.563114] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D514A-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=115, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:22.563135] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D514A-0-0] [lt=21] [PALF STAT LOG LIFE TIME](cur_stat_count=115, stat_interval=1000000, avg (count/cost)=658, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:22.563138] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=185824, hit_cnt=112, read_cnt=112, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:22.565643] WDIAG [PALF] is_prev_log_pid_match (log_sliding_window.cpp:3884) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=6][errcode=-4018] get_prev_log_info_ failed(tmp_ret=-4018, log_id=218497107, palf_id=1003, self="10.0.4.7:2890", log_id=218497107, lsn={lsn:207589290543}, prev_lsn={lsn:207589288919}, prev_log_pid=81) [2024-10-16 14:57:22.566511] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:22.566525] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:22.567787] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28] handle_sp_rollback_resp(tx_id={txid:392223183}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.567833] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54] all savepoint rollback succeed(tx.tx_id={txid:392223183}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.567843] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223183}, start_ts=1729061842567491, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223183}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842566631, active_ts:1729061842566631, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442566631, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333552178783, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842566631, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.569247] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.569251] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.569269] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.569285] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.570696] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223184}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.570723] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223184}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.570731] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223184}, start_ts=1729061842570474, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223184}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842569819, active_ts:1729061842569819, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442569819, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333555317888, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842569818, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.572109] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.579735] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [22451][T1003_ObXAHbWor][T1003][Y0-0000000000000000-0-0] [lt=12] XA scheduler heartbeat task statistics(avg_time=1) [2024-10-16 14:57:22.580299] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1}, {id:1001}]) [2024-10-16 14:57:22.581747] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047499767809},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202891} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.582872] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:9733}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202891} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.582923] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] handle_sp_rollback_resp(tx_id={txid:392223185}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.583179] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223185}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.583189] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223185}, start_ts=1729061842581351, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223185}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:8142}, op_sn:3, alloc_ts:1729061842573054, active_ts:1729061842581204, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442581204, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333564860549, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842573062, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.585621] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904360181760},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223186} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.585698] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223186} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.588956] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061842588184000, start_id=1729061842588184000, end_id=1729061842588184001) [2024-10-16 14:57:22.589109] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] handle_sp_rollback_resp(tx_id={txid:392223187}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.589149] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] all savepoint rollback succeed(tx.tx_id={txid:392223187}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.589181] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223187}, start_ts=1729061842587483, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223187}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061842586007, active_ts:1729061842587067, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442587067, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008711826397, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842586006, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.595466] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223188}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.595518] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=64] all savepoint rollback succeed(tx.tx_id={txid:392223188}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.595531] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223188}, start_ts=1729061842594844, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223188}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842593534, active_ts:1729061842594592, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442594592, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008719214309, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842593533, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.596259] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4576, mem_used=8318976) [2024-10-16 14:57:22.601700] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=51] get number(ret=0, service_type_=0, range=1, base_id=1729061842600945000, start_id=1729061842600945000, end_id=1729061842600945001) [2024-10-16 14:57:22.601801] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618590296068096},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223189} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.601886] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223189} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.602794] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047537516545},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202893} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.603732] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223190}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.603760] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] all savepoint rollback succeed(tx.tx_id={txid:392223190}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.603769] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223190}, start_ts=1729061842603313, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223190}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1090}, op_sn:3, alloc_ts:1729061842602160, active_ts:1729061842603249, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442603249, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333588031929, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842602159, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.604384] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3372}, to_seq_no={branch:0, seq:11}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202893} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.605808] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830050041516033},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223191} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.605879] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223191} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.607623] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5A-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:22.607651] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5A-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061842126108004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061842126108004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:22.608065] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392223192}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.608098] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223192}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.608109] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223192}, start_ts=1729061842607746, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223192}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2076}, op_sn:3, alloc_ts:1729061842606514, active_ts:1729061842607556, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442607556, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008732105925, first_scn:{branch:0, seq:1040}, last_scn:{branch:0, seq:1040}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842605480, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.608843] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D5A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061842608589000, start_id=1729061842608589000, end_id=1729061842608589001) [2024-10-16 14:57:22.611529] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D5A-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061842610702000, start_id=1729061842610702000, end_id=1729061842610702001) [2024-10-16 14:57:22.612004] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061842611773000, start_id=1729061842611773000, end_id=1729061842611773001) [2024-10-16 14:57:22.615614] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513899440263168},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202894} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.615912] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904410513408},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223193} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.615978] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223193} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.617088] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830050070876160},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223194} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.617144] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223194} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.618941] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6672}, to_seq_no={branch:0, seq:3172}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202894} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.619230] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":675513904473427968},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223195} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.619286] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223195} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.622217] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25] handle_sp_rollback_resp(tx_id={txid:392223196}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.622305] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223196}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.622316] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223196}, start_ts=1729061842621961, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223196}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1066}, op_sn:3, alloc_ts:1729061842620587, active_ts:1729061842621651, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442621651, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333606583773, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842620586, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.626381] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319662583808},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223197} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.626459] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223197} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.627064] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B97-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061842626912000, start_id=1729061842626912000, end_id=1729061842626912001) [2024-10-16 14:57:22.627094] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0B9B-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061842626912000, start_id=1729061842626912000, end_id=1729061842626912001) [2024-10-16 14:57:22.627260] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B98-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061842626912000, start_id=1729061842626912000, end_id=1729061842626912001) [2024-10-16 14:57:22.627809] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545345052069888},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223198} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.627870] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223198} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.630359] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223199}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.630424] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223199}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.630439] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223199}, start_ts=1729061842629331, retries=0, savepoint={branch:0, seq:5}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223199}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061842627966, active_ts:1729061842629025, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442629025, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008753676064, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842627967, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.634573] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=25] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=154268) [2024-10-16 14:57:22.636185] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=14] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:57:22.636294] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=32] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=155892, hit_cnt=93, read_cnt=93, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:22.636721] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904548925440},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223200} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.636795] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223200} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.639634] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=16, replace_time=0, replace_start_pos=1321194, replace_num=31457) [2024-10-16 14:57:22.639648] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=16, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:22.640763] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223201}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.640801] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392223201}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.640810] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223201}, start_ts=1729061842639965, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223201}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1063}, op_sn:3, alloc_ts:1729061842637748, active_ts:1729061842637748, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442637748, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333623112931, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842636687, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.641228] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C98-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510310509621, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410514, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:22.642146] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C9B-0-0] [lt=19][errcode=-4201] hashtable not init [2024-10-16 14:57:22.642161] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C9B-0-0] [lt=14][errcode=-4016] get cursor info failed(cursor_id=1109, get_sessid()=3221565766) [2024-10-16 14:57:22.642218] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C9C-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061842641344000, start_id=1729061842641344000, end_id=1729061842641344001) [2024-10-16 14:57:22.645674] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223202}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.645712] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223202}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.645743] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223202}, start_ts=1729061842644630, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223202}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1046}, op_sn:3, alloc_ts:1729061842643465, active_ts:1729061842644512, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442644512, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008768994414, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842643466, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.650457] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:57:22.650473] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:22.650480] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:22.650485] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=5] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:22.651688] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:22.652842] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=111, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:57:22.652987] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=143] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:22.654309] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223203}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.654387] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223203}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.654397] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223203}, start_ts=1729061842653904, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223203}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1029}, op_sn:3, alloc_ts:1729061842651908, active_ts:1729061842652934, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442652934, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008778230050, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842651907, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.657612] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=24] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:57:22.658529] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061842658287000, start_id=1729061842658287000, end_id=1729061842658287001) [2024-10-16 14:57:22.662347] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=36][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.663842] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830050129596417},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223204} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.663926] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223204} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.664855] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904637005824},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223205} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.664911] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223205} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.669932] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24] handle_sp_rollback_resp(tx_id={txid:392223206}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.669970] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] all savepoint rollback succeed(tx.tx_id={txid:392223206}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.669984] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223206}, start_ts=1729061842667188, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223206}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061842665779, active_ts:1729061842666836, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442666836, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008791503337, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842665778, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.672563] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.678898] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223207}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.678936] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223207}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.678947] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223207}, start_ts=1729061842678652, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223207}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1051}, op_sn:3, alloc_ts:1729061842677105, active_ts:1729061842678155, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442678155, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333662994670, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842677104, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.681874] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=95, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:22.684552] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=94, stat_interval=1000000, avg (count/cost)=261, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:22.684570] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=95, stat_interval=1000000, avg (count/cost)=49, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:22.684594] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=95, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:22.684860] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223208}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.684917] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223208}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.684926] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223208}, start_ts=1729061842684688, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223208}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1069}, op_sn:3, alloc_ts:1729061842683458, active_ts:1729061842684526, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442684526, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333669517144, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842683457, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.686881] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513899490594816},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202900} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.687134] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618590623223808},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223209} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.687195] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223209} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.687741] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4249}, to_seq_no={branch:0, seq:1074}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202900} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.688092] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904699920384},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223210} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.688146] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223210} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.688332] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=19] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:22.688935] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830050230259713},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223211} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.688985] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223211} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.690136] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] handle_sp_rollback_resp(tx_id={txid:392223212}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.690165] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392223212}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.690175] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223212}, start_ts=1729061842689936, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223212}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1020}, op_sn:3, alloc_ts:1729061842688766, active_ts:1729061842689793, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442689793, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333674750393, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842688773, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.693451] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061842692961000, start_id=1729061842692961000, end_id=1729061842692961001) [2024-10-16 14:57:22.693595] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223213}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.693912] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223213}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.693927] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223213}, start_ts=1729061842693294, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223213}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061842691897, active_ts:1729061842692961, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442692961, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333678106299, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842691903, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.694962] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1, min_version={val:1729061842495415000, v:0}, base_version={val:1729061842294817000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061842495415000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061842694949, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061842495415000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061842586032, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061842495415000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061842645425, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:22.695591] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:22.695999] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061842695078000, start_id=1729061842695078000, end_id=1729061842695078001) [2024-10-16 14:57:22.696054] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:22.696066] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3979) [2024-10-16 14:57:22.696073] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:22.697071] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] handle_sp_rollback_resp(tx_id={txid:392223214}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.697135] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=45] all savepoint rollback succeed(tx.tx_id={txid:392223214}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.697146] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223214}, start_ts=1729061842696880, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223214}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1150}, op_sn:3, alloc_ts:1729061842695078, active_ts:1729061842696236, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442696236, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333681661269, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842695086, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.699378] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904762834944},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223215} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.699450] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223215} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.703052] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392223216}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.703088] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] all savepoint rollback succeed(tx.tx_id={txid:392223216}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.703109] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223216}, start_ts=1729061842700961, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223216}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1019}, op_sn:3, alloc_ts:1729061842699397, active_ts:1729061842700415, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442700415, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008825326662, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842699396, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.707505] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] get number(ret=0, service_type_=0, range=1, base_id=1729061842706801000, start_id=1729061842706801001, end_id=1729061842706801002) [2024-10-16 14:57:22.708909] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223217}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.708975] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392223217}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.709013] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223217}, start_ts=1729061842708225, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223217}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842706801, active_ts:1729061842707861, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442707861, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008832473396, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842706802, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.714180] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061842713163000, start_id=1729061842713163000, end_id=1729061842713163001) [2024-10-16 14:57:22.717914] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C56-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:22.717929] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C56-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10803, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:22.717934] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C56-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=460, avg_log_submit_to_slide_time=10797) [2024-10-16 14:57:22.717942] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C56-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172958, log_task={header:{begin_lsn:{lsn:14352351107}, end_lsn:{lsn:14352351229}, log_id:115172958, min_scn:{val:1729061842127159004, v:0}, max_scn:{val:1729061842127159004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352350985}, prev_proposal_id:17, committed_end_lsn:{lsn:14352351107}, data_checksum:940009244, accum_checksum:4041355406, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061842617820, freeze_ts:1729061842617821, submit_ts:1729061842617824, flushed_ts:1729061842619989, gen_to_freeze cost time:1, gen_to_submit cost time:4, submit_to_flush cost time:2165}, fs_cb_begin_ts=1729061842717907, log_life_time=100087) [2024-10-16 14:57:22.717957] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:22.718126] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=402, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:22.718134] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:22.718164] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=43, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:22.718192] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:22.719921] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24] handle_sp_rollback_resp(tx_id={txid:392223218}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.719959] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=40] all savepoint rollback succeed(tx.tx_id={txid:392223218}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.719982] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223218}, start_ts=1729061842717716, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223218}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1026}, op_sn:3, alloc_ts:1729061842716317, active_ts:1729061842717342, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442717342, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008842013892, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842716316, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.721739] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:22.725112] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223219}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.725165] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223219}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.725194] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223219}, start_ts=1729061842724886, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223219}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1018}, op_sn:3, alloc_ts:1729061842723649, active_ts:1729061842724666, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442724666, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333709669985, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842723648, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.726079] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1204781056, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:22.727281] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:22.727297] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:22.727490] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15CE-0-0] [lt=15] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:22.731175] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] handle_sp_rollback_resp(tx_id={txid:392223220}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.731205] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223220}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.731215] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223220}, start_ts=1729061842729690, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223220}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2090}, op_sn:3, alloc_ts:1729061842726780, active_ts:1729061842728873, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442728873, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008853897339, first_scn:{branch:0, seq:1062}, last_scn:{branch:0, seq:1062}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842726783, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.731859] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513899528343552},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202904} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.735071] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10500}, to_seq_no={branch:0, seq:6282}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202904} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.736764] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830050322534400},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223221} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.736832] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223221} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.740620] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061842740519000, start_id=1729061842740519000, end_id=1729061842740519001) [2024-10-16 14:57:22.741485] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061842740519000, start_id=1729061842740519000, end_id=1729061842740519001) [2024-10-16 14:57:22.742289] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061842741641000, start_id=1729061842741641000, end_id=1729061842741641001) [2024-10-16 14:57:22.742483] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23] handle_sp_rollback_resp(tx_id={txid:392223222}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.742524] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392223222}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.742532] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223222}, start_ts=1729061842738134, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223222}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842737334, active_ts:1729061842737334, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442737334, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008862507593, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842737333, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.743151] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info(lsn={lsn:207589313276}, scn={val:1729061842721552002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061842723967, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589313276}, next_to_submit_scn_:{val:1729061842721552002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:1624, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1624, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589311652}, end_lsn:{lsn:207589313276}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061842721552001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:1536, scn_:{val:1729061842721552001, v:0}, data_checksum:1367406714, flag:1}}, init_mode_version:0, accumulate_checksum:847972793, curr_entry_is_padding:0, padding_entry_size:1568, padding_entry_scn:{val:1729061842721552001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598572, miss_cnt:9, cache_read_size:149777677717, hit_ratio:"9.999999345923487049e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:57:22.748934] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:22.748961] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:22.748972] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=10] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:22.749328] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904850915328},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223223} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.749408] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223223} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.751392] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223224}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.751424] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25] all savepoint rollback succeed(tx.tx_id={txid:392223224}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.751435] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223224}, start_ts=1729061842750731, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223224}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1062}, op_sn:3, alloc_ts:1729061842749622, active_ts:1729061842750683, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442750683, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008875100698, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842749621, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.755278] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654830047843700736},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202905} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.755967] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223225}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.755993] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=40] all savepoint rollback succeed(tx.tx_id={txid:392223225}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.756001] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223225}, start_ts=1729061842755723, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223225}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842754888, active_ts:1729061842754888, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442754888, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333740517339, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842754887, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.758542] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223226}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.758575] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223226}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.758593] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223226}, start_ts=1729061842758338, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223226}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1018}, op_sn:3, alloc_ts:1729061842757005, active_ts:1729061842758026, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442758026, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333743175756, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842757008, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.759538] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5243}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202905} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.761230] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23] handle_sp_rollback_resp(tx_id={txid:392223227}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.761267] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=43] all savepoint rollback succeed(tx.tx_id={txid:392223227}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.761281] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223227}, start_ts=1729061842761027, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223227}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842760108, active_ts:1729061842760108, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442760108, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333745799333, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842760107, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.762375] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.763407] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:22.763771] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223228}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.763799] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223228}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.763810] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223228}, start_ts=1729061842763592, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223228}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1055}, op_sn:3, alloc_ts:1729061842762223, active_ts:1729061842763283, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442763283, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333748458351, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842762228, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.766438] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618590849716224},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223229} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.766508] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223229} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.768108] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223230}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.768136] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392223230}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.768148] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223230}, start_ts=1729061842767638, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223230}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842766686, active_ts:1729061842766686, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442766686, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333752424715, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842766685, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.768813] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:22.771953] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223231}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.771982] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29] all savepoint rollback succeed(tx.tx_id={txid:392223231}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.771989] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223231}, start_ts=1729061842770925, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223231}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:8}, op_sn:3, alloc_ts:1729061842769887, active_ts:1729061842769887, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442769887, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008895252070, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842769886, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.772591] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.777916] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223232}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.777951] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223232}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.777966] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223232}, start_ts=1729061842777670, retries=0, savepoint={branch:0, seq:5}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223232}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1080}, op_sn:3, alloc_ts:1729061842776291, active_ts:1729061842777371, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442777371, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333762473626, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842776291, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.779552] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:22.779573] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=14] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:22.779582] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=47) [2024-10-16 14:57:22.780596] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:22.782630] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223233}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.782661] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223233}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.782670] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223233}, start_ts=1729061842782323, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223233}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1062}, op_sn:3, alloc_ts:1729061842780545, active_ts:1729061842781606, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442781606, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008906586721, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842780544, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.784172] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:22.788508] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.788577] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.788618] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.788994] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061842787021000, start_id=1729061842787021000, end_id=1729061842787021001) [2024-10-16 14:57:22.789065] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:22.789077] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9692, avg_group_log_size=570) [2024-10-16 14:57:22.789086] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9692) [2024-10-16 14:57:22.789092] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:22.789116] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.789127] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.789135] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.789150] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.789165] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.789176] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.789191] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.789328] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=525, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:22.789340] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9692) [2024-10-16 14:57:22.789351] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=201, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:22.789369] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9692) [2024-10-16 14:57:22.789378] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:22.791120] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:22.791195] INFO [LIB] stat (utility.h:1165) [106984][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D56EB-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:22.791202] INFO [LIB] stat (utility.h:1165) [106984][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D56EB-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1441, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:22.791207] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106984][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D56EB-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=969, avg_log_submit_to_slide_time=1410) [2024-10-16 14:57:22.794507] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223234}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.794539] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=48] all savepoint rollback succeed(tx.tx_id={txid:392223234}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.794552] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223234}, start_ts=1729061842794160, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223234}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842793365, active_ts:1729061842793365, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442793365, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333778944806, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842793364, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.796856] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830047969529856},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202907} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.797216] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223235}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.797242] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223235}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.797251] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223235}, start_ts=1729061842797003, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223235}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1020}, op_sn:3, alloc_ts:1729061842795481, active_ts:1729061842796505, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442796505, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333781795995, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842795486, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.797450] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:11606}, to_seq_no={branch:0, seq:10582}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202907} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.799224] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513904964161536},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223236} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.799292] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223236} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.799567] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061842798627000, start_id=1729061842798627002, end_id=1729061842798627003) [2024-10-16 14:57:22.800075] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319750664192},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223237} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.800128] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223237} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.801323] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] handle_sp_rollback_resp(tx_id={txid:392223238}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.801370] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392223238}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.801378] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223238}, start_ts=1729061842801091, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223238}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1081}, op_sn:3, alloc_ts:1729061842799684, active_ts:1729061842800772, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442800772, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333785947711, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842799691, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.803450] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] new mysql sessid created(conn.sessid_=3221733000, support_ssl=false) [2024-10-16 14:57:22.803484] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13] sm conn init succ(conn.sessid_=3221733000, sess.client_addr_="10.0.4.7:53348") [2024-10-16 14:57:22.803491] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0f27404030, session_id:3221733000, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:22.803736] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27404030, session_id:3221733000, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:22.803752] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=14] kill and revert session(conn.sessid_=3221733000, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:22.803758] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] sock ref clean, do destroy(*s={this:0x2b0f27404030, session_id:3221733000, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:22.803765] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:223) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] mark session id unused(conn.sessid_=3221733000) [2024-10-16 14:57:22.803769] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221733000, proxy_sessid=0, tenant_id=0, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=false, ret=0, trace_id=Y0-0000000000000000-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:0], disconnect_state=0) [2024-10-16 14:57:22.807041] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392223239}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.807073] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223239}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.807083] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223239}, start_ts=1729061842805801, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223239}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061842803935, active_ts:1729061842804995, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442804995, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008930144908, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842803934, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.810631] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [21910][T1001_Compactio][T1001][YB4A0A000407-00061E9973FD0D6E-0-0] [lt=14] InfoPoolResizeTask(cost_ts=7) [2024-10-16 14:57:22.811828] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061842811543000, start_id=1729061842811543000, end_id=1729061842811543001) [2024-10-16 14:57:22.812475] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223241}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.812519] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392223241}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.812527] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223241}, start_ts=1729061842812262, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223241}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842811543, active_ts:1729061842811543, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442811543, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333797043974, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842811542, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.815585] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061842814752000, start_id=1729061842814752000, end_id=1729061842814752001) [2024-10-16 14:57:22.816493] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830050435780608},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223243} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.816579] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223243} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.817854] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1003}, {id:1}, {id:1002}]) [2024-10-16 14:57:22.817911] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392223244}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.817956] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223244}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.817968] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223244}, start_ts=1729061842817716, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223244}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842816885, active_ts:1729061842816885, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442816885, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333802550554, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842816884, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.818159] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C58-0-0] [lt=22][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172959, log_task={header:{begin_lsn:{lsn:14352351229}, end_lsn:{lsn:14352351351}, log_id:115172959, min_scn:{val:1729061842627966000, v:0}, max_scn:{val:1729061842627966000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352351107}, prev_proposal_id:17, committed_end_lsn:{lsn:14352351229}, data_checksum:940009244, accum_checksum:2410625654, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061842717893, freeze_ts:1729061842717893, submit_ts:1729061842717898, flushed_ts:1729061842718169, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:271}, fs_cb_begin_ts=1729061842818153, log_life_time=100260) [2024-10-16 14:57:22.819386] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656762, miss_cnt:17, cache_read_size:8050141694, hit_ratio:"9.999997286805949459e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:22.819833] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513905014493184},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223245} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.819897] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223245} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.821036] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223246}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.821061] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223246}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.821069] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223246}, start_ts=1729061842820861, retries=0, savepoint={branch:0, seq:4}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223246}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:7}, op_sn:3, alloc_ts:1729061842820021, active_ts:1729061842820021, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442820021, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333805712160, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842820020, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.822711] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513905027076096},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223247} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.822783] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223247} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.824010] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223248}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.824073] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392223248}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.824087] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223248}, start_ts=1729061842823793, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223248}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842823187, active_ts:1729061842823187, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442823187, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333808635039, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842823186, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.826896] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24] handle_sp_rollback_resp(tx_id={txid:392223249}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.826929] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=43] all savepoint rollback succeed(tx.tx_id={txid:392223249}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.826936] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223249}, start_ts=1729061842826653, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223249}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1024}, op_sn:3, alloc_ts:1729061842825303, active_ts:1729061842826332, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442826332, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333811507448, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842825308, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.827656] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=236, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:22.827675] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=38, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:22.829644] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223250}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.829748] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30] all savepoint rollback succeed(tx.tx_id={txid:392223250}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.829759] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223250}, start_ts=1729061842829480, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223250}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842828420, active_ts:1729061842828420, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442828420, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333814313805, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842828421, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.831989] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223251}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.832015] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30] all savepoint rollback succeed(tx.tx_id={txid:392223251}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.832023] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223251}, start_ts=1729061842831798, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223251}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1002}, op_sn:3, alloc_ts:1729061842830544, active_ts:1729061842831551, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442831551, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333816644824, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842830549, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.834506] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048120524800},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202911} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.835025] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2324}, to_seq_no={branch:0, seq:11}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202911} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.835158] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28] handle_sp_rollback_resp(tx_id={txid:392223252}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.835311] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31] all savepoint rollback succeed(tx.tx_id={txid:392223252}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.835323] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223252}, start_ts=1729061842834729, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223252}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1056}, op_sn:3, alloc_ts:1729061842832611, active_ts:1729061842833671, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442833671, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008958562725, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842832616, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.841228] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=20, replace_time=0, replace_start_pos=1352651, replace_num=31457) [2024-10-16 14:57:22.841248] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=20, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:22.844078] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223253}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.844148] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223253}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.844169] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223253}, start_ts=1729061842840547, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223253}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1074}, op_sn:3, alloc_ts:1729061842838162, active_ts:1729061842839237, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442839237, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008964270651, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842838163, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.845260] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061842844124000, start_id=1729061842844124000, end_id=1729061842844124001) [2024-10-16 14:57:22.849939] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3251, mem_used=6239232) [2024-10-16 14:57:22.851435] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392223254}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.851471] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223254}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.851493] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223254}, start_ts=1729061842851247, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223254}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842850293, active_ts:1729061842850293, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442850293, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333836091856, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842850292, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.853030] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=23] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=122, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:22.853472] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561028, miss_cnt:1, cache_read_size:152590510018, hit_ratio:"9.999999928346758882e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:22.855789] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223255}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.855854] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223255}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.855863] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223255}, start_ts=1729061842854395, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223255}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:8}, op_sn:3, alloc_ts:1729061842853383, active_ts:1729061842853383, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442853383, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333838889717, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842853382, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.857829] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:22.857865] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830050523860992},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223256} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.857934] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223256} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.858153] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=90, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:22.858169] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=90, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:22.858441] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=90, stat_interval=1000000, avg (count/cost)=270, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:22.858456] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=146141) [2024-10-16 14:57:22.858465] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=90, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:22.858475] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=146141) [2024-10-16 14:57:22.858486] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=90, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:22.859174] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223257}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.859200] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392223257}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.859213] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223257}, start_ts=1729061842859003, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223257}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1062}, op_sn:3, alloc_ts:1729061842858617, active_ts:1729061842858617, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442858617, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333843863801, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842857555, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.860579] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2ADB-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:22.860697] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=175, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:22.860714] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:22.860743] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:22.861990] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] handle_sp_rollback_resp(tx_id={txid:392223258}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.862018] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=50] all savepoint rollback succeed(tx.tx_id={txid:392223258}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.862025] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223258}, start_ts=1729061842861546, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223258}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842860744, active_ts:1729061842860744, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442860744, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008985920289, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842860743, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.862462] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.867175] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=46] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:22.867790] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:22.867840] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061837234027000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:22.868182] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223259}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.868336] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28] all savepoint rollback succeed(tx.tx_id={txid:392223259}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.868346] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223259}, start_ts=1729061842866932, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223259}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842866040, active_ts:1729061842866040, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442866040, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042008991144902, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842864981, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.872653] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.873576] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3979) [2024-10-16 14:57:22.873622] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3979) [2024-10-16 14:57:22.874480] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B26-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:22.874489] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B26-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=872, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:22.876583] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598583, miss_cnt:9, cache_read_size:149777695577, hit_ratio:"9.999999345923539229e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:22.877401] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545345274368001},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223260} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.877479] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223260} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.879822] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223261}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.879876] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] all savepoint rollback succeed(tx.tx_id={txid:392223261}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.879888] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223261}, start_ts=1729061842879106, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223261}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1064}, op_sn:3, alloc_ts:1729061842877819, active_ts:1729061842878882, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442878882, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009003363390, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842877818, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.881515] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601312, offset=9604160}, end_cursor:ObLogCursor{file_id=192, log_id=47601313, offset=9604444}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a4c050, len:960, log_data_len:284, seq:47601312, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:22.881904] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513899641589760},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202914} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.883367] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:22.884225] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8536}, to_seq_no={branch:0, seq:4299}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202914} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.884255] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA491-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061842884188000, start_id=1729061842884188000, end_id=1729061842884188001) [2024-10-16 14:57:22.884629] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257678, offset=39009322}, end_cursor:ObLogCursor{file_id=197, log_id=49257679, offset=39009605}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a50050, len:982, log_data_len:283, seq:49257678, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:22.886660] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB35C-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061842886410000, start_id=1729061842886410000, end_id=1729061842886410001) [2024-10-16 14:57:22.886894] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601313, offset=9604444}, end_cursor:ObLogCursor{file_id=192, log_id=47601314, offset=9604718}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a28050, len:676, log_data_len:274, seq:47601313, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:22.887762] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=25] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257679, offset=39009605}, end_cursor:ObLogCursor{file_id=197, log_id=49257680, offset=39009887}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ab8050, len:699, log_data_len:282, seq:49257679, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:22.889666] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513905152905216},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223262} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.889735] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223262} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.890341] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601314, offset=9604718}, end_cursor:ObLogCursor{file_id=192, log_id=47601315, offset=9604992}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ae8050, len:402, log_data_len:274, seq:47601314, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:22.890758] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257680, offset=39009887}, end_cursor:ObLogCursor{file_id=197, log_id=49257681, offset=39010169}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b5c050, len:417, log_data_len:282, seq:49257680, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:22.891039] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=31] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601315, offset=9604992}, end_cursor:ObLogCursor{file_id=192, log_id=47601316, offset=9605266}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c38050, len:4224, log_data_len:274, seq:47601315, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:22.891121] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257681, offset=39010169}, end_cursor:ObLogCursor{file_id=197, log_id=49257682, offset=39010451}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ad4050, len:4231, log_data_len:282, seq:49257681, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:22.891164] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=35] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650029, miss_cnt:32, cache_read_size:8050394557, hit_ratio:"9.999994892263552870e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:22.891763] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD0-0-0] [lt=15] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061842859554, keepalive_ts:-1}) [2024-10-16 14:57:22.891777] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD0-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061842859554, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:22.891953] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23] handle_sp_rollback_resp(tx_id={txid:392223263}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.891982] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392223263}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.891991] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223263}, start_ts=1729061842891590, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223263}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842890382, active_ts:1729061842891440, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442891440, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009015869916, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842890381, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.895452] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86D-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:22.895471] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86D-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061842886410000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061842886410000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:22.895973] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061842894653000, start_id=1729061842894653000, end_id=1729061842894653001) [2024-10-16 14:57:22.896636] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:22.897699] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA86D-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061842897409000, start_id=1729061842897409000, end_id=1729061842897409001) [2024-10-16 14:57:22.897983] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061832896914, end_time=1729061842897982, dag_status={}) [2024-10-16 14:57:22.897996] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:22.898003] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:22.898007] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:22.898010] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:22.898014] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:22.898017] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:22.898021] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:22.898024] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:22.898027] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:22.898030] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898036] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898040] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898044] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898048] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898052] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898056] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898060] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898063] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898067] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898071] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898075] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898079] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898083] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898087] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898091] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898095] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898098] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898102] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898106] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898110] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898113] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898117] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898121] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898125] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898128] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898132] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898136] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898140] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898143] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898147] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898151] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898155] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898159] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898162] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898166] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898170] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898174] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898177] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898181] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898185] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898188] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898192] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898196] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898200] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898203] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898207] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898211] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898215] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898226] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898230] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898233] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898237] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898241] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898245] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898248] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898252] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898256] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:22.898260] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:22.898263] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:22.898267] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:22.898269] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:22.898272] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:22.898274] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:22.898277] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:22.898279] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:22.898282] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:22.898284] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:22.898287] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:22.899632] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223264}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.899666] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223264}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.899678] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223264}, start_ts=1729061842899348, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223264}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1859}, op_sn:3, alloc_ts:1729061842897409, active_ts:1729061842899267, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442899267, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333883337197, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842897408, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.899838] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061842899267000, start_id=1729061842899267000, end_id=1729061842899267001) [2024-10-16 14:57:22.902722] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513905190653952},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223265} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.902789] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223265} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.903215] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=25] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 610.00; ) [2024-10-16 14:57:22.903235] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:22.903280] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 245.75; ) [2024-10-16 14:57:22.903290] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:22.903301] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:22.903862] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA86D-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061842903534000, start_id=1729061842903534000, end_id=1729061842903534001) [2024-10-16 14:57:22.904946] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:22.906304] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [22268][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4] handle_sp_rollback_resp(tx_id={txid:392223266}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.906409] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223266}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.906418] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223266}, start_ts=1729061842905673, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223266}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1130}, op_sn:3, alloc_ts:1729061842904667, active_ts:1729061842904667, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442904667, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009029685336, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842903537, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.912783] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1729061842912074000, start_id=1729061842912074000, end_id=1729061842912074001) [2024-10-16 14:57:22.912888] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513905203236864},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223267} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.912969] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223267} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.913858] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513905228402688},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223268} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.913930] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223268} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.914727] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910319813578752},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223269} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.914774] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223269} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.916851] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061842916318000, start_id=1729061842916318000, end_id=1729061842916318001) [2024-10-16 14:57:22.918632] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] handle_sp_rollback_resp(tx_id={txid:392223270}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.918712] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] all savepoint rollback succeed(tx.tx_id={txid:392223270}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.918726] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223270}, start_ts=1729061842916391, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223270}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2097}, op_sn:3, alloc_ts:1729061842915253, active_ts:1729061842916318, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442916318, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009040668113, first_scn:{branch:0, seq:1034}, last_scn:{branch:0, seq:1034}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842914221, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.919911] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:22.924420] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106820][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2011-0-0] [lt=18] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:22.924441] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106820][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2011-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:22.926439] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:22.926711] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048267325441},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202917} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.929334] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Cache wash clean map node details(ret=0, clean_node_count=32, clean_time=0, clean_start_pos=440398, clean_num=62914) [2024-10-16 14:57:22.932205] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7958}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202917} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.933049] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=38] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:22.933136] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:57:22.933145] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=7] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:22.933151] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:22.933155] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:22.933148] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618591604690944},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223271} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.933210] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223271} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.935369] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:57:22.935722] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=351] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:22.935915] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061842935914, bytes: 0, bw: 0.000000 MB/s, add_ts: 1014683, add_bytes: 0 [2024-10-16 14:57:22.938266] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=19] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:22.938316] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2060, mem_used=6239232) [2024-10-16 14:57:22.938949] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] handle_sp_rollback_resp(tx_id={txid:392223272}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.938980] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392223272}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.938991] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223272}, start_ts=1729061842938625, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223272}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2179}, op_sn:3, alloc_ts:1729061842936099, active_ts:1729061842938277, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442938277, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009062673775, first_scn:{branch:0, seq:1091}, last_scn:{branch:0, seq:1091}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842936098, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.941575] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061842940391000, start_id=1729061842940391000, end_id=1729061842940391001) [2024-10-16 14:57:22.942102] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=66] get number(ret=0, service_type_=0, range=1, base_id=1729061842941660000, start_id=1729061842941660000, end_id=1729061842941660001) [2024-10-16 14:57:22.942382] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061842941660000, start_id=1729061842941660000, end_id=1729061842941660001) [2024-10-16 14:57:22.943557] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061842459324000, v:0}, new_max_version={val:1729061842459324000, v:0}, last_min_version={val:1729061842446423000, v:0}, last_max_version={val:1729061842446423000, v:0}, total_time_us=2207, delta_us=484232) [2024-10-16 14:57:22.943573] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061842459324000, v:0}, min_version={val:1729061842459324000, v:0}, max_version={val:1729061842459324000, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:57:22.946189] INFO [STORAGE] refresh_queuing_mode (ob_tenant_tablet_stat_mgr.cpp:1160) [21732][T1001_TabletSta][T1001][Y0-0000000000000000-0-0] [lt=9] refresh queuing mode(ret=0, tenant_id=1001, stream_cnt=19, update_schema_cnt=19, cost_time=33) [2024-10-16 14:57:22.946202] INFO [STORAGE] runTimerTask (ob_tenant_tablet_stat_mgr.cpp:1210) [21732][T1001_TabletSta][T1001][Y0-0000000000000000-0-0] [lt=11] TenantTabletStatMgr refresh all tablet stream(MTL_ID()=1001, interval_step=1, global_iter_pool={is_inited:false, is_washing:false, is_disabled:false, get_cnt:0, tenant_id:1001, bucket_cnt:0, tenant_mem_user_limit:0, tenant_mem_user_hold:0, cached_node_array:0x2b0df73b6360}) [2024-10-16 14:57:22.946569] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:22.946616] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513899704504320},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202918} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.946958] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:22.946976] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=15] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27838040, tenant_id=1004) [2024-10-16 14:57:22.946985] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:22.947233] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2522}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202918} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.949610] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223273}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.949671] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38] all savepoint rollback succeed(tx.tx_id={txid:392223273}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.949688] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223273}, start_ts=1729061842949098, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223273}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2151}, op_sn:3, alloc_ts:1729061842947379, active_ts:1729061842948441, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442948441, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009073278958, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842946290, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.957154] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:22.957172] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:22.957180] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=155) [2024-10-16 14:57:22.957564] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392223274}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.957614] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223274}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.957622] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223274}, start_ts=1729061842957229, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223274}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842952677, active_ts:1729061842952677, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442952677, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333941911095, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842952690, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.957893] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=19] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:22.960685] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2ADD-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:22.960709] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2ADD-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:22.961026] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061842961025, bytes: 4446804219462, bw: 1.832221 MB/s, add_ts: 1001100, add_bytes: 1923336 [2024-10-16 14:57:22.961406] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=116, stat_interval=1000000, avg (count/cost)=267, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:22.961418] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=118, stat_interval=1000000, avg (count/cost)=64, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:22.961434] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=118, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:22.962544] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:22.962564] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:22.962571] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.962858] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] LogLoopThread round_cost_time(us)(round_cost_time=17) [2024-10-16 14:57:22.964369] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] handle_sp_rollback_resp(tx_id={txid:392223275}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.964414] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223275}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.964425] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223275}, start_ts=1729061842963910, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223275}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061842962172, active_ts:1729061842963230, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442963230, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009087762037, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842962171, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.968351] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=10] ====== tenant freeze timer task ====== [2024-10-16 14:57:22.969542] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C402A-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=6, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=360, avg_log_submit_to_slide_time=997) [2024-10-16 14:57:22.969687] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=43, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:57:22.969699] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=1626, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:57:22.969703] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:57:22.969707] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=-704, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:57:22.969710] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=965, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:57:22.971335] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223276}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.971375] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=40] all savepoint rollback succeed(tx.tx_id={txid:392223276}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.971392] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223276}, start_ts=1729061842970548, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223276}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061842969697, active_ts:1729061842969697, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442969697, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009094938373, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842969696, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.972732] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:22.976018] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:22.976755] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=10] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:22.976783] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=29] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:22.976801] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:22.976861] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061859029444000, latest_log_ts:{val:1729061842479541001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:22.976905] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=41] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:22.976922] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:57:22.976929] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061842479541001, v:0}) [2024-10-16 14:57:22.976933] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:22.976936] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:22.976942] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:22.976959] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=12][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:57:22.976967] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:22.976972] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:22.981220] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048372183040},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202921} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:22.981423] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] handle_sp_rollback_resp(tx_id={txid:392223277}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:22.981442] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223277}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:22.981456] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223277}, start_ts=1729061842980891, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223277}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:5691}, op_sn:3, alloc_ts:1729061842974923, active_ts:1729061842980615, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062442980615, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009105226663, first_scn:{branch:0, seq:2536}, last_scn:{branch:0, seq:2536}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061842974924, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:22.981564] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5694}, to_seq_no={branch:0, seq:2536}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202921} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:22.985660] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=23] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:22.989006] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061842988548000, start_id=1729061842988548000, end_id=1729061842988548001) [2024-10-16 14:57:22.990959] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:22.999506] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=9] worker thread began to shrink(tenant_->id()=1001, group_id_=1, token=0) [2024-10-16 14:57:23.002533] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684545345316311040},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223278} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.002619] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223278} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.003752] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106984][T1001_L0_G1][T1001][Y0-0000000000000000-0-0] [lt=7] worker thread exit(tenant_->id()=1001, workers_.get_size()=2) [2024-10-16 14:57:23.003768] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106984][T1001_L0_G1][T0][Y0-0000000000000000-0-0] [lt=11] tenant thread end_run(id_=1001, ret=0, thread_count_=171) [2024-10-16 14:57:23.003852] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=17] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:23.005706] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] handle_sp_rollback_resp(tx_id={txid:392223279}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.005753] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392223279}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.005765] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223279}, start_ts=1729061843005499, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223279}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:7}, op_sn:3, alloc_ts:1729061843004555, active_ts:1729061843004555, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443004555, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009129866368, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843004554, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.009682] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513899754835968},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202924} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.010089] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830050758742017},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223280} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.010145] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223280} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.010401] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=86, avg_log_batch_cnt=1, total_group_log_size=139649, avg_group_log_size=1623) [2024-10-16 14:57:23.010448] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=43] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=139649) [2024-10-16 14:57:23.010501] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2122}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202924} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.012022] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392223281}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.012073] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] all savepoint rollback succeed(tx.tx_id={txid:392223281}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.012088] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223281}, start_ts=1729061843011822, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223281}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1057}, op_sn:3, alloc_ts:1729061843009802, active_ts:1729061843010864, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443010864, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009136137715, first_scn:{branch:0, seq:1056}, last_scn:{branch:0, seq:1056}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843009810, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.013570] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061843012982000, start_id=1729061843012982001, end_id=1729061843012982002) [2024-10-16 14:57:23.014752] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061843014044000, start_id=1729061843014044000, end_id=1729061843014044001) [2024-10-16 14:57:23.015083] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=200, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:23.015102] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=200, stat_interval=1000000, avg (count/cost)=94, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:23.015409] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223282}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.015440] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223282}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.015466] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223282}, start_ts=1729061843015223, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223282}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1067}, op_sn:3, alloc_ts:1729061843014044, active_ts:1729061843015112, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443015112, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012333999982974, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843014045, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.015725] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048443486209},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202925} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.016041] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1074}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202925} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.017415] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061843017234000, start_id=1729061843017234000, end_id=1729061843017234001) [2024-10-16 14:57:23.017436] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=682, get_gts_with_stc_cnt=30, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=30, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:23.017446] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061843017411], gts:1729061843017234000, latest_srr:[mts=1729061843017411]}) [2024-10-16 14:57:23.017458] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:23.017472] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=924, get_gts_with_stc_cnt=889, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=522, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:23.019593] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C5C-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172961, log_task={header:{begin_lsn:{lsn:14352351473}, end_lsn:{lsn:14352351595}, log_id:115172961, min_scn:{val:1729061842627966002, v:0}, max_scn:{val:1729061842627966002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352351351}, prev_proposal_id:17, committed_end_lsn:{lsn:14352351473}, data_checksum:940009244, accum_checksum:1371593593, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061842918584, freeze_ts:1729061842918584, submit_ts:1729061842918588, flushed_ts:1729061842918768, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:180}, fs_cb_begin_ts=1729061843019587, log_life_time=101003) [2024-10-16 14:57:23.024436] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26] handle_sp_rollback_resp(tx_id={txid:392223283}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.024471] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] all savepoint rollback succeed(tx.tx_id={txid:392223283}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.024479] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223283}, start_ts=1729061843023538, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223283}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:5403}, op_sn:3, alloc_ts:1729061843017234, active_ts:1729061843022636, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443022636, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009147824517, first_scn:{branch:0, seq:4346}, last_scn:{branch:0, seq:4346}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843017233, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.025970] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061843025815000, start_id=1729061843025815000, end_id=1729061843025815001) [2024-10-16 14:57:23.027449] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:23.027467] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:23.027809] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:23.027833] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:23.030612] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223284}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.030641] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32] all savepoint rollback succeed(tx.tx_id={txid:392223284}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.030649] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223284}, start_ts=1729061843029668, retries=0, savepoint={branch:0, seq:5}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223284}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2127}, op_sn:3, alloc_ts:1729061843026958, active_ts:1729061843029084, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443029084, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009153932891, first_scn:{branch:0, seq:1066}, last_scn:{branch:0, seq:1066}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843026957, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.030836] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=91, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:23.030844] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=91, stat_interval=1000000, avg (count/cost)=3076, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:23.033117] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830048481234945},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202927} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.034046] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2121}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202927} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.035971] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18] handle_sp_rollback_resp(tx_id={txid:392223285}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.036005] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223285}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.036013] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223285}, start_ts=1729061843035739, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223285}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1077}, op_sn:3, alloc_ts:1729061843034355, active_ts:1729061843035433, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443035433, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334020538934, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843034357, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.036525] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:23.036534] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=7] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:23.036550] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291852105, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:23.038594] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830050926514176},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223286} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.038664] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223286} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.038684] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513899792584704},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202928} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.039124] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2361}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202928} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.040472] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392223287}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.040517] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392223287}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.040528] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223287}, start_ts=1729061843040267, retries=0, savepoint={branch:0, seq:5}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223287}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061843038880, active_ts:1729061843039939, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443039939, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009164521901, first_scn:{branch:0, seq:7}, last_scn:{branch:0, seq:7}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843038881, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.042733] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] Cache replace map node details(ret=0, replace_node_count=29, replace_time=0, replace_start_pos=1384108, replace_num=31457) [2024-10-16 14:57:23.042750] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=29, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:23.047505] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223288}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.048353] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21574][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:23.048421] WDIAG read_multi_row (ob_table_access_helper.h:238) [21574][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:23.049205] INFO delete_row (ob_table_access_helper.h:312) [21574][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061843048431) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061843048431) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:23.049412] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223288}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.049425] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223288}, start_ts=1729061843047287, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223288}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061843046366, active_ts:1729061843046366, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443046366, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334032074411, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843046365, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.052032] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618591919263744},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223289} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.052102] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223289} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.054908] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=39] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:23.055517] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:23.055613] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=49] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061840187603000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:23.056711] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223290}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.056748] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] all savepoint rollback succeed(tx.tx_id={txid:392223290}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.056762] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223290}, start_ts=1729061843054216, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223290}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1018}, op_sn:3, alloc_ts:1729061843052892, active_ts:1729061843053909, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443053909, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334038782062, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843052891, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.062612] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.063478] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] handle_sp_rollback_resp(tx_id={txid:392223291}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.063508] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223291}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.063528] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223291}, start_ts=1729061843063032, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223291}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1059}, op_sn:3, alloc_ts:1729061843061064, active_ts:1729061843062122, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443062122, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009187235387, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843061063, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.067592] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG IO WORKER QUEUE SIZE](cur_stat_count=4, stat_interval=1000000, avg (count/cost)=1, this=0x2b0e0d9bb7a8, extra_info="log_io_worker_num:4, cb_thread_pool_tg_id:485") [2024-10-16 14:57:23.068410] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=14][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5024, dropped:364, tid:123287}]) [2024-10-16 14:57:23.069877] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [22268][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223292}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.070026] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=37] all savepoint rollback succeed(tx.tx_id={txid:392223292}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.070039] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223292}, start_ts=1729061843069288, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223292}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1072}, op_sn:3, alloc_ts:1729061843067391, active_ts:1729061843068463, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443068463, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009193422811, first_scn:{branch:0, seq:8}, last_scn:{branch:0, seq:8}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843067393, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.072760] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.076953] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4809-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061843044205, keepalive_ts:-1}) [2024-10-16 14:57:23.076972] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4809-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061843044205, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:23.077612] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223293}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.077648] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223293}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.077656] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223293}, start_ts=1729061843077289, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223293}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061843076231, active_ts:1729061843076231, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443076231, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009201620005, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843076230, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.080616] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f4732d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202934} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, prev=this=0x2b0e5abf9408 trans_version={val:1729061780153229000, v:0} scn={val:1729061780153229000, v:0} tx_id={txid:392201898} prev=0x2b0e5abf92c8 next=(nil) modify_count=1 acc_checksum=1218158122 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7049}, snapshot_version={val:1729061843079439000, v:0}, *this={this=0x2b0e5abf8fc0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abf9408 latest_compact_node=(nil) max_trans_version={val:1729061780153229000, v:0} max_trans_id=392201898 max_elr_trans_version={val:1729061780153229000, v:0} max_elr_trans_id=392201898 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780153229000, v:0} min_modify_scn={val:1729061780153229000, v:0}}) [2024-10-16 14:57:23.080650] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f4732d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202934} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606266, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606266, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843079439000, v:0}, tx_id:{txid:392202934}, scn:{branch:0, seq:1057}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202934}, tx_desc:{this:0x2b0ec37b9cd0, tx_id:{txid:392202934}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843080510, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843078383, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1060}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.080709] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202934} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202934}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:1060}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.080726] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513899842916352},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202934} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.080741] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.080747] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513899842916352},{"BIGINT":1729058400},{"VARCHAR":"A0Ä", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.080796] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.081477] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2128}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202934} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.083634] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31] handle_sp_rollback_resp(tx_id={txid:392223294}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.083815] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223294}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.083829] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223294}, start_ts=1729061843082786, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223294}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061843081577, active_ts:1729061843082634, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443082634, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009207165085, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843081576, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.090877] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223295}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.090922] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=57] all savepoint rollback succeed(tx.tx_id={txid:392223295}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.090933] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223295}, start_ts=1729061843090663, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223295}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1066}, op_sn:3, alloc_ts:1729061843088991, active_ts:1729061843090054, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443090054, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334074622426, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843088990, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.095839] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb2c770 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202935} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:8483}, prev=this=0x2b0fadabbff0 trans_version={val:1729061780156411000, v:0} scn={val:1729061780156411000, v:0} tx_id={txid:392201899} prev=0x2b0fadabbeb0 next=(nil) modify_count=1 acc_checksum=442080172 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3178}, snapshot_version={val:1729061843087932000, v:0}, *this={this=0x2b0fadabbba8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadabbff0 latest_compact_node=(nil) max_trans_version={val:1729061780156411000, v:0} max_trans_id=392201899 max_elr_trans_version={val:1729061780156411000, v:0} max_elr_trans_id=392201899 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780156411000, v:0} min_modify_scn={val:1729061780156411000, v:0}}) [2024-10-16 14:57:23.095881] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=40][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb2c770 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202935} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:8483}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606294, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606294, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843087932000, v:0}, tx_id:{txid:392202935}, scn:{branch:0, seq:6356}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202935}, tx_desc:{this:0x2b0ec37d1850, tx_id:{txid:392202935}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843095357, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843086875, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:8483}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.095939] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202935} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202935}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:8483}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.095954] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830048674172928},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202935} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.095966] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.095972] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048674172928},{"BIGINT":1729058400},{"VARCHAR":"B6ëL‘ê", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.096023] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.096056] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061843095357000, start_id=1729061843095357000, end_id=1729061843095357001) [2024-10-16 14:57:23.098263] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10581}, to_seq_no={branch:0, seq:6357}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202935} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.098309] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223296}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.098332] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223296}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.098344] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223296}, start_ts=1729061843096279, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223296}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3192}, op_sn:3, alloc_ts:1729061843092165, active_ts:1729061843095357, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443095357, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009219653705, first_scn:{branch:0, seq:1063}, last_scn:{branch:0, seq:1063}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843092168, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.099095] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:23.099112] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=152413380, tenant_id=1) [2024-10-16 14:57:23.099124] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:23.102089] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=29][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f4739b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223297} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f40f720 trans_version={val:1729061782101806000, v:0} scn={val:1729061782101806000, v:0} tx_id={txid:392222674} prev=0x2b0f3f40f5e0 next=(nil) modify_count=1 acc_checksum=2616557707 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843101724000, v:0}, *this={this=0x2b0f3f40f2d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f40f720 latest_compact_node=(nil) max_trans_version={val:1729061782101806000, v:0} max_trans_id=392222674 max_elr_trans_version={val:1729061782101806000, v:0} max_elr_trans_id=392222674 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782101806000, v:0} min_modify_scn={val:1729061782101806000, v:0}}) [2024-10-16 14:57:23.102122] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f4739b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223297} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843101724000, v:0}, tx_id:{txid:392223297}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223297}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223297}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843101724, active_ts:1729061843101724, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843101723, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.102177] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223297} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223297}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.102194] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654910319939407872},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223297} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.102209] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.102215] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910319939407872},{"BIGINT":1729058400},{"VARCHAR":"AóÇD", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.102261] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.102287] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223297} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.103979] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f53cd38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202936} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3142}, prev=this=0x2b0f7f4b47d8 trans_version={val:1729061780160613000, v:0} scn={val:1729061780160613000, v:0} tx_id={txid:392201900} prev=0x2b0f7f4b4698 next=(nil) modify_count=1 acc_checksum=1700754848 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3143}, snapshot_version={val:1729061843100667000, v:0}, *this={this=0x2b0f7f4b4390 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4b47d8 latest_compact_node=(nil) max_trans_version={val:1729061780160613000, v:0} max_trans_id=392201900 max_elr_trans_version={val:1729061780160613000, v:0} max_elr_trans_id=392201900 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780160613000, v:0} min_modify_scn={val:1729061780160613000, v:0}}) [2024-10-16 14:57:23.104009] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f53cd38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202936} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3142}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606256, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606256, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843100667000, v:0}, tx_id:{txid:392202936}, scn:{branch:0, seq:2084}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202936}, tx_desc:{this:0x2b0ec37b1e50, tx_id:{txid:392202936}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843103808, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843100667, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:3142}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.104059] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=50][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202936} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202936}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:3142}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.104076] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":675513899855499264},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202936} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.104090] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.104095] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513899855499264},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.104145] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.104379] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3145}, to_seq_no={branch:0, seq:2085}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202936} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.106612] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223298}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.106642] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223298}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.106651] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223298}, start_ts=1729061843106396, retries=0, savepoint={branch:0, seq:5}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223298}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3138}, op_sn:3, alloc_ts:1729061843102750, active_ts:1729061843105886, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443105886, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334091048691, first_scn:{branch:0, seq:9}, last_scn:{branch:0, seq:9}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843102749, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.110279] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:23.110294] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:23.112978] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15CC-0-0] [lt=15] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:23.113138] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223299}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.113202] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35] all savepoint rollback succeed(tx.tx_id={txid:392223299}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.113212] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223299}, start_ts=1729061843109754, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223299}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1060}, op_sn:3, alloc_ts:1729061843108005, active_ts:1729061843109064, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443109064, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009233702529, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843108004, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.117867] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061843117009000, start_id=1729061843117009001, end_id=1729061843117009002) [2024-10-16 14:57:23.117960] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb2ce50 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223300} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadacdef0 trans_version={val:1729061782107204001, v:0} scn={val:1729061782107204001, v:0} tx_id={txid:392222677} prev=0x2b0fadacddb0 next=(nil) modify_count=1 acc_checksum=3448391628 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843117009001, v:0}, *this={this=0x2b0fadacdaa8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadacdef0 latest_compact_node=(nil) max_trans_version={val:1729061782107204001, v:0} max_trans_id=392222677 max_elr_trans_version={val:1729061782107204001, v:0} max_elr_trans_id=392222677 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782107204001, v:0} min_modify_scn={val:1729061782107204001, v:0}}) [2024-10-16 14:57:23.117990] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb2ce50 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223300} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843117009001, v:0}, tx_id:{txid:392223300}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223300}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223300}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843117009, active_ts:1729061843117009, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843117010, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.118045] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223300} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223300}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.118062] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830051064926208},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223300} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.118076] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.118082] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051064926208},{"BIGINT":1729058400},{"VARCHAR":"AÙ≀", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.118134] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.118167] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223300} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.121876] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15] handle_sp_rollback_resp(tx_id={txid:392223301}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.121918] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223301}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.121930] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223301}, start_ts=1729061843121056, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223301}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:2094}, op_sn:3, alloc_ts:1729061843118066, active_ts:1729061843120159, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443120159, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009245420284, first_scn:{branch:0, seq:1036}, last_scn:{branch:0, seq:1036}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843118065, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.122191] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=13] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97923870720) [2024-10-16 14:57:23.123385] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:23.124925] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=10] current monitor number(seq_=-1) [2024-10-16 14:57:23.127142] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0B9C-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061843126496000, start_id=1729061843126496000, end_id=1729061843126496001) [2024-10-16 14:57:23.127222] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061843126496000, start_id=1729061843126496000, end_id=1729061843126496001) [2024-10-16 14:57:23.127226] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0B9D-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061843126496000, start_id=1729061843126496000, end_id=1729061843126496001) [2024-10-16 14:57:23.127365] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BA0-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061843126496000, start_id=1729061843126496000, end_id=1729061843126496001) [2024-10-16 14:57:23.127530] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:23.127561] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:23.127588] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:23.127594] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:23.127607] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0B9F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061843127570000, start_id=1729061843127570000, end_id=1729061843127570001) [2024-10-16 14:57:23.127846] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=489, mem_used=2079744) [2024-10-16 14:57:23.127865] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:23.128207] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB APPEND COST TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=22, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:57:23.128217] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=2406, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:57:23.128220] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:57:23.128224] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=-1034, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:57:23.128227] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=1394, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:57:23.129645] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:23.130411] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21] handle_sp_rollback_resp(tx_id={txid:392223302}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.130443] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=46] all savepoint rollback succeed(tx.tx_id={txid:392223302}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.130455] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223302}, start_ts=1729061843129239, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223302}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061843128612, active_ts:1729061843128612, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443128612, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009253540161, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843128611, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.133997] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C9D-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510311002390, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410515, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:23.141943] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061843140226000, start_id=1729061843140226000, end_id=1729061843140226001) [2024-10-16 14:57:23.142011] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=11240, avg_group_log_size=624) [2024-10-16 14:57:23.142028] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10769) [2024-10-16 14:57:23.142466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061843142193000, start_id=1729061843142193000, end_id=1729061843142193001) [2024-10-16 14:57:23.142903] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061843142193000, start_id=1729061843142193000, end_id=1729061843142193001) [2024-10-16 14:57:23.143140] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=339, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:23.143152] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11240) [2024-10-16 14:57:23.143161] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=126, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:23.143206] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10769) [2024-10-16 14:57:23.143237] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:23.143642] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B0E-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:23.143655] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B0E-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=1182, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:23.143659] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B0E-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=76, avg_log_gen_to_submit_time=102, avg_log_submit_to_flush_time=479, avg_log_submit_to_slide_time=1079) [2024-10-16 14:57:23.144606] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223303}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.144653] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34] all savepoint rollback succeed(tx.tx_id={txid:392223303}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.144664] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223303}, start_ts=1729061843144306, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223303}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1167}, op_sn:3, alloc_ts:1729061843143359, active_ts:1729061843143359, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443143359, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334128743369, first_scn:{branch:0, seq:1166}, last_scn:{branch:0, seq:1166}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843142195, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.146964] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ee9338 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223304} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f22eaa660 trans_version={val:1729061782122269001, v:0} scn={val:1729061782122269001, v:0} tx_id={txid:392222681} prev=0x2b0f22eaa520 next=(nil) modify_count=1 acc_checksum=4260134203 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843146564000, v:0}, *this={this=0x2b0f22eaa218 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22eaa660 latest_compact_node=(nil) max_trans_version={val:1729061782122269001, v:0} max_trans_id=392222681 max_elr_trans_version={val:1729061782122269001, v:0} max_elr_trans_id=392222681 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782122269001, v:0} min_modify_scn={val:1729061782122269001, v:0}}) [2024-10-16 14:57:23.147003] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ee9338 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223304} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209267}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843146564000, v:0}, tx_id:{txid:392223304}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223304}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223304}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843146564, active_ts:1729061843146564, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843146563, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.147064] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223304} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223304}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.147079] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":684618592435163136},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223304} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.147091] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.147097] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618592435163136},{"BIGINT":1729058400},{"VARCHAR":"Aòžp¬p", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.147152] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.147182] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223304} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.149060] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223305}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.149128] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392223305}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.149144] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223305}, start_ts=1729061843148573, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223305}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061843147651, active_ts:1729061843147651, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443147651, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334133159929, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843147650, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.151279] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=44][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be078d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223306} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d84c78 trans_version={val:1729061782126493001, v:0} scn={val:1729061782126493001, v:0} tx_id={txid:392222683} prev=0x2b0ec6d84b38 next=(nil) modify_count=1 acc_checksum=2338144557 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1053}, snapshot_version={val:1729061843150789000, v:0}, *this={this=0x2b0ec6d84830 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d84c78 latest_compact_node=(nil) max_trans_version={val:1729061782126493001, v:0} max_trans_id=392222683 max_elr_trans_version={val:1729061782126493001, v:0} max_elr_trans_id=392222683 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782126493001, v:0} min_modify_scn={val:1729061782126493001, v:0}}) [2024-10-16 14:57:23.151312] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be078d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223306} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843150789000, v:0}, tx_id:{txid:392223306}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223306}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223306}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843150789, active_ts:1729061843150789, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843150788, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.151369] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223306} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223306}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.151386] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830051140423680},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223306} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.151398] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.151403] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051140423680},{"BIGINT":1729058400},{"VARCHAR":"?ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.151460] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.151487] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223306} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.152987] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223307}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.153047] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14] all savepoint rollback succeed(tx.tx_id={txid:392223307}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.153063] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223307}, start_ts=1729061843152779, retries=0, savepoint={branch:0, seq:4}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223307}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:7}, op_sn:3, alloc_ts:1729061843151932, active_ts:1729061843151932, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443151932, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334137428002, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843151931, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.155178] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f53dd30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223308} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f7f4d3220 trans_version={val:1729061782129689000, v:0} scn={val:1729061782129689000, v:0} tx_id={txid:392222685} prev=0x2b0f7f4d30e0 next=(nil) modify_count=1 acc_checksum=3978429105 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843155118000, v:0}, *this={this=0x2b0f7f4d2dd8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4d3220 latest_compact_node=(nil) max_trans_version={val:1729061782129689000, v:0} max_trans_id=392222685 max_elr_trans_version={val:1729061782129689000, v:0} max_elr_trans_id=392222685 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782129689000, v:0} min_modify_scn={val:1729061782129689000, v:0}}) [2024-10-16 14:57:23.155219] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f53dd30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223308} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209285}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843155118000, v:0}, tx_id:{txid:392223308}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223308}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223308}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843154047, active_ts:1729061843155118, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843154050, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.155280] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223308} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223308}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.155297] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":684545345530220544},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223308} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.155311] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.155320] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545345530220544},{"BIGINT":1729058400},{"VARCHAR":"?ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.155365] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.155396] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223308} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.158745] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24] handle_sp_rollback_resp(tx_id={txid:392223309}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.158802] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392223309}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.158814] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223309}, start_ts=1729061843157689, retries=0, savepoint={branch:0, seq:4}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223309}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1051}, op_sn:3, alloc_ts:1729061843156141, active_ts:1729061843157191, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443157191, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009282015698, first_scn:{branch:0, seq:6}, last_scn:{branch:0, seq:6}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843156140, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.162628] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.164939] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=46] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:23.166911] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5019-0-0] [lt=8] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:23.166931] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5019-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:23.169128] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ee9a18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223310} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0f22eaad60 trans_version={val:1729061782132878001, v:0} scn={val:1729061782132878001, v:0} tx_id={txid:392222687} prev=0x2b0f22eaac20 next=(nil) modify_count=1 acc_checksum=127404800 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843168036000, v:0}, *this={this=0x2b0f22eaa918 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22eaad60 latest_compact_node=(nil) max_trans_version={val:1729061782132878001, v:0} max_trans_id=392222687 max_elr_trans_version={val:1729061782132878001, v:0} max_elr_trans_id=392222687 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782132878001, v:0} min_modify_scn={val:1729061782132878001, v:0}}) [2024-10-16 14:57:23.169164] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ee9a18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223310} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209267}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843168036000, v:0}, tx_id:{txid:392223310}, scn:{branch:0, seq:5}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223310}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223310}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843168036, active_ts:1729061843168036, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843168038, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.169221] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223310} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223310}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.169235] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654830051178172416},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223310} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.169247] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.169252] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051178172416},{"BIGINT":1729058400},{"VARCHAR":"AÐ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.169298] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.169323] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223310} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.170342] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be07fb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223311} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5}, prev=this=0x2b0ec6d85378 trans_version={val:1729061782133939001, v:0} scn={val:1729061782133939001, v:0} tx_id={txid:392222688} prev=0x2b0ec6d85238 next=(nil) modify_count=1 acc_checksum=1030331572 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843169562001, v:0}, *this={this=0x2b0ec6d84f30 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d85378 latest_compact_node=(nil) max_trans_version={val:1729061782133939001, v:0} max_trans_id=392222688 max_elr_trans_version={val:1729061782133939001, v:0} max_elr_trans_id=392222688 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782133939001, v:0} min_modify_scn={val:1729061782133939001, v:0}}) [2024-10-16 14:57:23.170367] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be07fb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223311} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843169562001, v:0}, tx_id:{txid:392223311}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223311}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223311}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843169562, active_ts:1729061843169562, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843169563, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.170410] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=44][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223311} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223311}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:5}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.170423] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=0] primary key duplication conflict(key={"BIGINT":654830051203338240},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223311} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.170433] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.170438] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051203338240},{"BIGINT":1729058400},{"VARCHAR":"Aà", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.170482] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.170501] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223311} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.171444] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f474098 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223312} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f40fe20 trans_version={val:1729061782134997001, v:0} scn={val:1729061782134997001, v:0} tx_id={txid:392222689} prev=0x2b0f3f40fce0 next=(nil) modify_count=1 acc_checksum=2710601288 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843170621001, v:0}, *this={this=0x2b0f3f40f9d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f40fe20 latest_compact_node=(nil) max_trans_version={val:1729061782134997001, v:0} max_trans_id=392222689 max_elr_trans_version={val:1729061782134997001, v:0} max_elr_trans_id=392222689 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782134997001, v:0} min_modify_scn={val:1729061782134997001, v:0}}) [2024-10-16 14:57:23.171471] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f474098 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223312} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843170621001, v:0}, tx_id:{txid:392223312}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223312}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223312}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843170621, active_ts:1729061843170621, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843170622, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.171515] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=44][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223312} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223312}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.171528] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=0] primary key duplication conflict(key={"BIGINT":654910320002322432},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223312} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.171541] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.171545] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910320002322432},{"BIGINT":1729058400},{"VARCHAR":"Aà", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.171580] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.171615] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223312} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.172446] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb2d530 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223313} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadace5f0 trans_version={val:1729061782136016001, v:0} scn={val:1729061782136016001, v:0} tx_id={txid:392222690} prev=0x2b0fadace4b0 next=(nil) modify_count=1 acc_checksum=889577986 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843171704001, v:0}, *this={this=0x2b0fadace1a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadace5f0 latest_compact_node=(nil) max_trans_version={val:1729061782136016001, v:0} max_trans_id=392222690 max_elr_trans_version={val:1729061782136016001, v:0} max_elr_trans_id=392222690 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782136016001, v:0} min_modify_scn={val:1729061782136016001, v:0}}) [2024-10-16 14:57:23.172478] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb2d530 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223313} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843171704001, v:0}, tx_id:{txid:392223313}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223313}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223313}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843171704, active_ts:1729061843171704, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843171703, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.172530] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=51][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223313} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223313}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.172549] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":684545345555386368},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223313} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.172562] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.172567] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545345555386368},{"BIGINT":1729058400},{"VARCHAR":"Aô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.172619] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.172646] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223313} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.172822] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.173653] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be08698 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223314} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6}, prev=this=0x2b0ec6d85a78 trans_version={val:1729061782137080000, v:0} scn={val:1729061782137080000, v:0} tx_id={txid:392222691} prev=0x2b0ec6d85938 next=(nil) modify_count=1 acc_checksum=3806726290 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843172765001, v:0}, *this={this=0x2b0ec6d85630 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d85a78 latest_compact_node=(nil) max_trans_version={val:1729061782137080000, v:0} max_trans_id=392222691 max_elr_trans_version={val:1729061782137080000, v:0} max_elr_trans_id=392222691 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782137080000, v:0} min_modify_scn={val:1729061782137080000, v:0}}) [2024-10-16 14:57:23.173680] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=26][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be08698 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223314} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843172765001, v:0}, tx_id:{txid:392223314}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223314}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223314}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843172765, active_ts:1729061843172765, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843172764, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:6}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.173748] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=68][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223314} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223314}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:6}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.173765] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":684618592774901760},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223314} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.173786] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.173796] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618592774901760},{"BIGINT":1729058400},{"VARCHAR":"Aô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.173839] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.173864] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223314} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.174637] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f53e410 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223315} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f7f4d3920 trans_version={val:1729061782138139000, v:0} scn={val:1729061782138139000, v:0} tx_id={txid:392222692} prev=0x2b0f7f4d37e0 next=(nil) modify_count=1 acc_checksum=4016328069 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:9}, snapshot_version={val:1729061843173823001, v:0}, *this={this=0x2b0f7f4d34d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4d3920 latest_compact_node=(nil) max_trans_version={val:1729061782138139000, v:0} max_trans_id=392222692 max_elr_trans_version={val:1729061782138139000, v:0} max_elr_trans_id=392222692 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782138139000, v:0} min_modify_scn={val:1729061782138139000, v:0}}) [2024-10-16 14:57:23.174662] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f53e410 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223315} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209285}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843173823001, v:0}, tx_id:{txid:392223315}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223315}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223315}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843173823, active_ts:1729061843173823, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843173822, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.174719] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223315} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223315}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.174734] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830051228504064},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223315} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.174751] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.174756] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051228504064},{"BIGINT":1729058400},{"VARCHAR":"@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.174797] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.174823] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223315} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.177083] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] handle_sp_rollback_resp(tx_id={txid:392223316}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.177117] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223316}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.177159] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223316}, start_ts=1729061843176886, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223316}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6}, op_sn:3, alloc_ts:1729061843175916, active_ts:1729061843175916, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443175916, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334161528120, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843175915, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.179316] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f53eaf0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223317} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f7f4d4020 trans_version={val:1729061782141618001, v:0} scn={val:1729061782141618001, v:0} tx_id={txid:392222694} prev=0x2b0f7f4d3ee0 next=(nil) modify_count=1 acc_checksum=933622805 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843179124000, v:0}, *this={this=0x2b0f7f4d3bd8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4d4020 latest_compact_node=(nil) max_trans_version={val:1729061782141618001, v:0} max_trans_id=392222694 max_elr_trans_version={val:1729061782141618001, v:0} max_elr_trans_id=392222694 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782141618001, v:0} min_modify_scn={val:1729061782141618001, v:0}}) [2024-10-16 14:57:23.179348] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f53eaf0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223317} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209285}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843179124000, v:0}, tx_id:{txid:392223317}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223317}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223317}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843179124, active_ts:1729061843179124, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843179123, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.179403] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223317} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223317}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.179419] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654910320014905344},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223317} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.179434] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.179439] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910320014905344},{"BIGINT":1729058400},{"VARCHAR":"?ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.179500] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.179527] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223317} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.184678] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223318}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.184722] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] all savepoint rollback succeed(tx.tx_id={txid:392223318}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.184730] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223318}, start_ts=1729061843183470, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223318}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:4178}, op_sn:3, alloc_ts:1729061843180186, active_ts:1729061843183309, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443183309, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009307699425, first_scn:{branch:0, seq:1057}, last_scn:{branch:0, seq:1057}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843179131, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.186808] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f5b9217c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202943} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778513125 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3188}, prev=this=0x2b0f5b8cb7d8 trans_version={val:1729061780190089000, v:0} scn={val:1729061780190089000, v:0} tx_id={txid:392201907} prev=0x2b0f5b8cb698 next=(nil) modify_count=1 acc_checksum=2774095084 version=1729061778513125 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5289}, snapshot_version={val:1729061843184370000, v:0}, *this={this=0x2b0f5b8cb390 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f5b8cb7d8 latest_compact_node=(nil) max_trans_version={val:1729061780190089000, v:0} max_trans_id=392201907 max_elr_trans_version={val:1729061780190089000, v:0} max_elr_trans_id=392201907 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780190089000, v:0} min_modify_scn={val:1729061780190089000, v:0}}) [2024-10-16 14:57:23.186844] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f5b9217c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202943} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778513125 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3188}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606972, tablet_id:{id:209288}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606972, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843184370000, v:0}, tx_id:{txid:392202943}, scn:{branch:0, seq:1061}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202943}, tx_desc:{this:0x2b0ec37cf8b0, tx_id:{txid:392202943}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843186497, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843183310, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:3188}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.186898] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=52][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202943} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202943}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778513125, seq_no:{branch:0, seq:3188}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.186915] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830048808390657},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202943} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.186929] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.186934] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048808390657},{"BIGINT":1729058400},{"VARCHAR":"BÖIHPT›ÀBÖId'М@BÖId'М@BÖI~=þM", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.186983] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.187788] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4246}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202943} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.191664] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061843189665000, start_id=1729061843189665000, end_id=1729061843189665001) [2024-10-16 14:57:23.193687] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15C6-0-0] [lt=11] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:23.198485] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15C8-0-0] [lt=12] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:23.199632] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392223319}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.199680] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=36] all savepoint rollback succeed(tx.tx_id={txid:392223319}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.199722] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223319}, start_ts=1729061843199275, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223319}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:6466}, op_sn:3, alloc_ts:1729061843192082, active_ts:1729061843198546, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443198546, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009317614621, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843192081, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.201769] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061843201749000, start_id=1729061843201749000, end_id=1729061843201749001) [2024-10-16 14:57:23.206674] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f474778 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223320} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f410520 trans_version={val:1729061782146881002, v:0} scn={val:1729061782146881002, v:0} tx_id={txid:392222697} prev=0x2b0f3f4103e0 next=(nil) modify_count=1 acc_checksum=1192493103 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843205845000, v:0}, *this={this=0x2b0f3f4100d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f410520 latest_compact_node=(nil) max_trans_version={val:1729061782146881002, v:0} max_trans_id=392222697 max_elr_trans_version={val:1729061782146881002, v:0} max_elr_trans_id=392222697 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782146881002, v:0} min_modify_scn={val:1729061782146881002, v:0}}) [2024-10-16 14:57:23.206706] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f474778 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223320} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843205845000, v:0}, tx_id:{txid:392223320}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223320}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223320}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843205845, active_ts:1729061843205845, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843205844, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.206762] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223320} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223320}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.206777] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830051278835712},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223320} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.206789] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.206795] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051278835712},{"BIGINT":1729058400},{"VARCHAR":"AÐ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.206840] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.206872] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223320} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.208223] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb2dc10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202944} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1288}, prev=this=0x2b0fadabc6f0 trans_version={val:1729061780194046000, v:0} scn={val:1729061780194046000, v:0} tx_id={txid:392201908} prev=0x2b0fadabc5b0 next=(nil) modify_count=1 acc_checksum=3604629546 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2898}, snapshot_version={val:1729061843207130000, v:0}, *this={this=0x2b0fadabc2a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadabc6f0 latest_compact_node=(nil) max_trans_version={val:1729061780194046000, v:0} max_trans_id=392201908 max_elr_trans_version={val:1729061780194046000, v:0} max_elr_trans_id=392201908 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780194046000, v:0} min_modify_scn={val:1729061780194046000, v:0}}) [2024-10-16 14:57:23.208249] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb2dc10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202944} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1288}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606327, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606327, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843207130000, v:0}, tx_id:{txid:392202944}, scn:{branch:0, seq:7}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202944}, tx_desc:{this:0x2b0ec37c5a90, tx_id:{txid:392202944}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843207130, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843205845, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1288}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.208298] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=49][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202944} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202944}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1288}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.208311] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513899943579648},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202944} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.208322] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.208327] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513899943579648},{"BIGINT":1729058400},{"VARCHAR":"@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.208364] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.208911] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2555}, to_seq_no={branch:0, seq:8}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202944} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.214787] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:207589391218}], scn_array=[cnt:1, {val:1729061843188610000, v:0}], curr_size=1624, accum_size=146144, time_cost=19774) [2024-10-16 14:57:23.214811] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=146144) [2024-10-16 14:57:23.215144] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eea0f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223321} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f22eab460 trans_version={val:1729061782147940001, v:0} scn={val:1729061782147940001, v:0} tx_id={txid:392222698} prev=0x2b0f22eab320 next=(nil) modify_count=1 acc_checksum=4283760304 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843214871000, v:0}, *this={this=0x2b0f22eab018 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22eab460 latest_compact_node=(nil) max_trans_version={val:1729061782147940001, v:0} max_trans_id=392222698 max_elr_trans_version={val:1729061782147940001, v:0} max_elr_trans_id=392222698 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782147940001, v:0} min_modify_scn={val:1729061782147940001, v:0}}) [2024-10-16 14:57:23.215172] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eea0f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223321} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209267}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843214871000, v:0}, tx_id:{txid:392223321}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223321}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223321}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843214871, active_ts:1729061843214871, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843211621, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.215222] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=49][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223321} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223321}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.215235] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654830051304001536},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223321} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.215256] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.215262] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051304001536},{"BIGINT":1729058400},{"VARCHAR":"Aà", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.215313] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.215339] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223321} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.218867] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=16] handle_sp_rollback_resp(tx_id={txid:392223322}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.218962] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392223322}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.218971] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223322}, start_ts=1729061843218587, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223322}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1112}, op_sn:3, alloc_ts:1729061843216957, active_ts:1729061843218068, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443218068, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334203243099, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843216956, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.220967] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=159852) [2024-10-16 14:57:23.221619] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061843221288000, start_id=1729061843221288000, end_id=1729061843221288001) [2024-10-16 14:57:23.221699] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb2e2f0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223323} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0fadacecf0 trans_version={val:1729061782152187001, v:0} scn={val:1729061782152187001, v:0} tx_id={txid:392222700} prev=0x2b0fadacebb0 next=(nil) modify_count=1 acc_checksum=227209074 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843221288000, v:0}, *this={this=0x2b0fadace8a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadacecf0 latest_compact_node=(nil) max_trans_version={val:1729061782152187001, v:0} max_trans_id=392222700 max_elr_trans_version={val:1729061782152187001, v:0} max_elr_trans_id=392222700 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782152187001, v:0} min_modify_scn={val:1729061782152187001, v:0}}) [2024-10-16 14:57:23.221722] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb2e2f0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223323} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209279}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843221288000, v:0}, tx_id:{txid:392223323}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223323}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223323}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843221288, active_ts:1729061843221288, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843221287, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.221780] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223323} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223323}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.221797] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":684545345618300928},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223323} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.221809] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.221815] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545345618300928},{"BIGINT":1729058400},{"VARCHAR":"Aô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.221867] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.221892] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223323} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.222942] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be08d78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223324} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d86178 trans_version={val:1729061782153467001, v:0} scn={val:1729061782153467001, v:0} tx_id={txid:392222701} prev=0x2b0ec6d86038 next=(nil) modify_count=1 acc_checksum=1140803848 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843222493000, v:0}, *this={this=0x2b0ec6d85d30 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d86178 latest_compact_node=(nil) max_trans_version={val:1729061782153467001, v:0} max_trans_id=392222701 max_elr_trans_version={val:1729061782153467001, v:0} max_elr_trans_id=392222701 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782153467001, v:0} min_modify_scn={val:1729061782153467001, v:0}}) [2024-10-16 14:57:23.222965] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be08d78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223324} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843222493000, v:0}, tx_id:{txid:392223324}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223324}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223324}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843222493, active_ts:1729061843222493, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843222492, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.223008] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=43][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223324} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223324}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.223021] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12][errcode=0] primary key duplication conflict(key={"BIGINT":684618593026560000},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223324} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.223031] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.223036] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618593026560000},{"BIGINT":1729058400},{"VARCHAR":"Aô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.223070] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=18][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.223087] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223324} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.223855] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=159853) [2024-10-16 14:57:23.224807] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=19] handle_sp_rollback_resp(tx_id={txid:392223325}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.224859] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392223325}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.224867] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223325}, start_ts=1729061843224638, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223325}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1015}, op_sn:3, alloc_ts:1729061843223565, active_ts:1729061843224581, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443224581, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334209356899, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843223566, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.224922] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:207589392842}], scn_array=[cnt:1, {val:1729061843203829000, v:0}], curr_size=1624, accum_size=1624, time_cost=10097) [2024-10-16 14:57:23.228294] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=746, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:23.228321] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=52, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:23.228356] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=32] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:23.228361] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1244, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:23.228371] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:23.228404] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:23.228407] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1175, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:23.228411] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=794, avg_log_submit_to_slide_time=1158) [2024-10-16 14:57:23.228665] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061843227842000, start_id=1729061843227842000, end_id=1729061843227842001) [2024-10-16 14:57:23.229873] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be09458 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223326} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0ec6d86878 trans_version={val:1729061782160815000, v:0} scn={val:1729061782160815000, v:0} tx_id={txid:392222703} prev=0x2b0ec6d86738 next=(nil) modify_count=1 acc_checksum=3500201195 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843228927001, v:0}, *this={this=0x2b0ec6d86430 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d86878 latest_compact_node=(nil) max_trans_version={val:1729061782160815000, v:0} max_trans_id=392222703 max_elr_trans_version={val:1729061782160815000, v:0} max_elr_trans_id=392222703 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782160815000, v:0} min_modify_scn={val:1729061782160815000, v:0}}) [2024-10-16 14:57:23.229901] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be09458 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223326} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209261}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843228927001, v:0}, tx_id:{txid:392223326}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223326}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223326}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843228927, active_ts:1729061843228927, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843228928, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.229952] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=51][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223326} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223326}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.229967] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830051329167360},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223326} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.229984] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.229990] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830051329167360},{"BIGINT":1729058400},{"VARCHAR":"Aê£ÎBK^Aê£)ÆS&Aê£8Ào¼Aê£HNQ%FAê£WmMó¶Aê£fŠHÔ,Aê£uƒàDÐA꣄| A꣔ šžíA꣣(U)_AꣲFÛF Aê£Á?‘hAê£Ð7ÓÀAê£ßÆ+aåAê£îäúý"Aê£þäAê¤ ú¨bAê¤óJeAê¤+‚À7Aê¤: uòåAê¤I¾P½get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=5641, mem_used=10398720) [2024-10-16 14:57:23.320955] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:23.323362] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C62-0-0] [lt=30] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:23.323383] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C62-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:23.327082] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9B1-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061843326143000, start_id=1729061843326143000, end_id=1729061843326143001) [2024-10-16 14:57:23.328249] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=39] get number(ret=0, service_type_=0, range=1, base_id=1729061843327225000, start_id=1729061843327225001, end_id=1729061843327225002) [2024-10-16 14:57:23.329136] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061843328395000, start_id=1729061843328395000, end_id=1729061843328395001) [2024-10-16 14:57:23.329543] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=13] handle_sp_rollback_resp(tx_id={txid:392223333}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.329643] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392223333}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.329652] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223333}, start_ts=1729061843329273, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223333}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1174}, op_sn:3, alloc_ts:1729061843327225, active_ts:1729061843328395, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443328395, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009453630315, first_scn:{branch:0, seq:1173}, last_scn:{branch:0, seq:1173}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843327226, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.332677] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:23.332889] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1581383680, tenant_max_wash_size=7525990, tenant_min_wash_size=7525990, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:23.340625] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22] handle_sp_rollback_resp(tx_id={txid:392223334}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.340686] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54] all savepoint rollback succeed(tx.tx_id={txid:392223334}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.340696] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223334}, start_ts=1729061843340365, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223334}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:3637}, op_sn:3, alloc_ts:1729061843336268, active_ts:1729061843339904, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443339904, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334322073660, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843336267, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.341278] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:23.343408] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22235][T1002_TenantWea][T1001][YB4A0A000407-00061E9B3B0B75EF-0-0] [lt=12] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.162832403182983398e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=308, Throttle Trigger(MB)=184, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.933259662786948411e+03) [2024-10-16 14:57:23.343556] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061843342049000, start_id=1729061843342049000, end_id=1729061843342049001) [2024-10-16 14:57:23.343666] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] no need to update min/max version(new_version={val:1729061842941660000, v:0}, min_version={val:1729061842941660000, v:0}, max_version={val:1729061842941660000, v:0}) [2024-10-16 14:57:23.345217] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:23.345964] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA4-0-0] [lt=9] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:23.345978] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA4-0-0] [lt=10] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=15) [2024-10-16 14:57:23.346962] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=35][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f475538 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223335} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f411320 trans_version={val:1729061782182681001, v:0} scn={val:1729061782182681001, v:0} tx_id={txid:392222712} prev=0x2b0f3f4111e0 next=(nil) modify_count=1 acc_checksum=3986613894 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843345930001, v:0}, *this={this=0x2b0f3f410ed8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f411320 latest_compact_node=(nil) max_trans_version={val:1729061782182681001, v:0} max_trans_id=392222712 max_elr_trans_version={val:1729061782182681001, v:0} max_elr_trans_id=392222712 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782182681001, v:0} min_modify_scn={val:1729061782182681001, v:0}}) [2024-10-16 14:57:23.346995] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f475538 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223335} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843345930001, v:0}, tx_id:{txid:392223335}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223335}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223335}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843345930, active_ts:1729061843345930, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843345933, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.347050] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223335} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223335}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.347065] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":675513905379397632},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223335} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.347076] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.347081] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513905379397632},{"BIGINT":1729058400},{"VARCHAR":"AG", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.347130] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.347158] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223335} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.349253] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] rpc statistics(total_trans_msg_count=304, total_batch_msg_count=0) [2024-10-16 14:57:23.350025] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223336}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.350089] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=9] all savepoint rollback succeed(tx.tx_id={txid:392223336}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.350098] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223336}, start_ts=1729061843349233, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223336}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1061}, op_sn:3, alloc_ts:1729061843347422, active_ts:1729061843348482, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443348482, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009473558170, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843347421, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.357279] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=46][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eea7d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223337} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f22eabb60 trans_version={val:1729061782187003001, v:0} scn={val:1729061782187003001, v:0} tx_id={txid:392222714} prev=0x2b0f22eaba20 next=(nil) modify_count=1 acc_checksum=4171194676 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843357057000, v:0}, *this={this=0x2b0f22eab718 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22eabb60 latest_compact_node=(nil) max_trans_version={val:1729061782187003001, v:0} max_trans_id=392222714 max_elr_trans_version={val:1729061782187003001, v:0} max_elr_trans_id=392222714 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782187003001, v:0} min_modify_scn={val:1729061782187003001, v:0}}) [2024-10-16 14:57:23.357312] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eea7d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223337} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209267}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843357057000, v:0}, tx_id:{txid:392223337}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223337}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223337}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843357057, active_ts:1729061843357057, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843357056, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.357367] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223337} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223337}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.357382] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513905404563456},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223337} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.357394] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.357400] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513905404563456},{"BIGINT":1729058400},{"VARCHAR":"A ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.357460] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.357487] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223337} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.359793] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eeaeb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202955} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1057}, prev=this=0x2b0f22ea0028 trans_version={val:1729061780234469000, v:0} scn={val:1729061780234469000, v:0} tx_id={txid:392201919} prev=0x2b0f22e9fee8 next=(nil) modify_count=1 acc_checksum=2293356358 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4660}, snapshot_version={val:1729061843358117001, v:0}, *this={this=0x2b0f22e9fbe0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea0028 latest_compact_node=(nil) max_trans_version={val:1729061780234469000, v:0} max_trans_id=392201919 max_elr_trans_version={val:1729061780234469000, v:0} max_elr_trans_id=392201919 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780234469000, v:0} min_modify_scn={val:1729061780234469000, v:0}}) [2024-10-16 14:57:23.359821] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eeaeb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202955} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1057}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606293, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606293, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843358117001, v:0}, tx_id:{txid:392202955}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202955}, tx_desc:{this:0x2b0ec37a2150, tx_id:{txid:392202955}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843359176, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843358120, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1057}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.359872] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=50][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202955} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202955}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:1057}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.359886] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830048976162816},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202955} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.359897] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.359902] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830048976162816},{"BIGINT":1729058400},{"VARCHAR":"@åz", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.359948] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.360303] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2118}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202955} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.360558] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4] handle_sp_rollback_resp(tx_id={txid:392223338}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.360594] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11] all savepoint rollback succeed(tx.tx_id={txid:392223338}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.360608] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223338}, start_ts=1729061843359110, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223338}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:7}, op_sn:3, alloc_ts:1729061843358117, active_ts:1729061843358117, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443358117, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009483200488, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843358116, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.362772] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.364683] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061843363658000, start_id=1729061843363658000, end_id=1729061843363658001) [2024-10-16 14:57:23.369282] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] handle_sp_rollback_resp(tx_id={txid:392223339}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.369354] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=25] all savepoint rollback succeed(tx.tx_id={txid:392223339}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.369363] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223339}, start_ts=1729061843368285, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223339}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061843366896, active_ts:1729061843367953, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443367953, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009492465838, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843366895, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.371120] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f53f7e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202958} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1065}, prev=this=0x2b0f7f4b5688 trans_version={val:1729061780246289000, v:0} scn={val:1729061780246289000, v:0} tx_id={txid:392201922} prev=0x2b0f7f4b5548 next=(nil) modify_count=1 acc_checksum=2820984155 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6317}, snapshot_version={val:1729061843370069000, v:0}, *this={this=0x2b0f7f4b5240 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4b5688 latest_compact_node=(nil) max_trans_version={val:1729061780246289000, v:0} max_trans_id=392201922 max_elr_trans_version={val:1729061780246289000, v:0} max_elr_trans_id=392201922 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780246289000, v:0} min_modify_scn={val:1729061780246289000, v:0}}) [2024-10-16 14:57:23.371154] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f53f7e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202958} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1065}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606261, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606261, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843370069000, v:0}, tx_id:{txid:392202958}, scn:{branch:0, seq:1062}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202958}, tx_desc:{this:0x2b0ec37c7a30, tx_id:{txid:392202958}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843370069, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843369011, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1065}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.371214] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202958} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202958}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1065}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.371229] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513900044242944},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202958} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.371240] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.371245] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513900044242944},{"BIGINT":1729058400},{"VARCHAR":"@"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.371292] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.374492] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4512}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202958} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.378652] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f475c18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223340} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f3f411a20 trans_version={val:1729061782195604000, v:0} scn={val:1729061782195604000, v:0} tx_id={txid:392222717} prev=0x2b0f3f4118e0 next=(nil) modify_count=1 acc_checksum=1630021007 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843378176000, v:0}, *this={this=0x2b0f3f4115d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f411a20 latest_compact_node=(nil) max_trans_version={val:1729061782195604000, v:0} max_trans_id=392222717 max_elr_trans_version={val:1729061782195604000, v:0} max_elr_trans_id=392222717 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782195604000, v:0} min_modify_scn={val:1729061782195604000, v:0}}) [2024-10-16 14:57:23.378687] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f475c18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223340} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209273}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843378176000, v:0}, tx_id:{txid:392223340}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223340}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223340}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843378176, active_ts:1729061843378176, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843378175, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.378743] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223340} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223340}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.378758] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513905442312192},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223340} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.378769] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.378774] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513905442312192},{"BIGINT":1729058400},{"VARCHAR":"A", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.378821] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.378846] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223340} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.380817] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061843380452000, start_id=1729061843380452000, end_id=1729061843380452001) [2024-10-16 14:57:23.380894] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=12] handle_sp_rollback_resp(tx_id={txid:392223341}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.381127] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] all savepoint rollback succeed(tx.tx_id={txid:392223341}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.381139] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=10] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223341}, start_ts=1729061843380531, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223341}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1137}, op_sn:3, alloc_ts:1729061843379315, active_ts:1729061843380452, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443380452, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334365099915, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843379315, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.383518] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=6] query dest_info_ succ(dest_info=[], archive_dest_changed=false) [2024-10-16 14:57:23.383532] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=13] finish to resize log external storage handler(current_ts=1729061843383531, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:23.383781] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=7] gc stale ls task succ [2024-10-16 14:57:23.384377] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.386669] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=92, avg_log_batch_cnt=0, total_group_log_size=148488, avg_group_log_size=1614) [2024-10-16 14:57:23.388218] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] handle_sp_rollback_resp(tx_id={txid:392223342}, ls_id={id:1002}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.388275] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223342}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.388284] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223342}, start_ts=1729061843388027, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223342}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1058}, op_sn:3, alloc_ts:1729061843386782, active_ts:1729061843387839, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443387839, coord_id:{id:-1}, parts:[{id:{id:1002}, addr:"10.0.4.8:2890", epoch:25012334372676248, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843386781, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.389480] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A09-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061843380671, keepalive_ts:-1}) [2024-10-16 14:57:23.389514] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A09-0-0] [lt=33] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061843380671, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:23.391716] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061843389961000, start_id=1729061843389961000, end_id=1729061843389961001) [2024-10-16 14:57:23.404499] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] no need to update min/max version(new_version={val:1729061843095357000, v:0}, min_version={val:1729061843095357000, v:0}, max_version={val:1729061843095357000, v:0}) [2024-10-16 14:57:23.405100] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eeb598 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202960} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:11359}, prev=this=0x2b0f22ea0728 trans_version={val:1729061780256908000, v:0} scn={val:1729061780256908000, v:0} tx_id={txid:392201924} prev=0x2b0f22ea05e8 next=(nil) modify_count=1 acc_checksum=3900315906 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4708}, snapshot_version={val:1729061843392917000, v:0}, *this={this=0x2b0f22ea02e0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea0728 latest_compact_node=(nil) max_trans_version={val:1729061780256908000, v:0} max_trans_id=392201924 max_elr_trans_version={val:1729061780256908000, v:0} max_elr_trans_id=392201924 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780256908000, v:0} min_modify_scn={val:1729061780256908000, v:0}}) [2024-10-16 14:57:23.405130] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eeb598 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202960} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:11359}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437612773, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437612773, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843392917000, v:0}, tx_id:{txid:392202960}, scn:{branch:0, seq:2954}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202960}, tx_desc:{this:0x2b0ec37a40f0, tx_id:{txid:392202960}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843401322, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843389964, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:11359}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.405187] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202960} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202960}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:11359}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.405202] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":675513900056825856},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202960} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.405214] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.405220] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513900056825856},{"BIGINT":1729058400},{"VARCHAR":"@š4", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.405275] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.407933] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:17925}, to_seq_no={branch:0, seq:2955}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202960} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.408893] INFO [STORAGE.TRANS] handle_sp_rollback_resp (ob_trans_service_v4.cpp:2338) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=20] handle_sp_rollback_resp(tx_id={txid:392223343}, ls_id={id:1003}, status=0, downstream_parts=[]) [2024-10-16 14:57:23.408952] INFO [STORAGE.TRANS] sync_rollback_savepoint__ (ob_tx_api.cpp:1892) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33] all savepoint rollback succeed(tx.tx_id={txid:392223343}, remain_cnt=1, waittime=10000, retries=0) [2024-10-16 14:57:23.408961] INFO [STORAGE.TRANS] rollback_savepoint_slowpath_ (ob_tx_api.cpp:1791) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=7] rollback savepoint slowpath(ret=0, tx.tx_id={txid:392223343}, start_ts=1729061843408074, retries=0, savepoint={branch:0, seq:3}, expire_ts=1729062440767291, tx={this:0x2b0ec37d5790, tx_id:{txid:392223343}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:14973}, op_sn:3, alloc_ts:1729061843392917, active_ts:1729061843407888, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062443407888, coord_id:{id:-1}, parts:[{id:{id:1003}, addr:"10.0.4.9:2890", epoch:23042009525963666, first_scn:{branch:0, seq:5}, last_scn:{branch:0, seq:5}, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843392916, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, rollback_parts.count()=1) [2024-10-16 14:57:23.415332] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=33][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f53fec0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223344} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, prev=this=0x2b0f7f4d4af8 trans_version={val:1729061782211996001, v:0} scn={val:1729061782211996001, v:0} tx_id={txid:392222721} prev=0x2b0f7f4d49b8 next=(nil) modify_count=1 acc_checksum=1312108196 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, snapshot_version={val:1729061843414249000, v:0}, *this={this=0x2b0f7f4d46b0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4d4af8 latest_compact_node=(nil) max_trans_version={val:1729061782211996001, v:0} max_trans_id=392222721 max_elr_trans_version={val:1729061782211996001, v:0} max_elr_trans_id=392222721 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061782211996001, v:0} min_modify_scn={val:1729061782211996001, v:0}}) [2024-10-16 14:57:23.415365] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f53fec0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392223344} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4}, ctx={this:0x2b0e2b84d850, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062440737291, tablet_id:{id:209285}, table_iter:0x2b0e2b84c9d0, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062440737291, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843414249000, v:0}, tx_id:{txid:392223344}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392223344}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392223344}, state:1, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221535696, assoc_session_id:3221535696, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1729061843414249, active_ts:1729061843414249, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843414248, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.415420] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223344} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392223344}, data:0x2b0e2b849bf8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.415444] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=24][errcode=0] primary key duplication conflict(key={"BIGINT":684618593303384064},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392223344} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.415456] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.415462] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618593303384064},{"BIGINT":1729058400},{"VARCHAR":"A—Fþ„", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.415516] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.415542] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF228B-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392223344} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.421180] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C64-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:23.421302] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=60, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:23.421313] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:23.422895] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0EDF-0-0] [lt=12] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1002, table_id:10009, add_timestamp:1729061843422885}) [2024-10-16 14:57:23.423390] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0EDF-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510311291774, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893916, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:23.423509] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E15-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1002, table_id=10009, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061843423491, sql_renew_time:1729061843423491, is_mark_fail:false}]) [2024-10-16 14:57:23.423528] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E15-0-0] [lt=19] success to process renew task(task={tenant_id:1002, table_id:10009, add_timestamp:1729061843422885}, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061843423491, sql_renew_time:1729061843423491, is_mark_fail:false}]) [2024-10-16 14:57:23.427480] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3E-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:23.427504] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3E-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061843418289000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061843418289000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:23.428049] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:23.428111] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:23.428653] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C3E-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061843428286000, start_id=1729061843428286000, end_id=1729061843428286001) [2024-10-16 14:57:23.431316] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061843431250000, start_id=1729061843431250000, end_id=1729061843431250001) [2024-10-16 14:57:23.431499] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C3E-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061843431250000, start_id=1729061843431250000, end_id=1729061843431250001) [2024-10-16 14:57:23.440529] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] submit log success(service_type_=0, last_id=1729061860098662000, limited_id=1729061860098662000) [2024-10-16 14:57:23.440543] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] submit log with lock success(service_type_=0, last_id=1729061860098662000, limited_id=1729061860098662000) [2024-10-16 14:57:23.441102] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061860098662000, latest_scn={val:1729061843439703001, v:0}, *this={tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061840798710000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860098662000, latest_log_ts:{val:1729061843439703001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:23.441136] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=34] handle submit callback(service_type_=0, success=true, used_ts=608, log_ts={val:1729061843439703001, v:0}, limited_id=1729061860098662000) [2024-10-16 14:57:23.441144] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:23.441148] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:23.445781] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=19, replace_time=0, replace_start_pos=1447022, replace_num=31457) [2024-10-16 14:57:23.445798] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=19, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:23.447008] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061843443978000, start_id=1729061843443978000, end_id=1729061843443978001) [2024-10-16 14:57:23.449467] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=23] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:23.449489] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=20] get_min_unreplayed_log_info(lsn={lsn:14538466082}, scn={val:1729061843126496003, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061843363117, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538466082}, next_to_submit_scn_:{val:1729061843126496003, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538465960}, end_lsn:{lsn:14538466082}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061843126496002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061843126496002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:3573315511, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061843126496002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650034, miss_cnt:32, cache_read_size:8050395167, hit_ratio:"9.999994892263960322e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:23.449535] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=43] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:23.450715] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6D-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:23.450731] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6D-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:23.453294] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be0a450 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202965} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2073}, prev=this=0x2b0ec6d6a9d0 trans_version={val:1729061780284634000, v:0} scn={val:1729061780284634000, v:0} tx_id={txid:392201929} prev=0x2b0ec6d6a890 next=(nil) modify_count=1 acc_checksum=3925469721 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5588}, snapshot_version={val:1729061843450894000, v:0}, *this={this=0x2b0ec6d6a588 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d6a9d0 latest_compact_node=(nil) max_trans_version={val:1729061780284634000, v:0} max_trans_id=392201929 max_elr_trans_version={val:1729061780284634000, v:0} max_elr_trans_id=392201929 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780284634000, v:0} min_modify_scn={val:1729061780284634000, v:0}}) [2024-10-16 14:57:23.453323] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be0a450 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202965} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2073}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606262, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606262, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843450894000, v:0}, tx_id:{txid:392202965}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202965}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392202965}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843452967, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843450895, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2073}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.453379] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202965} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202965}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:2073}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.453394] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830049127157760},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202965} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.453405] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.453410] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049127157760},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.453457] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.453816] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2076}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202965} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.461792] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8FB0-0-0] [lt=19] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:23.461811] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8FB0-0-0] [lt=19] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=700, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:23.461816] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8FB0-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=379, avg_log_submit_to_slide_time=694) [2024-10-16 14:57:23.461843] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:23.463289] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.475716] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=50] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=4101, avg_group_log_size=372) [2024-10-16 14:57:23.477170] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=371, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:23.477187] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=300, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:23.478983] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=89, avg_log_batch_cnt=0, total_group_log_size=144518, avg_group_log_size=1623) [2024-10-16 14:57:23.482292] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be0ab30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202967} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6202}, prev=this=0x2b0ec6d6b0d0 trans_version={val:1729061780291758000, v:0} scn={val:1729061780291758000, v:0} tx_id={txid:392201931} prev=0x2b0ec6d6af90 next=(nil) modify_count=1 acc_checksum=1532628518 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2135}, snapshot_version={val:1729061843479727000, v:0}, *this={this=0x2b0ec6d6ac88 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d6b0d0 latest_compact_node=(nil) max_trans_version={val:1729061780291758000, v:0} max_trans_id=392201931 max_elr_trans_version={val:1729061780291758000, v:0} max_elr_trans_id=392201931 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780291758000, v:0} min_modify_scn={val:1729061780291758000, v:0}}) [2024-10-16 14:57:23.482331] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be0ab30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202967} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6202}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606960, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606960, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843479727000, v:0}, tx_id:{txid:392202967}, scn:{branch:0, seq:3804}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202967}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392202967}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843482125, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843475924, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:6202}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.482388] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202967} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392202967}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:6202}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:23.482406] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":654830049190072320},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202967} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.482418] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:23.482424] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049190072320},{"BIGINT":1729058400},{"VARCHAR":"@é ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:23.482469] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:23.482765] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:23.482774] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:23.482785] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:23.482790] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:23.482799] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:23.482801] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:23.482804] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:23.482806] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:23.482809] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:23.482811] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:23.482814] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:23.482834] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECB-0-0] [lt=8] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:23.482841] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECB-0-0] [lt=5] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:23.484609] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8612}, to_seq_no={branch:0, seq:3805}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202967} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.489090] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:23.489104] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:23.489116] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=70494771, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:23.490625] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.491027] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061843490361000, start_id=1729061843490361000, end_id=1729061843490361001) [2024-10-16 14:57:23.494464] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eebf80 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202968} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2115}, prev=this=0x2b0f22ea1200 trans_version={val:1729061780293883000, v:0} scn={val:1729061780293883000, v:0} tx_id={txid:392201932} prev=0x2b0f22ea10c0 next=(nil) modify_count=1 acc_checksum=2916512878 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2127}, snapshot_version={val:1729061843491418000, v:0}, *this={this=0x2b0f22ea0db8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea1200 latest_compact_node=(nil) max_trans_version={val:1729061780293883000, v:0} max_trans_id=392201932 max_elr_trans_version={val:1729061780293883000, v:0} max_elr_trans_id=392201932 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780293883000, v:0} min_modify_scn={val:1729061780293883000, v:0}}) [2024-10-16 14:57:23.494495] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eebf80 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392202968} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2115}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606288, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606288, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061843491418000, v:0}, tx_id:{txid:392202968}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392202968}, tx_desc:{this:0x2b0ec37c99d0, tx_id:{txid:392202968}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061843493532, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061843491418, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2115}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:23.494562] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513900119740416},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202968} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.494974] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=7] Garbage Collector is running(seq_=656493, gc_interval=10000000) [2024-10-16 14:57:23.495004] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=10] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:23.495011] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:23.495551] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=2] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:23.495893] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:23.495901] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] gc_check_member_list_ cost time(ret=0, time_us=917) [2024-10-16 14:57:23.495909] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:23.496699] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:23.497358] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=17] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:23.498023] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:23.498652] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:23.498664] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:23.498673] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:23.498678] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:23.498688] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:23.498693] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=30) [2024-10-16 14:57:23.498698] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:23.500024] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8208}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202968} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.502890] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:23.503587] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=9] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:23.503620] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=32] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:23.503639] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:23.503703] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061855590307000, latest_log_ts:{val:1729061838929180001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:23.503749] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=42] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:23.503766] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:23.503779] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061838929180001, v:0}) [2024-10-16 14:57:23.503782] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:23.503786] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:23.503794] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:23.503804] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:23.503811] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:23.503817] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:23.517825] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061843516920000, start_id=1729061843516920000, end_id=1729061843516920001) [2024-10-16 14:57:23.517845] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=16] ts response task statistics(alloc_count=261363706, free_count=130681852) [2024-10-16 14:57:23.528587] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CCFF1-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=798, avg_log_submit_to_slide_time=1235) [2024-10-16 14:57:23.532930] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061843532849000, start_id=1729061843532849000, end_id=1729061843532849001) [2024-10-16 14:57:23.533832] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:23.543684] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061843542263000, start_id=1729061843542263000, end_id=1729061843542263001) [2024-10-16 14:57:23.544627] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061843542263000, start_id=1729061843542263000, end_id=1729061843542263001) [2024-10-16 14:57:23.544678] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:23.544691] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:23.547526] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E998FCD75C5-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061843546142000, start_id=1729061843546142000, end_id=1729061843546142001) [2024-10-16 14:57:23.562496] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:23.563361] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.563595] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:23.563631] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830049252986880},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202971} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.566317] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:23.566332] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:23.567923] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:23.568006] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:13199}, to_seq_no={branch:0, seq:6482}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202971} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.569380] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.571634] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.571650] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.575633] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.578635] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3EB2-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=89, avg_log_gen_to_freeze_time=5, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=456, avg_log_submit_to_slide_time=695) [2024-10-16 14:57:23.580420] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1003}, {id:1002}, {id:1}]) [2024-10-16 14:57:23.581726] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26148][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB19CC6EA-0-0] [lt=24] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:23.590793] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.592060] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061843590530000, start_id=1729061843590530000, end_id=1729061843590530001) [2024-10-16 14:57:23.597627] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4594, mem_used=8318976) [2024-10-16 14:57:23.601589] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3EB3-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:23.601611] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3EB3-0-0] [lt=22] [PALF STAT LOG LIFE TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=1067, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:23.601639] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=45] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=125037, hit_cnt=73, read_cnt=73, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:23.603967] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830049278152704},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202973} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.605512] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=51] get number(ret=0, service_type_=0, range=1, base_id=1729061843604330000, start_id=1729061843604330000, end_id=1729061843604330001) [2024-10-16 14:57:23.605912] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7049}, to_seq_no={branch:0, seq:2453}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202973} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.610242] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061843610087000, start_id=1729061843610087000, end_id=1729061843610087001) [2024-10-16 14:57:23.614539] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5B-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:23.614565] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5B-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061843126496004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061843126496004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:23.617706] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D5B-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061843617633000, start_id=1729061843617633000, end_id=1729061843617633001) [2024-10-16 14:57:23.620406] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D5B-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061843619771000, start_id=1729061843619771000, end_id=1729061843619771001) [2024-10-16 14:57:23.623929] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830049303318528},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202975} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.624433] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202975} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.627624] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BA2-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061843621933000, start_id=1729061843621933000, end_id=1729061843621933001) [2024-10-16 14:57:23.627842] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:23.627849] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] try gc retain ctx [2024-10-16 14:57:23.627853] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:23.627867] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=2] get_max_applied_scn(scn={val:1729061843516920000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493440320}, palf_committed_end_scn_:{val:1729061843516920000, v:0}, last_check_scn_:{val:1729061843516920000, v:0}, max_applied_cb_scn_:{val:1729061843516920000, v:0}}) [2024-10-16 14:57:23.627883] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:23.628491] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=40] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061840514562003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061850721265000, latest_log_ts:{val:1729061834119131001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:23.628571] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=37] get_max_applied_scn(scn={val:1729061843516920000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645397210}, palf_committed_end_scn_:{val:1729061843516920000, v:0}, last_check_scn_:{val:1729061843516920000, v:0}, max_applied_cb_scn_:{val:1729061843516920000, v:0}}) [2024-10-16 14:57:23.628580] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:23.629244] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=34] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061840514562003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:23.629908] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=30] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061840514562003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:23.629947] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=33] get_min_unreplayed_log_info(lsn={lsn:14352352440}, scn={val:1729061843128612005, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061843621137, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352352440}, next_to_submit_scn_:{val:1729061843128612005, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352352318}, end_lsn:{lsn:14352352440}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061843128612004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061843128612004, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:234676778, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061843128612004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656770, miss_cnt:17, cache_read_size:8050142661, hit_ratio:"9.999997286806295849e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:23.630536] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061840514562003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:23.631324] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B91A-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061843621933000, start_id=1729061843621933000, end_id=1729061843621933001) [2024-10-16 14:57:23.634652] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061843633624000, start_id=1729061843633624000, end_id=1729061843633624001) [2024-10-16 14:57:23.641917] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=7] replay status is not follower(min_unreplayed_lsn={lsn:33216819320}, base_lsn={lsn:33149755392}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:23.641972] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=54] get_replay_process success(id={id:1}, replayed_log_size=67063928, unreplayed_log_size=0) [2024-10-16 14:57:23.641989] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=63, last_replayed_log_size(MB)=63, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:23.647462] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=269, replace_time=0, replace_start_pos=1478479, replace_num=31457) [2024-10-16 14:57:23.647483] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] replace map num details(ret=0, replace_node_count=269, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:23.648681] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061843645303000, start_id=1729061843645303000, end_id=1729061843645303001) [2024-10-16 14:57:23.649069] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=12][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:23.651224] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:23.651241] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=15] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:23.651248] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:23.651253] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:23.652808] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:23.653302] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=104, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}]) [2024-10-16 14:57:23.653464] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=161] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:23.660318] INFO [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:262) [22249][T1002_ArcSender][T1002][YB4A0A000407-00061E996F9960C1-0-0] [lt=14] ObArchiveSender is running(thread_index=1) [2024-10-16 14:57:23.662275] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E5F-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=125032) [2024-10-16 14:57:23.662292] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E5F-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497187, log_task={header:{begin_lsn:{lsn:207589420447}, end_lsn:{lsn:207589422070}, log_id:218497187, min_scn:{val:1729061843475920000, v:0}, max_scn:{val:1729061843475920000, v:0}, data_len:1567, proposal_id:81, prev_lsn:{lsn:207589418823}, prev_proposal_id:81, committed_end_lsn:{lsn:207589420447}, data_checksum:2519015070, accum_checksum:1209256897, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061843478973, freeze_ts:1729061843478973, submit_ts:1729061843478978, flushed_ts:1729061843482100, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:3122}, fs_cb_begin_ts=1729061843662288, log_life_time=183315) [2024-10-16 14:57:23.662356] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=123408, hit_cnt=73, read_cnt=73, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:23.663429] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.668005] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=7, avg_log_batch_cnt=1, total_group_log_size=860, avg_group_log_size=122) [2024-10-16 14:57:23.668024] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=860) [2024-10-16 14:57:23.668521] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=17] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:23.669407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061843668207000, start_id=1729061843668207000, end_id=1729061843668207001) [2024-10-16 14:57:23.679335] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061839674324, last_first_register_time_us=1722494150998907, register_time_us=1729061839674324, curr_time_us=1729061843679334) [2024-10-16 14:57:23.679377] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=19] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061843679355, last_submit_register_req_time_us=1729061843679355, register_parent_reason=3) [2024-10-16 14:57:23.679386] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:23.685395] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830049391398912},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202981} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.686268] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4203}, to_seq_no={branch:0, seq:3139}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202981} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.690844] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=32][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.691667] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=15] report RowHolderMapper summary info(count=1, bkt_cnt=248) [2024-10-16 14:57:23.695621] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=16] gc stale ls task succ [2024-10-16 14:57:23.712160] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061843711097000, start_id=1729061843711097000, end_id=1729061843711097001) [2024-10-16 14:57:23.722647] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:23.726845] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=215, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:23.726863] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:23.726876] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=614, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:23.726910] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:23.727988] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:23.728004] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:23.734840] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1204781056, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:23.736921] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=14] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:23.738997] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061843738046000, start_id=1729061843738046000, end_id=1729061843738046001) [2024-10-16 14:57:23.740387] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:23.740592] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=740, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:23.740612] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=402, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:23.740640] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=77, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:23.743823] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061843742287000, start_id=1729061843742287000, end_id=1729061843742287001) [2024-10-16 14:57:23.744794] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061843742287000, start_id=1729061843742287000, end_id=1729061843742287001) [2024-10-16 14:57:23.746367] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513900333649920},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202984} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.747728] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7467}, to_seq_no={branch:0, seq:5356}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202984} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.748975] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:23.748993] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=10] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:23.749000] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=5] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:23.756724] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15D0-0-0] [lt=9] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:23.763511] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.768879] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:23.769341] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=24] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}) [2024-10-16 14:57:23.776091] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:23.776121] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3544) [2024-10-16 14:57:23.776130] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:23.778364] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513900346232832},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202986} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.780674] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=23] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:23.783020] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8796}, to_seq_no={branch:0, seq:2856}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202986} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.786905] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=11] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:23.787619] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788445] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788454] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788470] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788620] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788634] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788630] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788645] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788655] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788673] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.788690] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:23.790896] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.792049] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061843790221000, start_id=1729061843790221000, end_id=1729061843790221001) [2024-10-16 14:57:23.792115] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:23.792133] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9693, avg_group_log_size=570) [2024-10-16 14:57:23.792143] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9693) [2024-10-16 14:57:23.792149] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:23.793310] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1446, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:23.793322] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9693) [2024-10-16 14:57:23.793334] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=478, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:23.793355] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9693) [2024-10-16 14:57:23.793363] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:23.795637] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93A0-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:23.795647] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93A0-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=2691, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:23.795652] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93A0-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=2118, avg_log_submit_to_slide_time=2660) [2024-10-16 14:57:23.805534] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061843803613000, start_id=1729061843803613000, end_id=1729061843803613001) [2024-10-16 14:57:23.808623] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [21908][T1001_MediumLoo][T1001][YB4A0A000407-00061E9973AF38E4-0-0] [lt=6] MediumCheckTask(cost_ts=7) [2024-10-16 14:57:23.820234] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:23.820812] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=13] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:23.820864] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061834390737000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:23.820979] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=36] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1}, {id:1002}]) [2024-10-16 14:57:23.821584] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C6C-0-0] [lt=24] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:23.821594] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C6C-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=55085, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:23.821598] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C6C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=795, avg_log_submit_to_slide_time=55079) [2024-10-16 14:57:23.821613] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C6C-0-0] [lt=13][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172969, log_task={header:{begin_lsn:{lsn:14352352440}, end_lsn:{lsn:14352352562}, log_id:115172969, min_scn:{val:1729061843621933000, v:0}, max_scn:{val:1729061843621933000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352352318}, prev_proposal_id:17, committed_end_lsn:{lsn:14352352440}, data_checksum:940009244, accum_checksum:1292824368, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061843720688, freeze_ts:1729061843720688, submit_ts:1729061843720699, flushed_ts:1729061843726887, gen_to_freeze cost time:0, gen_to_submit cost time:11, submit_to_flush cost time:6188}, fs_cb_begin_ts=1729061843821577, log_life_time=100889) [2024-10-16 14:57:23.821647] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:23.821658] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656771, miss_cnt:17, cache_read_size:8050142783, hit_ratio:"9.999997286806339147e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:23.827375] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:23.827387] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=7] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:23.827400] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=52228320, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:23.828320] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=745, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:23.828332] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=175, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:23.843994] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513900383981568},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202990} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.848993] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] Cache replace map node details(ret=0, replace_node_count=487, replace_time=0, replace_start_pos=1509936, replace_num=31457) [2024-10-16 14:57:23.849033] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=487, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:23.850459] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061843849456000, start_id=1729061843849456000, end_id=1729061843849456001) [2024-10-16 14:57:23.850653] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3308, mem_used=6239232) [2024-10-16 14:57:23.851008] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:14324}, to_seq_no={branch:0, seq:1568}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202990} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.863233] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:23.863897] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:23.863957] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061841826930000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:23.864319] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.865410] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AEF-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:23.865668] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=453, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:23.865683] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:23.865720] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:23.866057] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061843866010000, start_id=1729061843866010000, end_id=1729061843866010001) [2024-10-16 14:57:23.866135] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=60, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:23.866148] INFO [LIB] stat (utility.h:1165) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] [PALF STAT APPEND COST TIME](cur_stat_count=60, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:23.866383] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=60, stat_interval=1000000, avg (count/cost)=323, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:23.866410] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=97435) [2024-10-16 14:57:23.866420] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=60, stat_interval=1000000, avg (count/cost)=189, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:23.866444] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=97437) [2024-10-16 14:57:23.866451] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=60, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:23.868916] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061843867913000, start_id=1729061843867913000, end_id=1729061843867913001) [2024-10-16 14:57:23.883442] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:23.885572] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:23.886083] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=54] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561072, miss_cnt:1, cache_read_size:152590584711, hit_ratio:"9.999999928346782196e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:23.890298] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:14352352740}, scn={val:1729061843621933002, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:23.890328] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:12, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061843890262, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352352796}, next_to_submit_scn_:{val:1729061843621933003, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352352684}, end_lsn:{lsn:14352352796}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061843621933002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061843621933002, v:0}, data_checksum:1970726640, flag:1}}, init_mode_version:0, accumulate_checksum:825321077, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061843621933002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656773, miss_cnt:17, cache_read_size:8050143017, hit_ratio:"9.999997286806425745e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14352352740}, scn_:{val:1729061843621933002, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061843857368, first_handle_ts_:1729061843890328, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0ef5ff14a0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:23.890490] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=47] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060041826930000) [2024-10-16 14:57:23.890497] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=6] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14352352740}, scn_:{val:1729061843621933002, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061843857368, first_handle_ts_:1729061843890479, replay_cost_:16, retry_cost_:-1, read_log_buf_:0x2b0ef5ff14b0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061843890262, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352352796}, next_to_submit_scn_:{val:1729061843621933003, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352352684}, end_lsn:{lsn:14352352796}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061843621933002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061843621933002, v:0}, data_checksum:1970726640, flag:1}}, init_mode_version:0, accumulate_checksum:825321077, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061843621933002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656773, miss_cnt:17, cache_read_size:8050143017, hit_ratio:"9.999997286806425745e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}, replay_queue_idx=24, ret=0) [2024-10-16 14:57:23.890655] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675513900409147392},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392202992} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:23.890954] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.891100] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5464}, to_seq_no={branch:0, seq:4338}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392202992} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:23.893850] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257682, offset=39010451}, end_cursor:ObLogCursor{file_id=197, log_id=49257683, offset=39010734}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bd4050, len:3949, log_data_len:283, seq:49257682, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:23.897531] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB35D-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061843895257000, start_id=1729061843895257000, end_id=1729061843895257001) [2024-10-16 14:57:23.899162] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257683, offset=39010734}, end_cursor:ObLogCursor{file_id=197, log_id=49257684, offset=39011016}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ac4050, len:3666, log_data_len:282, seq:49257683, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:23.901187] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061843900963000, start_id=1729061843900963000, end_id=1729061843900963001) [2024-10-16 14:57:23.901999] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257684, offset=39011016}, end_cursor:ObLogCursor{file_id=197, log_id=49257685, offset=39011298}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd296c050, len:3384, log_data_len:282, seq:49257684, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:23.903632] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=38] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257685, offset=39011298}, end_cursor:ObLogCursor{file_id=197, log_id=49257686, offset=39011580}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ac8050, len:3102, log_data_len:282, seq:49257685, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:23.903706] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650039, miss_cnt:32, cache_read_size:8050395777, hit_ratio:"9.999994892264367774e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:23.904884] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:23.905496] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601316, offset=9605266}, end_cursor:ObLogCursor{file_id=192, log_id=47601317, offset=9605550}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bfc050, len:3950, log_data_len:284, seq:47601316, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:23.907244] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 1.00, rt: 685.00; ) [2024-10-16 14:57:23.907261] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:23.907301] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 485.00; ) [2024-10-16 14:57:23.907307] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:23.907314] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:23.910336] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA492-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061843908635000, start_id=1729061843908635000, end_id=1729061843908635001) [2024-10-16 14:57:23.911518] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601317, offset=9605550}, end_cursor:ObLogCursor{file_id=192, log_id=47601318, offset=9605824}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c10050, len:3666, log_data_len:274, seq:47601317, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:23.914633] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601318, offset=9605824}, end_cursor:ObLogCursor{file_id=192, log_id=47601319, offset=9606098}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b1c050, len:3392, log_data_len:274, seq:47601318, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:23.915650] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601319, offset=9606098}, end_cursor:ObLogCursor{file_id=192, log_id=47601320, offset=9606372}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b40050, len:3118, log_data_len:274, seq:47601319, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:23.915728] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598642, miss_cnt:9, cache_read_size:149777794752, hit_ratio:"9.999999345923820115e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:23.917386] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86E-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:23.917405] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86E-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061843738046000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061843738046000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:23.919126] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA86E-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061843918696000, start_id=1729061843918696000, end_id=1729061843918696001) [2024-10-16 14:57:23.920629] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:23.924626] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA86E-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061843924311000, start_id=1729061843924311000, end_id=1729061843924311001) [2024-10-16 14:57:23.926315] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=11] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:23.935154] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:23.936469] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=8] PNIO [ratelimit] time: 1729061843936467, bytes: 0, bw: 0.000000 MB/s, add_ts: 1000553, add_bytes: 0 [2024-10-16 14:57:23.938430] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2180, mem_used=6239232) [2024-10-16 14:57:23.944449] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061843943141000, start_id=1729061843943141000, end_id=1729061843943141001) [2024-10-16 14:57:23.945423] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061843944625000, start_id=1729061843944625000, end_id=1729061843944625001) [2024-10-16 14:57:23.947211] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497194, log_task={header:{begin_lsn:{lsn:207589430311}, end_lsn:{lsn:207589431935}, log_id:218497194, min_scn:{val:1729061843821398000, v:0}, max_scn:{val:1729061843821398000, v:0}, data_len:1568, proposal_id:81, prev_lsn:{lsn:207589428687}, prev_proposal_id:81, committed_end_lsn:{lsn:207589430311}, data_checksum:1413900671, accum_checksum:667754976, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061843821994, freeze_ts:1729061843821994, submit_ts:1729061843821997, flushed_ts:1729061843822146, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:149}, fs_cb_begin_ts=1729061843947205, log_life_time=125211) [2024-10-16 14:57:23.947695] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061843552356000, v:0}, new_max_version={val:1729061843552356000, v:0}, last_min_version={val:1729061843267140000, v:0}, last_max_version={val:1729061843267140000, v:0}, total_time_us=4133, delta_us=395338) [2024-10-16 14:57:23.947710] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061843552356000, v:0}, min_version={val:1729061843552356000, v:0}, max_version={val:1729061843552356000, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:57:23.962932] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AF1-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:23.962952] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AF1-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:23.963345] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=43, stat_interval=1000000, avg (count/cost)=314, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:23.963359] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=43, stat_interval=1000000, avg (count/cost)=137, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:23.963385] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=43, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:23.963437] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061843963436, bytes: 4446805418503, bw: 1.140744 MB/s, add_ts: 1002411, add_bytes: 1199041 [2024-10-16 14:57:23.964373] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.973111] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:23.974235] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D1-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061843963702, keepalive_ts:-1}) [2024-10-16 14:57:23.974250] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D1-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061843963702, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:23.977207] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3665) [2024-10-16 14:57:23.977251] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3665) [2024-10-16 14:57:23.979100] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F68-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:23.979112] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F68-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1505, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:23.991012] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:23.992143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061843990494000, start_id=1729061843990494000, end_id=1729061843990494001) [2024-10-16 14:57:23.992503] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:23.992521] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=10] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=208) [2024-10-16 14:57:23.998214] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061843997956000, start_id=1729061843997956000, end_id=1729061843997956001) [2024-10-16 14:57:24.006507] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061844004718000, start_id=1729061844004718000, end_id=1729061844004718001) [2024-10-16 14:57:24.008852] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=12] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:24.010497] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:57:24.011229] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.011256] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.011276] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:24.011410] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841860267000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860098662000, latest_log_ts:{val:1729061843439703001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:24.011457] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841860267000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860098662000, latest_log_ts:{val:1729061843439703001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:24.011487] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:24.011493] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061843439703001, v:0}) [2024-10-16 14:57:24.011497] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:24.011506] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:24.011515] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:24.011522] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:24.012212] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.012226] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.012236] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:24.012272] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841860267000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:24.012300] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841860267000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:24.012323] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:24.012328] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:24.012333] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:24.012338] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:24.013031] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.013043] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.013052] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:24.013086] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841860267000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:24.013113] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841860267000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:24.013141] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:24.013146] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:24.013151] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:24.013157] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:24.013850] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.013863] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=13] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.013873] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:24.013906] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841860267000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:24.013934] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841860267000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:24.013957] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:24.013962] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:24.013968] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:24.013972] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:24.013976] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:24.018187] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061844017195000, start_id=1729061844017195000, end_id=1729061844017195001) [2024-10-16 14:57:24.018221] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061844018217], gts:1729061844017195000, latest_srr:[mts=1729061844018217]}) [2024-10-16 14:57:24.022350] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:24.022421] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1002 and flag like "%DUPLICATE%", tenant_id=1002, exec_tenant_id=1001, need_member_list=false) [2024-10-16 14:57:24.023794] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=91, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:24.023808] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=91, stat_interval=1000000, avg (count/cost)=92, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:24.028184] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:24.028202] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:24.028719] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD000-0-0] [lt=24] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1636) [2024-10-16 14:57:24.028907] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD001-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:24.040814] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] Garbage Collector is running(seq_=656490, gc_interval=10000000) [2024-10-16 14:57:24.040847] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=15] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:24.040855] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=7] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:24.041498] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [22325][T1003_COMMONLSS][T1003][YB4A0A000407-00061E9B3A19CB02-0-0] [lt=16][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:24.050530] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=349, replace_time=0, replace_start_pos=1541393, replace_num=31457) [2024-10-16 14:57:24.050546] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=349, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:24.051860] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:24.052484] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844050563000, start_id=1729061844050563000, end_id=1729061844050563001) [2024-10-16 14:57:24.060868] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=11][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:24.063946] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=12] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:24.063961] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=10] gc_check_member_list_ cost time(ret=0, time_us=23132) [2024-10-16 14:57:24.063973] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=1) [2024-10-16 14:57:24.064434] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.064731] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:24.065346] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=16] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:24.065957] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=17] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:24.066523] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=12] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:24.066543] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=16] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:24.066555] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=11] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:24.066561] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:24.066566] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:24.066573] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] execute_gc cost time(ret=0, time_us=30) [2024-10-16 14:57:24.066577] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:24.066585] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:133) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-10-16 14:57:24.066598] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:140) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-10-16 14:57:24.068031] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C409B-0-0] [lt=24] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=48, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:24.068044] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C409B-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=48, stat_interval=1000000, avg (count/cost)=11177, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:24.068808] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=49, avg_log_batch_cnt=1, total_group_log_size=78069, avg_group_log_size=1593) [2024-10-16 14:57:24.068821] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=79571) [2024-10-16 14:57:24.069013] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=24530776, this={dio_aligned_buf:{buf_write_offset:4050, buf_padding_size:46, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:93866859, truncate_used_ts:39944877}, log_block_size:67108864, dir_fd:173, io_fd:196}, aligned_buf_len=4096, aligned_block_offset=24526848, buf_len=122, total_write_size_=1291215, total_write_size_after_dio_=4472832, ob_pwrite_used_ts=222464, count=777, each_pwrite_cost=286) [2024-10-16 14:57:24.069412] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3ECA-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=7, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=288, avg_log_submit_to_slide_time=904) [2024-10-16 14:57:24.091077] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.098833] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844098216000, start_id=1729061844098216000, end_id=1729061844098216001) [2024-10-16 14:57:24.110390] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:24.110409] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:24.112160] INFO [COMMON] loop_running_dag_net_list (ob_tenant_dag_scheduler.cpp:3738) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] loop running dag net list(ret=0, running_dag_net_list_size=0, slow_dag_net_cnt=0, finished_dag_net_list_size=0) [2024-10-16 14:57:24.112174] INFO [COMMON] loop_finished_dag_net_list (ob_tenant_dag_scheduler.cpp:3761) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=13] loop finsihed dag net list(ret=0, finished_dag_net_list_size=0) [2024-10-16 14:57:24.123640] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:24.124985] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=12] current monitor number(seq_=-1) [2024-10-16 14:57:24.127860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BA6-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061844127564000, start_id=1729061844127564000, end_id=1729061844127564001) [2024-10-16 14:57:24.127946] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=540, mem_used=2079744) [2024-10-16 14:57:24.128066] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BA9-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061844127564000, start_id=1729061844127564000, end_id=1729061844127564001) [2024-10-16 14:57:24.128254] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1636, avg_group_log_size=163) [2024-10-16 14:57:24.128267] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1636) [2024-10-16 14:57:24.128555] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BAA-0-0] [lt=10] submit log success(service_type_=0, last_id=1729061860721265000, limited_id=1729061860721265000) [2024-10-16 14:57:24.128566] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BAA-0-0] [lt=10] submit log with lock success(service_type_=0, last_id=1729061860721265000, limited_id=1729061860721265000) [2024-10-16 14:57:24.128572] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BAA-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061844127564000, start_id=1729061844127564000, end_id=1729061844127564001) [2024-10-16 14:57:24.128621] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:24.128995] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BA7-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061844128676000, start_id=1729061844128676000, end_id=1729061844128676001) [2024-10-16 14:57:24.129123] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061860721265000, latest_scn={val:1729061844127564001, v:0}, *this={tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841515837003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860721265000, latest_log_ts:{val:1729061844127564001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:24.129154] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=31] handle submit callback(service_type_=0, success=true, used_ts=600, log_ts={val:1729061844127564001, v:0}, limited_id=1729061860721265000) [2024-10-16 14:57:24.129161] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:24.129165] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:24.135512] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1204781056, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:24.138310] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3C9F-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510312006704, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410516, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:24.144621] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061844142782000, start_id=1729061844142782000, end_id=1729061844142782001) [2024-10-16 14:57:24.145332] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830049718554624},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203003} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.147619] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061844146033000, start_id=1729061844146033000, end_id=1729061844146033001) [2024-10-16 14:57:24.147687] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=9575, avg_group_log_size=598) [2024-10-16 14:57:24.147697] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9575) [2024-10-16 14:57:24.147998] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=362, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:24.148010] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9575) [2024-10-16 14:57:24.148020] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=273, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:24.148037] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9575) [2024-10-16 14:57:24.148045] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:24.148389] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD00A-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:24.148395] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD00A-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=2246, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:24.148399] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD00A-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=119, avg_log_gen_to_submit_time=147, avg_log_submit_to_flush_time=1266, avg_log_submit_to_slide_time=2098) [2024-10-16 14:57:24.153116] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22268][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:21264}, to_seq_no={branch:0, seq:1111}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203003} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.163821] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21659][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE87A-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510312032215, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410517, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:24.164548] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.164833] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061844164438000, start_id=1729061844164438000, end_id=1729061844164438001) [2024-10-16 14:57:24.168170] INFO [SHARE.LOCATION] renew_all_ls_locations_by_rpc (ob_ls_location_service.cpp:801) [21248][AutoLSLocRpc][T0][YB4A0A000407-00061E9B390A2246-0-0] [lt=45] [LS_LOCATION] Get ls leaders by RPC(ret=0, ret="OB_SUCCESS", dests=["10.0.4.7:2890", "10.0.4.8:2890", "10.0.4.9:2890"], leaders=[{key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}}, {key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}}, {key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}}, {key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}}, {key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}}, {key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}}, {key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}}, {key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}}, {key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}]) [2024-10-16 14:57:24.169994] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501A-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:24.170016] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501A-0-0] [lt=21] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:24.170135] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=15][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5024, dropped:58, tid:12153}]) [2024-10-16 14:57:24.191145] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.193992] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061844191546000, start_id=1729061844191546000, end_id=1729061844191546001) [2024-10-16 14:57:24.207646] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] no need to update min/max version(new_version={val:1729061843803613000, v:0}, min_version={val:1729061843803613000, v:0}, max_version={val:1729061843803613000, v:0}) [2024-10-16 14:57:24.214537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061844214174000, start_id=1729061844214174000, end_id=1729061844214174001) [2024-10-16 14:57:24.214653] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:114618340788}], scn_array=[cnt:1, {val:1729061844191546000, v:0}], curr_size=592, accum_size=5143, time_cost=20224) [2024-10-16 14:57:24.215799] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=47211) [2024-10-16 14:57:24.224882] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] ====== checkpoint timer task ====== [2024-10-16 14:57:24.225625] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.225664] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=37] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.225688] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=19] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:24.225820] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841515837003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860721265000, latest_log_ts:{val:1729061844127564001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:24.225870] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=46] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:24.225892] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:24.225903] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=0, rec_log_ts={val:1729061844127564001, v:0}) [2024-10-16 14:57:24.225910] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.225918] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:24.225933] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:24.225944] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:24.226643] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.226665] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=21] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.226694] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=26] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:24.226768] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841515837003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:24.226808] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=37] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841515837003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:24.226835] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:24.226842] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:24.226851] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:24.226857] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:24.227536] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.227552] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.227562] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:24.227649] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841515837003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:24.227684] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=33] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841515837003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:24.227708] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:24.227718] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:24.227729] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:24.227737] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:24.228182] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=0][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f477798 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203006} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:11680}, prev=this=0x2b0e5abfb008 trans_version={val:1729061780488752000, v:0} scn={val:1729061780488752000, v:0} tx_id={txid:392201970} prev=0x2b0e5abfaec8 next=(nil) modify_count=1 acc_checksum=833024085 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7500}, snapshot_version={val:1729061844219490000, v:0}, *this={this=0x2b0e5abfabc0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abfb008 latest_compact_node=(nil) max_trans_version={val:1729061780488752000, v:0} max_trans_id=392201970 max_elr_trans_version={val:1729061780488752000, v:0} max_elr_trans_id=392201970 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780488752000, v:0} min_modify_scn={val:1729061780488752000, v:0}}) [2024-10-16 14:57:24.228224] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=40][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f477798 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203006} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:11680}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606377, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606377, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844219490000, v:0}, tx_id:{txid:392203006}, scn:{branch:0, seq:6374}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203006}, tx_desc:{this:0x2b0ec37c7a30, tx_id:{txid:392203006}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844227953, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844216274, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:11680}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.228293] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=69][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203006} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203006}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:11680}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.228312] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513900534976512},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203006} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.228327] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.228334] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513900534976512},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.228389] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.228443] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.228463] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=20] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:24.228477] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:24.228550] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841515837003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:24.228590] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=38] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061841515837003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:24.228633] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=41] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:24.228643] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:24.228657] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:24.228666] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:24.228671] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:24.228837] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1333, avg_group_log_size=121) [2024-10-16 14:57:24.228849] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1333) [2024-10-16 14:57:24.229614] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD007-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:24.229625] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD007-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=927, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:24.229630] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD007-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=488, avg_log_submit_to_slide_time=913) [2024-10-16 14:57:24.229855] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:12767}, to_seq_no={branch:0, seq:6375}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203006} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.235882] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E08-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844234284000, start_id=1729061844234284000, end_id=1729061844234284001) [2024-10-16 14:57:24.238459] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1115, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:24.238483] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=269, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:24.238541] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:24.238553] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1901, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:24.238558] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:24.247205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1DFD-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061844246381000, start_id=1729061844246381000, end_id=1729061844246381001) [2024-10-16 14:57:24.247860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061844247530000, start_id=1729061844247530000, end_id=1729061844247530001) [2024-10-16 14:57:24.249439] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061844249305000, start_id=1729061844249305000, end_id=1729061844249305001) [2024-10-16 14:57:24.249552] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] no need to update min/max version(new_version={val:1729061843944625000, v:0}, min_version={val:1729061843944625000, v:0}, max_version={val:1729061843944625000, v:0}) [2024-10-16 14:57:24.250412] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7635-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=51962) [2024-10-16 14:57:24.252150] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=306, replace_time=0, replace_start_pos=1572850, replace_num=31457) [2024-10-16 14:57:24.252179] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=306, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:24.252447] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=50460) [2024-10-16 14:57:24.259700] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:17959669986}], scn_array=[cnt:1, {val:1729061844246381000, v:0}], curr_size=548, accum_size=1286, time_cost=10105) [2024-10-16 14:57:24.259748] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:204023023145}], scn_array=[cnt:1, {val:1729061844247530000, v:0}], curr_size=1624, accum_size=4993, time_cost=10092) [2024-10-16 14:57:24.259770] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=46][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:114618341380}], scn_array=[cnt:1, {val:1729061844249305000, v:0}], curr_size=592, accum_size=5735, time_cost=10087) [2024-10-16 14:57:24.260332] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:24.260344] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=1772, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:24.260725] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061844260382000, start_id=1729061844260382000, end_id=1729061844260382001) [2024-10-16 14:57:24.261076] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183801223, max_lsn:114264548555, max_scn:{val:1729061844260382000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45336899169}, reuse_lsn:{lsn:114264547963}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114222605515}}, last_submit_lsn:{lsn:114264547963}, last_submit_end_lsn:{lsn:114264548555}, last_submit_log_id:183801223, last_submit_log_pid:33576, max_flushed_lsn:{lsn:114264547841}, max_flushed_end_lsn:{lsn:114264547963}, max_flushed_log_pid:33576, committed_end_lsn:{lsn:114264547963}, last_slide_log_id:183801222, last_slide_scn:{val:1729061844146033001, v:0}, last_slide_lsn:{lsn:114264547841}, last_slide_end_lsn:{lsn:114264547963}, last_slide_log_pid:33576, last_slide_log_accum_checksum:97035154, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e2efe2530}, [StateMgr]={this:0x2b0e2efe6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, leader:"10.0.4.7:2890", leader_epoch:2, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183801223, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261840845, palf_role_change_cb:0x2b0e2efed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, reconfig_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, persistent_config_version:{proposal_id:33576, config_seq:9}, ms_ack_list:0, resend_config_version:{proposal_id:33576, config_seq:9}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494259837171, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494259837171, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e2efe3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262041271, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729031774558563002, v:0}, min_block_id:1697, min_block_min_scn:{val:1729025370959302001, v:0}, base_lsn_for_block_gc:{lsn:114212315136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, log_config_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, log_snapshot_meta:{version:1, base_lsn:{lsn:114212315136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:232079360}, readable_log_tail:{lsn:232079360}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114279419904}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/meta", dir_fd:187, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:36339712, block_header_serialize_buf_:0x2b0e2efe89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114264547963}, readable_log_tail:{lsn:114264547963}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114212315136}, min_scn:{val:1729057087482360001, v:0}, curr_block_id:1702, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/log", dir_fd:190, min_block_id:1697, max_block_id:1703, curr_writable_block_id:1702}, logical_block_size_:67104768, curr_block_writable_size_:14871941, block_header_serialize_buf_:0x2b0e2efea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261839896, this:0x2b0e2efe6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33576, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33575, majority_max_lsn:{lsn:45338084276}, saved_end_lsn:{lsn:45338084276}, last_submit_prepare_req_time_us:1722494261840968, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e2efe69b0}) [2024-10-16 14:57:24.261207] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=129] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061844254965, last_flushed_end_lsn:{lsn:114264547963}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061844258289, last_flushed_end_lsn:{lsn:114264547963}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061844258329, last_flushed_end_lsn:{lsn:114264547963}}]) [2024-10-16 14:57:24.261219] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183801223, log_task={header:{begin_lsn:{lsn:114264547963}, end_lsn:{lsn:114264548555}, log_id:183801223, min_scn:{val:1729061844260382000, v:0}, max_scn:{val:1729061844260382000, v:0}, data_len:536, proposal_id:33576, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:114264547963}, data_checksum:2637154184, accum_checksum:2205093685, is_padding_log:false, is_raw_write:false}, state_map:{val:30}, ref_cnt:0, gen_ts:1729061844260769, freeze_ts:1729061844260772, submit_ts:1729061844260808, flushed_ts:-1, gen_to_freeze cost time:3, gen_to_submit cost time:39, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061844254965, last_flushed_end_lsn:{lsn:114264547963}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061844258289, last_flushed_end_lsn:{lsn:114264547963}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061844258329, last_flushed_end_lsn:{lsn:114264547963}}]) [2024-10-16 14:57:24.264610] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.266522] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f477e78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203007} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779396338 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3121}, prev=this=0x2b0e5abfb708 trans_version={val:1729061780491928000, v:0} scn={val:1729061780491928000, v:0} tx_id={txid:392201971} prev=0x2b0e5abfb5c8 next=(nil) modify_count=1 acc_checksum=2905247574 version=1729061779396338 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2118}, snapshot_version={val:1729061844264320000, v:0}, *this={this=0x2b0e5abfb2c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abfb708 latest_compact_node=(nil) max_trans_version={val:1729061780491928000, v:0} max_trans_id=392201971 max_elr_trans_version={val:1729061780491928000, v:0} max_elr_trans_id=392201971 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780491928000, v:0} min_modify_scn={val:1729061780491928000, v:0}}) [2024-10-16 14:57:24.266555] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f477e78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203007} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779396338 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3121}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606987, tablet_id:{id:209282}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606987, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844264320000, v:0}, tx_id:{txid:392203007}, scn:{branch:0, seq:1058}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203007}, tx_desc:{this:0x2b0ec37b9cd0, tx_id:{txid:392203007}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844266383, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844263263, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3121}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.266616] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203007} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203007}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779396338, seq_no:{branch:0, seq:3121}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.266631] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830049777274881},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203007} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.266643] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.266649] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830049777274881},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.266705] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.269580] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5889}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203007} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.283621] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:14538467058}], scn_array=[cnt:1, {val:1729061844127564001, v:0}], curr_size=122, accum_size=854, time_cost=11858) [2024-10-16 14:57:24.289552] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=7] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:24.290244] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985C9-0-0] [lt=13] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=1320) [2024-10-16 14:57:24.290348] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061844289093000, start_id=1729061844289093001, end_id=1729061844289093002) [2024-10-16 14:57:24.290424] INFO [LIB] stat (utility.h:1165) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:24.290439] INFO [LIB] stat (utility.h:1165) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:24.291213] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.292224] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C40A6-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1408) [2024-10-16 14:57:24.296037] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B1F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844294387000, start_id=1729061844294387000, end_id=1729061844294387001) [2024-10-16 14:57:24.297626] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT WRITE LOG COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=1793, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:24.297640] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1834) [2024-10-16 14:57:24.297652] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=843, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:24.297683] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:24.297696] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=3704, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:24.297700] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:24.308807] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061844307188000, start_id=1729061844307188000, end_id=1729061844307188001) [2024-10-16 14:57:24.309811] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:24.310283] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F6C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=3946, avg_log_submit_to_slide_time=4664) [2024-10-16 14:57:24.310999] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:24.311014] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:24.311030] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373521241, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:24.312917] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5704, mem_used=10398720) [2024-10-16 14:57:24.316343] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=21] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):113, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:24.317051] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061844316611000, start_id=1729061844316611000, end_id=1729061844316611001) [2024-10-16 14:57:24.321219] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:24.321795] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:24.321815] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=14] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53593280, tenant_id=1003) [2024-10-16 14:57:24.321827] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:24.327675] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD009-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1445) [2024-10-16 14:57:24.328823] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=6] ====== tenant freeze timer task ====== [2024-10-16 14:57:24.328836] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:24.328851] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:24.335826] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1206878208, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:24.344262] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B2-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:24.344292] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B2-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061844128676001, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061844128676001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:24.350561] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061844349048000, start_id=1729061844349048000, end_id=1729061844349048001) [2024-10-16 14:57:24.353690] INFO [SHARE] adaptive_update_limit (ob_share_throttle_define.cpp:112) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789D8-0-0] [lt=11] adaptive update(Tenant ID=1004, Config Specify Resource Limit(MB)=511, TxShare Current Memory Limit(MB)=158, Holding Memory(MB)=46, Tenant Remain Memory(MB)=185, Usable Remain Memory(MB)=111, Last Update Limit Timestamp=1729061844352845, Is Updated=true) [2024-10-16 14:57:24.353708] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789D8-0-0] [lt=17] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=3.164879131317138672e+01, this=0x2b0e4009f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=158, Throttle Trigger(MB)=94, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.697391673526216982e+04) [2024-10-16 14:57:24.353855] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789D8-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061844352845000, start_id=1729061844352845000, end_id=1729061844352845001) [2024-10-16 14:57:24.355267] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=46206256, this={dio_aligned_buf:{buf_write_offset:3923, buf_padding_size:173, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33201854, truncate_used_ts:11502094}, log_block_size:67108864, dir_fd:195, io_fd:131}, aligned_buf_len=4096, aligned_block_offset=46202880, buf_len=547, total_write_size_=15666, total_write_size_after_dio_=430080, ob_pwrite_used_ts=47698, count=102, each_pwrite_cost=467) [2024-10-16 14:57:24.355674] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9B2-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061844355317000, start_id=1729061844355317000, end_id=1729061844355317001) [2024-10-16 14:57:24.359178] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb2fe70 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203012} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2316}, prev=this=0x2b0fadabdbf0 trans_version={val:1729061780514619000, v:0} scn={val:1729061780514619000, v:0} tx_id={txid:392201976} prev=0x2b0fadabdab0 next=(nil) modify_count=1 acc_checksum=2489265403 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4200}, snapshot_version={val:1729061844358688000, v:0}, *this={this=0x2b0fadabd7a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadabdbf0 latest_compact_node=(nil) max_trans_version={val:1729061780514619000, v:0} max_trans_id=392201976 max_elr_trans_version={val:1729061780514619000, v:0} max_elr_trans_id=392201976 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780514619000, v:0} min_modify_scn={val:1729061780514619000, v:0}}) [2024-10-16 14:57:24.359211] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb2fe70 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203012} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2316}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606265, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606265, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844358688000, v:0}, tx_id:{txid:392203012}, scn:{branch:0, seq:2313}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203012}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392203012}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844358688, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844356376, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2316}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.359275] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203012} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203012}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:2316}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.359289] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513900635639808},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203012} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.359301] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.359307] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513900635639808},{"BIGINT":1729058400},{"VARCHAR":"@¹É", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.359352] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.364367] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6763}, to_seq_no={branch:0, seq:2314}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203012} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.364408] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23] rpc statistics(total_trans_msg_count=46, total_batch_msg_count=0) [2024-10-16 14:57:24.364708] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=65][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.365555] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B5A0C6794-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844365433000, start_id=1729061844365433000, end_id=1729061844365433001) [2024-10-16 14:57:24.366674] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9B2-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844366494000, start_id=1729061844366494000, end_id=1729061844366494001) [2024-10-16 14:57:24.368009] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=11] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:24.377238] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=26] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:24.382453] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061844381570000, start_id=1729061844381570000, end_id=1729061844381570001) [2024-10-16 14:57:24.387482] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:24.389120] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B5A0C6794-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844388984000, start_id=1729061844388984000, end_id=1729061844388984001) [2024-10-16 14:57:24.391810] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f478558 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203013} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5233}, prev=this=0x2b0e5abfbe08 trans_version={val:1729061780518849000, v:0} scn={val:1729061780518849000, v:0} tx_id={txid:392201977} prev=0x2b0e5abfbcc8 next=(nil) modify_count=1 acc_checksum=3665169907 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3217}, snapshot_version={val:1729061844387926000, v:0}, *this={this=0x2b0e5abfb9c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abfbe08 latest_compact_node=(nil) max_trans_version={val:1729061780518849000, v:0} max_trans_id=392201977 max_elr_trans_version={val:1729061780518849000, v:0} max_elr_trans_id=392201977 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780518849000, v:0} min_modify_scn={val:1729061780518849000, v:0}}) [2024-10-16 14:57:24.391843] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f478558 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203013} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5233}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606764, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606764, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844387926000, v:0}, tx_id:{txid:392203013}, scn:{branch:0, seq:1549}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203013}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392203013}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844391610, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844386378, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:5233}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.391899] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203013} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203013}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:5233}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.391925] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=0] primary key duplication conflict(key={"BIGINT":675513900660805632},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203013} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.391938] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.391944] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513900660805632},{"BIGINT":1729058400},{"VARCHAR":"AT¼”@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.391998] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.392629] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.392644] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] Garbage Collector is running(seq_=656747, gc_interval=10000000) [2024-10-16 14:57:24.392672] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=6] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:24.392682] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=5] gc_check_member_list_ cost time(ret=0, time_us=32) [2024-10-16 14:57:24.392688] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=3] execute_gc cost time(ret=0, time_us=1) [2024-10-16 14:57:24.392693] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:24.392699] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:24.392704] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=6) [2024-10-16 14:57:24.392707] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=2] ObSafeDestroyHandler start process [2024-10-16 14:57:24.394100] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061844392764000, start_id=1729061844392764000, end_id=1729061844392764001) [2024-10-16 14:57:24.394690] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8083}, to_seq_no={branch:0, seq:1550}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203013} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.395939] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=11] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:24.397794] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:24.418218] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B5A0C6794-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061844417898000, start_id=1729061844417898000, end_id=1729061844417898001) [2024-10-16 14:57:24.419637] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:66390053505}], scn_array=[cnt:1, {val:1729061844396711001, v:0}], curr_size=815, accum_size=2657, time_cost=11514) [2024-10-16 14:57:24.421813] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C78-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:24.421828] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C78-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:24.429053] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=27, avg_log_batch_cnt=0, total_group_log_size=42341, avg_group_log_size=1568) [2024-10-16 14:57:24.430735] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:24.433110] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb30550 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203015} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2264}, prev=this=0x2b0fadabe2f0 trans_version={val:1729061780534968000, v:0} scn={val:1729061780534968000, v:0} tx_id={txid:392201979} prev=0x2b0fadabe1b0 next=(nil) modify_count=1 acc_checksum=3070807670 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7571}, snapshot_version={val:1729061844432564000, v:0}, *this={this=0x2b0fadabdea8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadabe2f0 latest_compact_node=(nil) max_trans_version={val:1729061780534968000, v:0} max_trans_id=392201979 max_elr_trans_version={val:1729061780534968000, v:0} max_elr_trans_id=392201979 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780534968000, v:0} min_modify_scn={val:1729061780534968000, v:0}}) [2024-10-16 14:57:24.433142] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb30550 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203015} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2264}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606276, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606276, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844432564000, v:0}, tx_id:{txid:392203015}, scn:{branch:0, seq:2261}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203015}, tx_desc:{this:0x2b0ec37cf8b0, tx_id:{txid:392203015}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844432564, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844430304, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2264}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.433206] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=63][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203015} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203015}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:2264}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.433220] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513900698554368},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203015} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.433232] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.433237] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513900698554368},{"BIGINT":1729058400},{"VARCHAR":"A8hŽ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.433292] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.435362] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4462}, to_seq_no={branch:0, seq:2262}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203015} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.437560] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3F-0-0] [lt=22] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:24.437584] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C3F-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061844428721000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061844428721000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:24.438967] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B5A0C6794-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061844438568000, start_id=1729061844438568000, end_id=1729061844438568001) [2024-10-16 14:57:24.444634] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C3F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844444174000, start_id=1729061844444174000, end_id=1729061844444174001) [2024-10-16 14:57:24.449044] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=60] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:24.449662] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:24.449707] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061838714540000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:24.450830] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844445260000, start_id=1729061844445260000, end_id=1729061844445260001) [2024-10-16 14:57:24.452975] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6E-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:24.452992] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6E-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:24.453735] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=571, replace_time=0, replace_start_pos=1604307, replace_num=31457) [2024-10-16 14:57:24.453763] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=571, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:24.461739] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061844459568000, start_id=1729061844459568000, end_id=1729061844459568001) [2024-10-16 14:57:24.467127] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=15][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497202, log_task={header:{begin_lsn:{lsn:207589443301}, end_lsn:{lsn:207589443414}, log_id:218497202, min_scn:{val:1729061844358688000, v:0}, max_scn:{val:1729061844358688000, v:0}, data_len:57, proposal_id:81, prev_lsn:{lsn:207589441677}, prev_proposal_id:81, committed_end_lsn:{lsn:207589443301}, data_checksum:1133849145, accum_checksum:3974218180, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061844364355, freeze_ts:1729061844364355, submit_ts:1729061844364359, flushed_ts:1729061844364738, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:379}, fs_cb_begin_ts=1729061844467120, log_life_time=102765) [2024-10-16 14:57:24.468649] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.471796] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb30c30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203018} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4633}, prev=this=0x2b0fadabe9f0 trans_version={val:1729061780550864000, v:0} scn={val:1729061780550864000, v:0} tx_id={txid:392201982} prev=0x2b0fadabe8b0 next=(nil) modify_count=1 acc_checksum=1494545975 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6461}, snapshot_version={val:1729061844467565000, v:0}, *this={this=0x2b0fadabe5a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadabe9f0 latest_compact_node=(nil) max_trans_version={val:1729061780550864000, v:0} max_trans_id=392201982 max_elr_trans_version={val:1729061780550864000, v:0} max_elr_trans_id=392201982 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780550864000, v:0} min_modify_scn={val:1729061780550864000, v:0}}) [2024-10-16 14:57:24.471828] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb30c30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203018} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4633}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606316, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606316, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844467565000, v:0}, tx_id:{txid:392203018}, scn:{branch:0, seq:1105}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203018}, tx_desc:{this:0x2b0ec37b7d30, tx_id:{txid:392203018}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844471093, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844466461, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4633}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.471882] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203018} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203018}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4633}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.471896] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513900761468928},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203018} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.471908] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.471914] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513900761468928},{"BIGINT":1729058400},{"VARCHAR":"@¾9", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.471968] INFO [STORAGE] print_count (ob_query_iterator_factory.cpp:50) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34] ObQueryIteratorFactory statistics(multi_scan_merge_alloc_count=0, multi_scan_merge_release_count=0, multi_get_merge_alloc_count=0, multi_get_merge_release_count=0, table_scan_alloc_count=0, table_scan_release_count=0, insert_dup_alloc_count=89263597, insert_dup_release_count=89263597, col_map_alloc_count=0, col_map_release_count=0) [2024-10-16 14:57:24.471981] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=8][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.480066] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [21736][T1001_COMMONLSS][T1001][YB4A0A000407-00061E9B3A29CAE9-0-0] [lt=20][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:24.482191] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:15630}, to_seq_no={branch:0, seq:1106}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203018} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.489326] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C82-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061844488019000, start_id=1729061844488019000, end_id=1729061844488019001) [2024-10-16 14:57:24.489736] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2736) [2024-10-16 14:57:24.492683] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.494253] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844493760000, start_id=1729061844493760000, end_id=1729061844493760001) [2024-10-16 14:57:24.498489] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=27] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=17, avg_log_batch_cnt=0, total_group_log_size=24592, avg_group_log_size=1446) [2024-10-16 14:57:24.509557] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061844508168000, start_id=1729061844508168000, end_id=1729061844508168001) [2024-10-16 14:57:24.509629] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=5296, avg_group_log_size=441) [2024-10-16 14:57:24.509870] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=29] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=2321, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:24.509897] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=2823, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:24.510264] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C9C-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=43, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:24.510274] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C9C-0-0] [lt=9] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=3047, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:24.510277] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C9C-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:24.510285] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C9C-0-0] [lt=7] [APPLY STAT CB EXECUTE TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=-476, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:24.510288] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3C9C-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=2614, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:24.510693] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be0cbf0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203022} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1564}, prev=this=0x2b0ec6d6d908 trans_version={val:1729061780567121000, v:0} scn={val:1729061780567121000, v:0} tx_id={txid:392201986} prev=0x2b0ec6d6d7c8 next=(nil) modify_count=1 acc_checksum=1906597853 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3148}, snapshot_version={val:1729061844509728000, v:0}, *this={this=0x2b0ec6d6d4c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d6d908 latest_compact_node=(nil) max_trans_version={val:1729061780567121000, v:0} max_trans_id=392201986 max_elr_trans_version={val:1729061780567121000, v:0} max_elr_trans_id=392201986 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780567121000, v:0} min_modify_scn={val:1729061780567121000, v:0}}) [2024-10-16 14:57:24.510722] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be0cbf0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203022} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1564}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606256, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606256, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844509728000, v:0}, tx_id:{txid:392203022}, scn:{branch:0, seq:1561}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203022}, tx_desc:{this:0x2b0ec37a2150, tx_id:{txid:392203022}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844509728, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844508170, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1564}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.510776] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203022} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203022}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:1564}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.510794] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513900836966400},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203022} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.510806] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.510811] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513900836966400},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.510860] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.517263] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8040}, to_seq_no={branch:0, seq:1562}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203022} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.518572] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061844518343000, start_id=1729061844518343000, end_id=1729061844518343001) [2024-10-16 14:57:24.518588] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:24.518605] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=505, get_gts_with_stc_cnt=75, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=75, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:24.518613] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061844518595], gts:1729061844518343000, latest_srr:[mts=1729061844518595]}) [2024-10-16 14:57:24.518770] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=513, get_gts_with_stc_cnt=9, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:24.523081] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C7A-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172977, log_task={header:{begin_lsn:{lsn:14352353406}, end_lsn:{lsn:14352353528}, log_id:115172977, min_scn:{val:1729061844128676002, v:0}, max_scn:{val:1729061844128676002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352353284}, prev_proposal_id:17, committed_end_lsn:{lsn:14352353406}, data_checksum:940009244, accum_checksum:540877559, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061844421809, freeze_ts:1729061844421809, submit_ts:1729061844421824, flushed_ts:1729061844422389, gen_to_freeze cost time:0, gen_to_submit cost time:15, submit_to_flush cost time:565}, fs_cb_begin_ts=1729061844523075, log_life_time=101266) [2024-10-16 14:57:24.523156] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=29] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=53, stat_interval=1000000, avg (count/cost)=300, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:24.523164] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=53, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:24.528827] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061844528010000, start_id=1729061844528010000, end_id=1729061844528010001) [2024-10-16 14:57:24.531892] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:24.531908] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:24.531926] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444258474, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:24.536138] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:24.538645] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=10] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=902713344, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=171028480, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:24.538710] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=25] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:24.538718] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:24.538759] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=13] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=967602176, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=106139648, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:24.538793] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:24.538798] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:24.538827] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=940462080, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=133279744, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:24.538863] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:24.538867] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011855, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:24.538897] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=956841984, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=116899840, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:24.538929] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:24.538933] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:24.552553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061844550678000, start_id=1729061844550678000, end_id=1729061844550678001) [2024-10-16 14:57:24.552634] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:24.552648] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:24.568714] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.569619] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.570684] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22405][T1003_DupTbLeas][T1003][Y0-0000000000000000-0-0] [lt=8][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:24.570743] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22405][T1003_DupTbLeas][T1003][Y0-0000000000000000-0-0] [lt=11][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1003 and flag like "%DUPLICATE%", tenant_id=1003, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:24.571628] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.571647] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.571659] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.576198] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f542288 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203030} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3204}, prev=this=0x2b0f7f4b7ec0 trans_version={val:1729061780613211000, v:0} scn={val:1729061780613211000, v:0} tx_id={txid:392201994} prev=0x2b0f7f4b7d80 next=(nil) modify_count=1 acc_checksum=2450229236 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5415}, snapshot_version={val:1729061844575088000, v:0}, *this={this=0x2b0f7f4b7a78 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4b7ec0 latest_compact_node=(nil) max_trans_version={val:1729061780613211000, v:0} max_trans_id=392201994 max_elr_trans_version={val:1729061780613211000, v:0} max_elr_trans_id=392201994 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780613211000, v:0} min_modify_scn={val:1729061780613211000, v:0}}) [2024-10-16 14:57:24.576230] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f542288 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203030} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3204}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606252, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606252, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844575088000, v:0}, tx_id:{txid:392203030}, scn:{branch:0, seq:3201}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203030}, tx_desc:{this:0x2b0ec37bbc70, tx_id:{txid:392203030}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844575088, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844571888, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3204}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.576286] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203030} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203030}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:3204}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.576301] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513900987961344},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203030} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.576313] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.576319] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513900987961344},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.576375] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.577393] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:24.577414] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:24.581127] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:8938}, to_seq_no={branch:0, seq:3202}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203030} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.581678] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1003}, {id:1001}]) [2024-10-16 14:57:24.585170] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B29-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061844583984000, start_id=1729061844583984000, end_id=1729061844583984001) [2024-10-16 14:57:24.585591] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23443965, this={dio_aligned_buf:{buf_write_offset:3103, buf_padding_size:993, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:47003900, truncate_used_ts:19010228}, log_block_size:67108864, dir_fd:134, io_fd:183}, aligned_buf_len=4096, aligned_block_offset=23441408, buf_len=546, total_write_size_=40704, total_write_size_after_dio_=479232, ob_pwrite_used_ts=61131, count=107, each_pwrite_cost=571) [2024-10-16 14:57:24.587772] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1DFC-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061844586423000, start_id=1729061844586423000, end_id=1729061844586423001) [2024-10-16 14:57:24.589010] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=52240690, this={dio_aligned_buf:{buf_write_offset:855, buf_padding_size:3241, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80310756, truncate_used_ts:29009417}, log_block_size:67108864, dir_fd:190, io_fd:179}, aligned_buf_len=4096, aligned_block_offset=52240384, buf_len=549, total_write_size_=104789, total_write_size_after_dio_=864256, ob_pwrite_used_ts=65504, count=185, each_pwrite_cost=354) [2024-10-16 14:57:24.592754] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:24.592772] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:24.592779] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.593380] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C40B6-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=1218, avg_log_submit_to_slide_time=2188) [2024-10-16 14:57:24.594286] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844592443000, start_id=1729061844592443000, end_id=1729061844592443001) [2024-10-16 14:57:24.598019] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4623, mem_used=8318976) [2024-10-16 14:57:24.603713] INFO [ARCHIVE] do_thread_task_ (ob_archive_fetcher.cpp:310) [22814][T1004_ArcFetche][T1004][YB4A0A000407-00061E9972D960C1-0-0] [lt=3] ObArchiveFetcher is running(thread_index=1) [2024-10-16 14:57:24.607485] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D521B-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:24.607501] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D521B-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=8591, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:24.607525] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=37471, hit_cnt=24, read_cnt=24, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:24.617015] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eec968 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203033} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6587}, prev=this=0x2b0f22ea1cd8 trans_version={val:1729061780630242001, v:0} scn={val:1729061780630242001, v:0} tx_id={txid:392201997} prev=0x2b0f22ea1b98 next=(nil) modify_count=1 acc_checksum=1252910652 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:18}, snapshot_version={val:1729061844616510000, v:0}, *this={this=0x2b0f22ea1890 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea1cd8 latest_compact_node=(nil) max_trans_version={val:1729061780630242001, v:0} max_trans_id=392201997 max_elr_trans_version={val:1729061780630242001, v:0} max_elr_trans_id=392201997 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780630242001, v:0} min_modify_scn={val:1729061780630242001, v:0}}) [2024-10-16 14:57:24.617051] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=35][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eec968 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203033} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6587}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606286, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606286, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844616510000, v:0}, tx_id:{txid:392203033}, scn:{branch:0, seq:6584}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203033}, tx_desc:{this:0x2b0ec37b9cd0, tx_id:{txid:392203033}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844616510, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844609927, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:6587}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.617112] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203033} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203033}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:6587}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.617127] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513901063458816},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203033} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.617139] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.617146] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901063458816},{"BIGINT":1729058400},{"VARCHAR":"A3Õš", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.617202] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.622227] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:11820}, to_seq_no={branch:0, seq:6585}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203033} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.624911] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5C-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:24.624936] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5C-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061844127564002, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061844127564002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:24.629795] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BAB-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061844629288000, start_id=1729061844629288000, end_id=1729061844629288001) [2024-10-16 14:57:24.629827] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BAF-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061844629288000, start_id=1729061844629288000, end_id=1729061844629288001) [2024-10-16 14:57:24.629904] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BAC-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061844629288000, start_id=1729061844629288000, end_id=1729061844629288001) [2024-10-16 14:57:24.629978] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BAD-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061844629288000, start_id=1729061844629288000, end_id=1729061844629288001) [2024-10-16 14:57:24.630109] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D5C-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061844629288000, start_id=1729061844629288000, end_id=1729061844629288001) [2024-10-16 14:57:24.631365] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FC70-0-0] [lt=21] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=9, avg_log_gen_to_submit_time=38, avg_log_submit_to_flush_time=1410, avg_log_submit_to_slide_time=1947) [2024-10-16 14:57:24.633074] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061844632974000, start_id=1729061844632974000, end_id=1729061844632974001) [2024-10-16 14:57:24.635172] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=16] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:24.635652] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D5C-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061844635402000, start_id=1729061844635402000, end_id=1729061844635402001) [2024-10-16 14:57:24.652962] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:24.653305] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061844651768000, start_id=1729061844651768000, end_id=1729061844651768001) [2024-10-16 14:57:24.655274] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=605, replace_time=0, replace_start_pos=1635764, replace_num=31457) [2024-10-16 14:57:24.655300] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=605, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:24.662550] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061844661364000, start_id=1729061844661364000, end_id=1729061844661364001) [2024-10-16 14:57:24.662819] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AFD-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:24.662828] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AFD-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=38943, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:24.662833] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AFD-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=2562, avg_log_submit_to_slide_time=38937) [2024-10-16 14:57:24.662840] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2AFD-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178285, log_task={header:{begin_lsn:{lsn:14538467302}, end_lsn:{lsn:14538467424}, log_id:115178285, min_scn:{val:1729061844455920000, v:0}, max_scn:{val:1729061844455920000, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538467180}, prev_proposal_id:23, committed_end_lsn:{lsn:14538467302}, data_checksum:1715074492, accum_checksum:242355806, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061844472848, freeze_ts:1729061844472848, submit_ts:1729061844472851, flushed_ts:1729061844474363, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:1512}, fs_cb_begin_ts=1729061844662812, log_life_time=189964) [2024-10-16 14:57:24.663850] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:24.664160] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=33] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:24.665163] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=15] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:57:24.671174] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be0d5d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203038} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3139}, prev=this=0x2b0ec6d6e008 trans_version={val:1729061780667298000, v:0} scn={val:1729061780667298000, v:0} tx_id={txid:392202002} prev=0x2b0ec6d6dec8 next=(nil) modify_count=1 acc_checksum=919578128 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4250}, snapshot_version={val:1729061844670421000, v:0}, *this={this=0x2b0ec6d6dbc0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d6e008 latest_compact_node=(nil) max_trans_version={val:1729061780667298000, v:0} max_trans_id=392202002 max_elr_trans_version={val:1729061780667298000, v:0} max_elr_trans_id=392202002 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780667298000, v:0} min_modify_scn={val:1729061780667298000, v:0}}) [2024-10-16 14:57:24.671213] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be0d5d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203038} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3139}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606267, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606267, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844670421000, v:0}, tx_id:{txid:392203038}, scn:{branch:0, seq:3136}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203038}, tx_desc:{this:0x2b0ec37d1850, tx_id:{txid:392203038}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844670421, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844667286, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3139}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.671277] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=64][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203038} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203038}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:3139}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.671292] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513901151539200},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203038} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.671304] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.671319] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901151539200},{"BIGINT":1729058400},{"VARCHAR":"A8P", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.671381] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.672839] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=8, avg_log_batch_cnt=1, total_group_log_size=1828, avg_group_log_size=228) [2024-10-16 14:57:24.672852] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1828) [2024-10-16 14:57:24.674977] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.675039] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7473}, to_seq_no={branch:0, seq:3137}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203038} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.681917] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061840675208, last_first_register_time_us=1722494150998907, register_time_us=1729061840675208, curr_time_us=1729061844681915) [2024-10-16 14:57:24.681959] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=24] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061844681942, last_submit_register_req_time_us=1729061844681942, register_parent_reason=3) [2024-10-16 14:57:24.681967] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:24.691799] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=9] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:24.692622] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061844691661000, start_id=1729061844691661000, end_id=1729061844691661001) [2024-10-16 14:57:24.694966] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22398][T1003_TntShared][T1003][Y0-0000000000000000-0-0] [lt=16] clear dtl interm result cost(us)(clear_cost=16, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:24.695697] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:24.695823] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.696624] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B1C-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844695885000, start_id=1729061844695885001, end_id=1729061844695885002) [2024-10-16 14:57:24.696921] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=42702366, this={dio_aligned_buf:{buf_write_offset:2114, buf_padding_size:1982, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35133142, truncate_used_ts:12254300}, log_block_size:67108864, dir_fd:169, io_fd:144}, aligned_buf_len=4096, aligned_block_offset=42700800, buf_len=548, total_write_size_=11889, total_write_size_after_dio_=303104, ob_pwrite_used_ts=42844, count=71, each_pwrite_cost=603) [2024-10-16 14:57:24.698272] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb31310 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203040} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2135}, prev=this=0x2b0fadabf0f0 trans_version={val:1729061780681614000, v:0} scn={val:1729061780681614000, v:0} tx_id={txid:392202004} prev=0x2b0fadabefb0 next=(nil) modify_count=1 acc_checksum=3476380219 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4837}, snapshot_version={val:1729061844696964000, v:0}, *this={this=0x2b0fadabeca8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadabf0f0 latest_compact_node=(nil) max_trans_version={val:1729061780681614000, v:0} max_trans_id=392202004 max_elr_trans_version={val:1729061780681614000, v:0} max_elr_trans_id=392202004 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780681614000, v:0} min_modify_scn={val:1729061780681614000, v:0}}) [2024-10-16 14:57:24.698312] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=39][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb31310 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203040} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2135}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606282, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606282, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844696964000, v:0}, tx_id:{txid:392203040}, scn:{branch:0, seq:1078}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203040}, tx_desc:{this:0x2b0ec37b1e50, tx_id:{txid:392203040}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844698023, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844695889, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2135}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.698369] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203040} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203040}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:2135}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.698384] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513901201870848},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203040} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.698396] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.698402] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901201870848},{"BIGINT":1729058400},{"VARCHAR":"A}ô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.698451] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.698827] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2138}, to_seq_no={branch:0, seq:1079}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203040} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.707242] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31896][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB1FCC6D9-0-0] [lt=30] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:24.709274] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f542f78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203042} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0f7f4b8d70 trans_version={val:1729061780696597000, v:0} scn={val:1729061780696597000, v:0} tx_id={txid:392202006} prev=0x2b0f7f4b8c30 next=(nil) modify_count=1 acc_checksum=1913462952 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7471}, snapshot_version={val:1729061844708197000, v:0}, *this={this=0x2b0f7f4b8928 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4b8d70 latest_compact_node=(nil) max_trans_version={val:1729061780696597000, v:0} max_trans_id=392202006 max_elr_trans_version={val:1729061780696597000, v:0} max_elr_trans_id=392202006 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780696597000, v:0} min_modify_scn={val:1729061780696597000, v:0}}) [2024-10-16 14:57:24.709307] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f542f78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203042} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606276, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606276, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844708197000, v:0}, tx_id:{txid:392203042}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203042}, tx_desc:{this:0x2b0ec37cb970, tx_id:{txid:392203042}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844708197, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844708198, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.709383] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=75][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203042} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203042}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.709398] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":675513901239619584},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203042} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.709410] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.709417] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901239619584},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.709472] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.709748] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1332}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203042} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.711631] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061844508168000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=203455, in_cluster_service=true, cluster_version={val:1729061844017195000, v:0}, min_cluster_version={val:1729061844017195000, v:0}, max_cluster_version={val:1729061844017195000, v:0}, get_cluster_version_err=0, cluster_version_delta=694428, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65248730, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=341984338, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:24.712575] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061844710612000, start_id=1729061844710612000, end_id=1729061844710612001) [2024-10-16 14:57:24.713170] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=38] success to update min reserved snapshot(reserved_snapshot=1729060041826930000, duration=1800, snapshot_gc_ts=1729061841826930000) [2024-10-16 14:57:24.715540] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:24.715562] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=36] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=209) [2024-10-16 14:57:24.717141] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=24] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060041826930000) [2024-10-16 14:57:24.722742] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:24.724969] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23134735, this={dio_aligned_buf:{buf_write_offset:639, buf_padding_size:3457, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33384857, truncate_used_ts:14052825}, log_block_size:67108864, dir_fd:200, io_fd:174}, aligned_buf_len=4096, aligned_block_offset=23134208, buf_len=112, total_write_size_=12284, total_write_size_after_dio_=425984, ob_pwrite_used_ts=34080, count=101, each_pwrite_cost=337) [2024-10-16 14:57:24.726667] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=11] submit reserved snapshot log success(ls_id={id:1001}, new_snapshot_version=1729060041826930000) [2024-10-16 14:57:24.726682] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=12] update reserved snapshot finished(cost_ts=13515, reserved_snapshot=1729060041826930000) [2024-10-16 14:57:24.729569] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=35958) [2024-10-16 14:57:24.730725] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:24.730738] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:24.733188] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061844733092000, start_id=1729061844733092000, end_id=1729061844733092001) [2024-10-16 14:57:24.736463] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:24.740540] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:24.741897] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=37581, hit_cnt=25, read_cnt=25, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:24.742175] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=263, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:24.742187] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=898, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:24.742216] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:24.753273] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:24.753308] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=19] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:24.753316] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:24.754539] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061844743126000, start_id=1729061844743126000, end_id=1729061844743126001) [2024-10-16 14:57:24.755291] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb319f0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203045} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3365}, prev=this=0x2b0fadabf7f0 trans_version={val:1729061780712289000, v:0} scn={val:1729061780712289000, v:0} tx_id={txid:392202009} prev=0x2b0fadabf6b0 next=(nil) modify_count=1 acc_checksum=2821766335 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7425}, snapshot_version={val:1729061844740790000, v:0}, *this={this=0x2b0fadabf3a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadabf7f0 latest_compact_node=(nil) max_trans_version={val:1729061780712289000, v:0} max_trans_id=392202009 max_elr_trans_version={val:1729061780712289000, v:0} max_elr_trans_id=392202009 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780712289000, v:0} min_modify_scn={val:1729061780712289000, v:0}}) [2024-10-16 14:57:24.755322] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb319f0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203045} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3365}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437617452, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437617452, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844740790000, v:0}, tx_id:{txid:392203045}, scn:{branch:0, seq:1029}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203045}, tx_desc:{this:0x2b0ec37b5d90, tx_id:{txid:392203045}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844743126, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844739762, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3365}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.755376] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203045} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203045}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:3365}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.755391] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513901327699968},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203045} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.755411] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.755417] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901327699968},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.755459] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.756776] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:16836}, to_seq_no={branch:0, seq:1030}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203045} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.763088] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061844762282000, start_id=1729061844762282000, end_id=1729061844762282001) [2024-10-16 14:57:24.764659] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f543658 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203046} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, prev=this=0x2b0f7f4b9470 trans_version={val:1729061780721843000, v:0} scn={val:1729061780721843000, v:0} tx_id={txid:392202010} prev=0x2b0f7f4b9330 next=(nil) modify_count=1 acc_checksum=2687560014 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5272}, snapshot_version={val:1729061844763366000, v:0}, *this={this=0x2b0f7f4b9028 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4b9470 latest_compact_node=(nil) max_trans_version={val:1729061780721843000, v:0} max_trans_id=392202010 max_elr_trans_version={val:1729061780721843000, v:0} max_elr_trans_id=392202010 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780721843000, v:0} min_modify_scn={val:1729061780721843000, v:0}}) [2024-10-16 14:57:24.764693] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f543658 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203046} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606269, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606269, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844763366000, v:0}, tx_id:{txid:392203046}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203046}, tx_desc:{this:0x2b0ec37a6090, tx_id:{txid:392203046}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844764425, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844763366, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1060}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.764751] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203046} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203046}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1060}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.764765] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513901365448704},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203046} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.764778] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.764783] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901365448704},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.764842] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=36][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.768941] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:24.773177] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:9509}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203046} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.775045] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.780676] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:24.788304] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.788325] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.788341] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.788353] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.788364] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.788380] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.788492] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.788503] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.788525] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.789638] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.789684] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:24.794412] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061844792473000, start_id=1729061844792473000, end_id=1729061844792473001) [2024-10-16 14:57:24.794481] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:24.794494] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=9998, avg_group_log_size=624) [2024-10-16 14:57:24.794513] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9998) [2024-10-16 14:57:24.794521] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:24.794683] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=2813, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:24.794700] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=589, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:24.794721] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:24.794948] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93B0-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10590) [2024-10-16 14:57:24.795651] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:24.796520] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.798017] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=11] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061840757606, last_first_register_time_us=1722494157071428, register_time_us=1729061840757606, curr_time_us=1729061844798016) [2024-10-16 14:57:24.798058] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=27] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061844798044, last_submit_register_req_time_us=1729061844798044, register_parent_reason=3) [2024-10-16 14:57:24.798069] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:24.813150] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:24.821079] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=9] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1003}, {id:1002}, {id:1}]) [2024-10-16 14:57:24.823129] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C80-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:24.823142] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C80-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=43926, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:24.823147] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C80-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=699, avg_log_submit_to_slide_time=43920) [2024-10-16 14:57:24.823157] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C80-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172980, log_task={header:{begin_lsn:{lsn:14352353772}, end_lsn:{lsn:14352353894}, log_id:115172980, min_scn:{val:1729061844629288001, v:0}, max_scn:{val:1729061844629288001, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352353650}, prev_proposal_id:17, committed_end_lsn:{lsn:14352353772}, data_checksum:940009244, accum_checksum:2016730433, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061844723050, freeze_ts:1729061844723050, submit_ts:1729061844723054, flushed_ts:1729061844724938, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1884}, fs_cb_begin_ts=1729061844823122, log_life_time=100072) [2024-10-16 14:57:24.823250] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:24.823285] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656782, miss_cnt:17, cache_read_size:8050144115, hit_ratio:"9.999997286806815433e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:24.823397] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=59042406, this={dio_aligned_buf:{buf_write_offset:2784, buf_padding_size:1312, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:38334718, truncate_used_ts:14110944}, log_block_size:67108864, dir_fd:207, io_fd:202}, aligned_buf_len=4096, aligned_block_offset=59039744, buf_len=122, total_write_size_=12172, total_write_size_after_dio_=421888, ob_pwrite_used_ts=29321, count=100, each_pwrite_cost=293) [2024-10-16 14:57:24.823418] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=291, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:24.823431] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1454) [2024-10-16 14:57:24.823441] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=360, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:24.823476] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:24.825106] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=54] success to update min reserved snapshot(reserved_snapshot=1729060037234027000, duration=1800, snapshot_gc_ts=1729061837234027000) [2024-10-16 14:57:24.825739] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=14] update reserved snapshot finished(cost_ts=636, reserved_snapshot=1729060037234027000) [2024-10-16 14:57:24.830324] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f543d38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203053} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3143}, prev=this=0x2b0f7f4b9b70 trans_version={val:1729061780746089000, v:0} scn={val:1729061780746089000, v:0} tx_id={txid:392202017} prev=0x2b0f7f4b9a30 next=(nil) modify_count=1 acc_checksum=2064661527 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6517}, snapshot_version={val:1729061844828871000, v:0}, *this={this=0x2b0f7f4b9728 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4b9b70 latest_compact_node=(nil) max_trans_version={val:1729061780746089000, v:0} max_trans_id=392202017 max_elr_trans_version={val:1729061780746089000, v:0} max_elr_trans_id=392202017 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780746089000, v:0} min_modify_scn={val:1729061780746089000, v:0}}) [2024-10-16 14:57:24.830370] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=45][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f543d38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203053} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3143}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606296, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606296, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844828871000, v:0}, tx_id:{txid:392203053}, scn:{branch:0, seq:2085}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203053}, tx_desc:{this:0x2b0ec37a40f0, tx_id:{txid:392203053}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844829929, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844826787, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3143}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.830429] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203053} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203053}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:3143}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.830446] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513901554192384},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203053} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.830461] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.830468] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901554192384},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.830519] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.831056] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=268, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:24.831070] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1246, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:24.833298] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6301}, to_seq_no={branch:0, seq:2086}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203053} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.837090] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061844836270000, start_id=1729061844836270000, end_id=1729061844836270001) [2024-10-16 14:57:24.839580] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb320d0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203054} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3196}, prev=this=0x2b0fadabfef0 trans_version={val:1729061780755245000, v:0} scn={val:1729061780755245000, v:0} tx_id={txid:392202018} prev=0x2b0fadabfdb0 next=(nil) modify_count=1 acc_checksum=1151956139 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7037}, snapshot_version={val:1729061844836270000, v:0}, *this={this=0x2b0fadabfaa8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadabfef0 latest_compact_node=(nil) max_trans_version={val:1729061780755245000, v:0} max_trans_id=392202018 max_elr_trans_version={val:1729061780755245000, v:0} max_elr_trans_id=392202018 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780755245000, v:0} min_modify_scn={val:1729061780755245000, v:0}}) [2024-10-16 14:57:24.839623] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=41][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb320d0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203054} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3196}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606301, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606301, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844836270000, v:0}, tx_id:{txid:392203054}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203054}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392203054}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844839465, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844836270, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3196}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.839683] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203054} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203054}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:3196}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.839701] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513901579358208},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203054} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.839716] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.839722] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901579358208},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.839778] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.839983] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3199}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203054} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.850772] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3377, mem_used=6239232) [2024-10-16 14:57:24.854435] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061844853285000, start_id=1729061844853285000, end_id=1729061844853285001) [2024-10-16 14:57:24.854738] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10589) [2024-10-16 14:57:24.855431] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93B1-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:24.855443] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93B1-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4462, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:24.855447] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93B1-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=3305, avg_log_submit_to_slide_time=4429) [2024-10-16 14:57:24.856888] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] Cache replace map node details(ret=0, replace_node_count=466, replace_time=0, replace_start_pos=1667221, replace_num=31457) [2024-10-16 14:57:24.856914] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] replace map num details(ret=0, replace_node_count=466, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:24.862821] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061844862120000, start_id=1729061844862120000, end_id=1729061844862120001) [2024-10-16 14:57:24.870738] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be0e5d0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203060} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0ec6d6f290 trans_version={val:1729061780781350001, v:0} scn={val:1729061780781350001, v:0} tx_id={txid:392202024} prev=0x2b0ec6d6f150 next=(nil) modify_count=1 acc_checksum=2408122896 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5351}, snapshot_version={val:1729061844869531000, v:0}, *this={this=0x2b0ec6d6ee48 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d6f290 latest_compact_node=(nil) max_trans_version={val:1729061780781350001, v:0} max_trans_id=392202024 max_elr_trans_version={val:1729061780781350001, v:0} max_elr_trans_id=392202024 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780781350001, v:0} min_modify_scn={val:1729061780781350001, v:0}}) [2024-10-16 14:57:24.870771] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be0e5d0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203060} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606371, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606371, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844869531000, v:0}, tx_id:{txid:392203060}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203060}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392203060}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844870627, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844869531, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.870826] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203060} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203060}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.870841] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":675513901717770240},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203060} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.870853] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.870859] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901717770240},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.870914] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.875270] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.876285] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6579}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203060} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.883525] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=18] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:24.883965] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061844882885000, start_id=1729061844882885000, end_id=1729061844882885001) [2024-10-16 14:57:24.884034] INFO [LIB] stat (utility.h:1165) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:24.884048] INFO [LIB] stat (utility.h:1165) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:24.884469] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=839, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:24.884480] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=30973) [2024-10-16 14:57:24.884491] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=541, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:24.884516] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=30971) [2024-10-16 14:57:24.884526] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:24.896583] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.896921] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:24.896938] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=6267) [2024-10-16 14:57:24.896946] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:24.898134] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f478c38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203061} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:12647}, prev=this=0x2b0e5abfcc08 trans_version={val:1729061780793740000, v:0} scn={val:1729061780793740000, v:0} tx_id={txid:392202025} prev=0x2b0e5abfcac8 next=(nil) modify_count=1 acc_checksum=2802232450 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5343}, snapshot_version={val:1729061844896067000, v:0}, *this={this=0x2b0e5abfc7c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abfcc08 latest_compact_node=(nil) max_trans_version={val:1729061780793740000, v:0} max_trans_id=392202025 max_elr_trans_version={val:1729061780793740000, v:0} max_elr_trans_id=392202025 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780793740000, v:0} min_modify_scn={val:1729061780793740000, v:0}}) [2024-10-16 14:57:24.898174] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=39][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f478c38 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203061} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:12647}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606431, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606431, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844896067000, v:0}, tx_id:{txid:392203061}, scn:{branch:0, seq:11616}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203061}, tx_desc:{this:0x2b0ec37cf8b0, tx_id:{txid:392203061}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844897096, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844884452, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:12647}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.898233] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203061} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203061}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:12647}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.898251] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513901730353152},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203061} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.898265] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.898271] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901730353152},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.898324] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.898893] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:13706}, to_seq_no={branch:0, seq:11617}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203061} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.904948] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD1-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061844871845, keepalive_ts:-1}) [2024-10-16 14:57:24.904969] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD1-0-0] [lt=20] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061844871845, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:24.907650] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=26, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:24.908194] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=65] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561098, miss_cnt:1, cache_read_size:152590625432, hit_ratio:"9.999999928346795519e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:24.910265] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 2.99, rt: 636.00; ) [2024-10-16 14:57:24.910286] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:24.910324] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:24.910909] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2348-0-0] [lt=25] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:57:24.911447] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=21][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb327b0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203063} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2114}, prev=this=0x2b0fadac05f0 trans_version={val:1729061780804359000, v:0} scn={val:1729061780804359000, v:0} tx_id={txid:392202031} prev=0x2b0fadac04b0 next=(nil) modify_count=1 acc_checksum=4138092798 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4222}, snapshot_version={val:1729061844908834000, v:0}, *this={this=0x2b0fadac01a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac05f0 latest_compact_node=(nil) max_trans_version={val:1729061780804359000, v:0} max_trans_id=392202031 max_elr_trans_version={val:1729061780804359000, v:0} max_elr_trans_id=392202031 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780804359000, v:0} min_modify_scn={val:1729061780804359000, v:0}}) [2024-10-16 14:57:24.911476] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb327b0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203063} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2114}, ctx={this:0x2b0e2b854080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606289, tablet_id:{id:209279}, table_iter:0x2b0e2b853210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606289, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844908834000, v:0}, tx_id:{txid:392203063}, scn:{branch:0, seq:1058}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203063}, tx_desc:{this:0x2b0e0fcc01b0, tx_id:{txid:392203063}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844909890, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844907777, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2114}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.911530] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203063} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203063}, data:0x2b0e2b850438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:2114}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.911545] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513901768101888},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203063} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.911556] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.911562] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901768101888},{"BIGINT":1729058400},{"VARCHAR":"?ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.911616] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.911696] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2348-0-0] [lt=10] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1004) [2024-10-16 14:57:24.911705] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2348-0-0] [lt=8] no transfer partition job [2024-10-16 14:57:24.911712] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2348-0-0] [lt=3] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:57:24.912064] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3981}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203063} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.913196] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2348-0-0] [lt=6] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1004, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:57:24.913222] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2348-0-0] [lt=20] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:57:24.914555] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2348-0-0] [lt=15] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1002, status:0}], ls_array_=[{tenant_id:1004, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729059563578310, last_statistic_bg_stat_time=1725269669481368, last_statistic_schema_version=1725266205820024, last_statistic_max_transfer_task_id={id:5}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:57:24.915505] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061844914965000, start_id=1729061844914965000, end_id=1729061844914965001) [2024-10-16 14:57:24.915747] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061844914965000, start_id=1729061844914965000, end_id=1729061844914965001) [2024-10-16 14:57:24.916241] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:24.917129] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15CC-0-0] [lt=12] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:24.917390] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f479318 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203064} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1054}, prev=this=0x2b0e5abfd308 trans_version={val:1729061780813156002, v:0} scn={val:1729061780813156002, v:0} tx_id={txid:392202033} prev=0x2b0e5abfd1c8 next=(nil) modify_count=1 acc_checksum=2676597234 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7734}, snapshot_version={val:1729061844916027000, v:0}, *this={this=0x2b0e5abfcec0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abfd308 latest_compact_node=(nil) max_trans_version={val:1729061780813156002, v:0} max_trans_id=392202033 max_elr_trans_version={val:1729061780813156002, v:0} max_elr_trans_id=392202033 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780813156002, v:0} min_modify_scn={val:1729061780813156002, v:0}}) [2024-10-16 14:57:24.917417] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f479318 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203064} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1054}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606305, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606305, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844916027000, v:0}, tx_id:{txid:392203064}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203064}, tx_desc:{this:0x2b0ec37bbc70, tx_id:{txid:392203064}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844917082, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844916029, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1054}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.917474] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203064} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203064}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:1054}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.917488] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513901793267712},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203064} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.917499] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.917504] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901793267712},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.917551] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.917744] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=29][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:24.917783] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601320, offset=9606372}, end_cursor:ObLogCursor{file_id=192, log_id=47601321, offset=9606656}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c2c050, len:2844, log_data_len:284, seq:47601320, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:24.918052] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1057}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203064} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.920280] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA493-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061844918107000, start_id=1729061844918107000, end_id=1729061844918107001) [2024-10-16 14:57:24.920698] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:24.921281] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601321, offset=9606656}, end_cursor:ObLogCursor{file_id=192, log_id=47601322, offset=9606930}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c58050, len:2560, log_data_len:274, seq:47601321, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:24.923563] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257686, offset=39011580}, end_cursor:ObLogCursor{file_id=197, log_id=49257687, offset=39011863}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a30050, len:2820, log_data_len:283, seq:49257686, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:24.925613] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22628][T1004_TeRec][T1004][YB4A0A000407-00061E9B221FB35E-0-0] [lt=16] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=0, MAX_GAP=900000000000, new_sync_scn={val:1729061843668207001, v:0}, new_replayable_scn={val:1729061843668207001, v:0}, new_readable_scn={val:1729061843668207001, v:0}, old_tenant_info={tenant_id:1004, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061842516796003, v:0}, replayable_scn:{val:1729061842516796003, v:0}, standby_scn:{val:1729061842516796003, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"ARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:57:24.925820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB35E-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061844925435000, start_id=1729061844925435000, end_id=1729061844925435001) [2024-10-16 14:57:24.926285] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f544418 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203065} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5015}, prev=this=0x2b0f7f4ba970 trans_version={val:1729061780820536000, v:0} scn={val:1729061780820536000, v:0} tx_id={txid:392202034} prev=0x2b0f7f4ba830 next=(nil) modify_count=1 acc_checksum=227002979 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3837}, snapshot_version={val:1729061844923326000, v:0}, *this={this=0x2b0f7f4ba528 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4ba970 latest_compact_node=(nil) max_trans_version={val:1729061780820536000, v:0} max_trans_id=392202034 max_elr_trans_version={val:1729061780820536000, v:0} max_elr_trans_id=392202034 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780820536000, v:0} min_modify_scn={val:1729061780820536000, v:0}}) [2024-10-16 14:57:24.926314] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f544418 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203065} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5015}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437607101, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437607101, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844923326000, v:0}, tx_id:{txid:392203065}, scn:{branch:0, seq:3940}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203065}, tx_desc:{this:0x2b0ec37afeb0, tx_id:{txid:392203065}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844925435, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844920421, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:5015}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.926372] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203065} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203065}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:5015}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.926386] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513901805850624},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203065} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.926398] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.926403] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901805850624},{"BIGINT":1729058400},{"VARCHAR":"?ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.926452] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.926671] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601322, offset=9606930}, end_cursor:ObLogCursor{file_id=192, log_id=47601323, offset=9607204}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b08050, len:2286, log_data_len:274, seq:47601322, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:24.927012] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257687, offset=39011863}, end_cursor:ObLogCursor{file_id=197, log_id=49257688, offset=39012145}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a7c050, len:2537, log_data_len:282, seq:49257687, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:24.927045] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=33] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601323, offset=9607204}, end_cursor:ObLogCursor{file_id=192, log_id=47601324, offset=9607478}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a6c050, len:2012, log_data_len:274, seq:47601323, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:24.927179] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598669, miss_cnt:9, cache_read_size:149777838706, hit_ratio:"9.999999345923948901e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:24.930853] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:9639}, to_seq_no={branch:0, seq:3941}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203065} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.931579] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257688, offset=39012145}, end_cursor:ObLogCursor{file_id=197, log_id=49257689, offset=39012427}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2998050, len:2255, log_data_len:282, seq:49257688, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:24.931873] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257689, offset=39012427}, end_cursor:ObLogCursor{file_id=197, log_id=49257690, offset=39012709}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2aa0050, len:1973, log_data_len:282, seq:49257689, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:24.932037] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650047, miss_cnt:32, cache_read_size:8050396875, hit_ratio:"9.999994892265020585e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:24.932166] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86F-0-0] [lt=27] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:24.932188] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA86F-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061844868473001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061844868473001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:24.933553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA86F-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061844933335000, start_id=1729061844933335000, end_id=1729061844933335001) [2024-10-16 14:57:24.936774] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:24.938581] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2217, mem_used=6239232) [2024-10-16 14:57:24.939654] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Cache wash clean map node details(ret=0, clean_node_count=46, clean_time=0, clean_start_pos=503312, clean_num=62914) [2024-10-16 14:57:24.941658] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA86F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061844940795000, start_id=1729061844940795000, end_id=1729061844940795001) [2024-10-16 14:57:24.942275] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:24.944082] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061844944081, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007614, add_bytes: 0 [2024-10-16 14:57:24.948131] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061844947138000, start_id=1729061844947138000, end_id=1729061844947138001) [2024-10-16 14:57:24.953815] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be0f2c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203067} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2196}, prev=this=0x2b0ec6d70140 trans_version={val:1729061780830200000, v:0} scn={val:1729061780830200000, v:0} tx_id={txid:392202036} prev=0x2b0ec6d70000 next=(nil) modify_count=1 acc_checksum=4287257022 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5474}, snapshot_version={val:1729061844952437000, v:0}, *this={this=0x2b0ec6d6fcf8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d70140 latest_compact_node=(nil) max_trans_version={val:1729061780830200000, v:0} max_trans_id=392202036 max_elr_trans_version={val:1729061780830200000, v:0} max_elr_trans_id=392202036 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780830200000, v:0} min_modify_scn={val:1729061780830200000, v:0}}) [2024-10-16 14:57:24.953849] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be0f2c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203067} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2196}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606422, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606422, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844952437000, v:0}, tx_id:{txid:392203067}, scn:{branch:0, seq:1063}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203067}, tx_desc:{this:0x2b0ec37adf10, tx_id:{txid:392203067}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844953570, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844951375, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2196}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.953906] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203067} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203067}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:2196}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.953920] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513901843599360},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203067} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.953933] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.953938] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901843599360},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.953999] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.954342] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2199}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203067} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.955552] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061844953570000, start_id=1729061844953570000, end_id=1729061844953570001) [2024-10-16 14:57:24.956354] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1729061844635402000, v:0}, new_max_version={val:1729061844635402000, v:0}, last_min_version={val:1729061844519406001, v:0}, last_max_version={val:1729061844519406001, v:0}, total_time_us=1724, delta_us=320951) [2024-10-16 14:57:24.956374] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1729061844635402000, v:0}, min_version={val:1729061844635402000, v:0}, max_version={val:1729061844635402000, v:0}, in_service=true, leader_epoch=33576) [2024-10-16 14:57:24.958457] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] worker thread began to shrink(tenant_->id()=1001, group_id_=19, token=0) [2024-10-16 14:57:24.961193] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:24.961249] WDIAG read_multi_row (ob_table_access_helper.h:238) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:24.962061] INFO delete_row (ob_table_access_helper.h:312) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=10] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1004, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061844961258) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061844961258) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:24.962119] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=14] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:24.962169] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=9] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:24.962217] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=8] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:24.962562] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B03-0-0] [lt=41] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:24.962866] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1555, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:24.962884] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=995, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:24.962928] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:24.963492] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO [ratelimit] time: 1729061844963491, bytes: 4446806364218, bw: 0.901855 MB/s, add_ts: 1000055, add_bytes: 945715 [2024-10-16 14:57:24.965395] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:24.965406] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:57:24.965416] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:24.966570] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21882][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=11][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:24.966628] WDIAG read_multi_row (ob_table_access_helper.h:238) [21882][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=16][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:24.967340] INFO delete_row (ob_table_access_helper.h:312) [21882][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1001, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061844966638) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061844966638) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:24.967606] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106743][T1001_L0_G19][T1001][Y0-0000000000000000-0-0] [lt=13] worker thread exit(tenant_->id()=1001, workers_.get_size()=1) [2024-10-16 14:57:24.967622] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106743][T1001_L0_G19][T0][Y0-0000000000000000-0-0] [lt=12] tenant thread end_run(id_=1001, ret=0, thread_count_=170) [2024-10-16 14:57:24.972421] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f4799f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203070} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2128}, prev=this=0x2b0e5abfef08 trans_version={val:1729061780835536000, v:0} scn={val:1729061780835536000, v:0} tx_id={txid:392202039} prev=0x2b0e5abfedc8 next=(nil) modify_count=1 acc_checksum=212569810 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2111}, snapshot_version={val:1729061844970620000, v:0}, *this={this=0x2b0e5abfeac0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e5abfef08 latest_compact_node=(nil) max_trans_version={val:1729061780835536000, v:0} max_trans_id=392202039 max_elr_trans_version={val:1729061780835536000, v:0} max_elr_trans_id=392202039 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780835536000, v:0} min_modify_scn={val:1729061780835536000, v:0}}) [2024-10-16 14:57:24.972457] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=35][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f4799f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203070} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2128}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606289, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606289, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061844970620000, v:0}, tx_id:{txid:392203070}, scn:{branch:0, seq:1072}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203070}, tx_desc:{this:0x2b0ec37c99d0, tx_id:{txid:392203070}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061844971676, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061844969549, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2128}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:24.972519] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203070} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203070}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:2128}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:24.972534] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513901919096832},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203070} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:24.972549] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:24.972554] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901919096832},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:24.972611] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:24.973441] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3185}, to_seq_no={branch:0, seq:1073}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203070} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:24.973699] INFO [SERVER.OMT] recv_group_request (ob_tenant.cpp:1419) [21190][pnio1][T0][YB4A0A000408-00061E9B44AE2011-0-0] [lt=2] worker thread created(id()=1001, group->group_id_=19) [2024-10-16 14:57:24.973700] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107002][][T1001][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1001, ret=0, thread_count_=171) [2024-10-16 14:57:24.973720] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107002][][T1001][Y0-0000000000000000-0-0] [lt=18] Init thread local success [2024-10-16 14:57:24.973742] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2011-0-0] [lt=4] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:24.973754] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2011-0-0] [lt=8] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:24.975624] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:24.975695] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=25] ====== tenant freeze timer task ====== [2024-10-16 14:57:24.980289] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E9B-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061844947538, keepalive_ts:-1}) [2024-10-16 14:57:24.980305] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E9B-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061844947538, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:24.981126] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E9C-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061844947580, keepalive_ts:-1}) [2024-10-16 14:57:24.981137] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106753][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E9C-0-0] [lt=11] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061844947580, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:24.983650] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061844983463000, start_id=1729061844983463000, end_id=1729061844983463001) [2024-10-16 14:57:24.994466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061844994045000, start_id=1729061844994045000, end_id=1729061844994045001) [2024-10-16 14:57:24.996839] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.007145] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=509, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:25.007168] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=334, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:25.007211] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:25.011077] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb32e90 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203072} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3269}, prev=this=0x2b0fadac1af0 trans_version={val:1729061780853403000, v:0} scn={val:1729061780853403000, v:0} tx_id={txid:392202041} prev=0x2b0fadac19b0 next=(nil) modify_count=1 acc_checksum=2597942770 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3133}, snapshot_version={val:1729061845007845000, v:0}, *this={this=0x2b0fadac16a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac1af0 latest_compact_node=(nil) max_trans_version={val:1729061780853403000, v:0} max_trans_id=392202041 max_elr_trans_version={val:1729061780853403000, v:0} max_elr_trans_id=392202041 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780853403000, v:0} min_modify_scn={val:1729061780853403000, v:0}}) [2024-10-16 14:57:25.011115] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb32e90 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203072} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3269}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606759, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606759, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845007845000, v:0}, tx_id:{txid:392203072}, scn:{branch:0, seq:2213}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203072}, tx_desc:{this:0x2b0ec37a2150, tx_id:{txid:392203072}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845009971, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845006703, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3269}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.011175] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203072} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203072}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:3269}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.011191] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":675513901956845568},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203072} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.011202] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.011208] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513901956845568},{"BIGINT":1729058400},{"VARCHAR":"@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.011257] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.012098] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4376}, to_seq_no={branch:0, seq:2214}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203072} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.015532] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:57:25.015635] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DA8-0-0] [lt=9] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061844994045000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061855590307000, latest_log_ts:{val:1729061838929180001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:25.015695] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DA8-0-0] [lt=57] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:25.018944] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061845018464000, start_id=1729061845018464000, end_id=1729061845018464001) [2024-10-16 14:57:25.018971] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061845018967], gts:1729061845018464000, latest_srr:[mts=1729061845018967]}) [2024-10-16 14:57:25.018983] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=543, get_gts_with_stc_cnt=79, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=76, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:25.031226] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:25.031265] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:25.031688] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:25.031715] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2310) [2024-10-16 14:57:25.032140] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=44, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:57:25.032154] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=14] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=1037, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:57:25.032187] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=33] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:57:25.032191] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=-434, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:57:25.032194] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=647, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:57:25.033266] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=126, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:25.033279] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=95, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:25.033362] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:25.033376] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=152413380, tenant_id=1) [2024-10-16 14:57:25.033393] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=11] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:25.036619] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:25.036635] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:25.036648] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=291861329, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:25.050678] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061845049681000, start_id=1729061845049681000, end_id=1729061845049681001) [2024-10-16 14:57:25.054416] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061845053871000, start_id=1729061845053871000, end_id=1729061845053871001) [2024-10-16 14:57:25.058478] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=106, replace_time=0, replace_start_pos=1698678, replace_num=31457) [2024-10-16 14:57:25.058494] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=106, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:25.062556] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B05-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:25.062576] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B05-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:25.062761] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061845061804000, start_id=1729061845061804000, end_id=1729061845061804001) [2024-10-16 14:57:25.075725] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.077788] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=31, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:25.077804] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=31, stat_interval=1000000, avg (count/cost)=10467, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:25.084826] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:25.085566] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=15] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:25.085596] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=29] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:25.085626] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=25] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:25.085751] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061857008855000, latest_log_ts:{val:1729061840488170001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:25.085815] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=60] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:25.085826] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:57:25.085841] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=13] get rec log scn(service_type_=0, rec_log_ts={val:1729061840488170001, v:0}) [2024-10-16 14:57:25.085848] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:25.085851] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:25.085862] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:25.085875] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:57:25.085886] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:25.085891] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:25.091210] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f544af8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203084} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1064}, prev=this=0x2b0f7f4bb448 trans_version={val:1729061780926289000, v:0} scn={val:1729061780926289000, v:0} tx_id={txid:392202053} prev=0x2b0f7f4bb308 next=(nil) modify_count=1 acc_checksum=3290599196 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4688}, snapshot_version={val:1729061845090557000, v:0}, *this={this=0x2b0f7f4bb000 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4bb448 latest_compact_node=(nil) max_trans_version={val:1729061780926289000, v:0} max_trans_id=392202053 max_elr_trans_version={val:1729061780926289000, v:0} max_elr_trans_id=392202053 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780926289000, v:0} min_modify_scn={val:1729061780926289000, v:0}}) [2024-10-16 14:57:25.091250] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f544af8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203084} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1064}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606269, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606269, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845090557000, v:0}, tx_id:{txid:392203084}, scn:{branch:0, seq:1061}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203084}, tx_desc:{this:0x2b0ec37bbc70, tx_id:{txid:392203084}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845090557, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845089497, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1064}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.091311] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203084} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203084}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1064}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.091329] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513902183337984},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203084} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.091344] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.091350] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513902183337984},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.091406] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.092070] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2101}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203084} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.093734] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get_max_applied_scn(scn={val:1729061844994045000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618350661}, palf_committed_end_scn_:{val:1729061845053871000, v:0}, last_check_scn_:{val:1729061845053871000, v:0}, max_applied_cb_scn_:{val:1729061844994045000, v:0}}) [2024-10-16 14:57:25.093758] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:25.096871] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.097242] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=33] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=6268) [2024-10-16 14:57:25.097287] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=6268) [2024-10-16 14:57:25.097906] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F75-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:25.097917] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F75-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=5226, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:25.098969] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061845097930000, start_id=1729061845097930000, end_id=1729061845097930001) [2024-10-16 14:57:25.099050] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=26, avg_log_batch_cnt=1, total_group_log_size=42217, avg_group_log_size=1623) [2024-10-16 14:57:25.099060] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=40716) [2024-10-16 14:57:25.102744] INFO [STORAGE] refresh_queuing_mode (ob_tenant_tablet_stat_mgr.cpp:1160) [21452][T1_TabletStatRp][T1][Y0-0000000000000000-0-0] [lt=10] refresh queuing mode(ret=0, tenant_id=1, stream_cnt=61, update_schema_cnt=61, cost_time=97) [2024-10-16 14:57:25.102758] INFO [STORAGE] runTimerTask (ob_tenant_tablet_stat_mgr.cpp:1210) [21452][T1_TabletStatRp][T1][Y0-0000000000000000-0-0] [lt=13] TenantTabletStatMgr refresh all tablet stream(MTL_ID()=1, interval_step=1, global_iter_pool={is_inited:false, is_washing:false, is_disabled:false, get_cnt:0, tenant_id:1, bucket_cnt:0, tenant_mem_user_limit:2147483648, tenant_mem_user_hold:262868992, cached_node_array:0x2b0ddc5c9e80}) [2024-10-16 14:57:25.103767] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb33570 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203085} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4239}, prev=this=0x2b0fadac21f0 trans_version={val:1729061780948225000, v:0} scn={val:1729061780948225000, v:0} tx_id={txid:392202054} prev=0x2b0fadac20b0 next=(nil) modify_count=1 acc_checksum=1452494293 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:17543}, snapshot_version={val:1729061845102197000, v:0}, *this={this=0x2b0fadac1da8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac21f0 latest_compact_node=(nil) max_trans_version={val:1729061780948225000, v:0} max_trans_id=392202054 max_elr_trans_version={val:1729061780948225000, v:0} max_elr_trans_id=392202054 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061780948225000, v:0} min_modify_scn={val:1729061780948225000, v:0}}) [2024-10-16 14:57:25.103799] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb33570 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203085} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4239}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606269, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606269, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845102197000, v:0}, tx_id:{txid:392203085}, scn:{branch:0, seq:3176}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203085}, tx_desc:{this:0x2b0ec37afeb0, tx_id:{txid:392203085}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845103260, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845099022, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4239}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.103859] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203085} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203085}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:4239}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.103874] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":675513902208503808},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203085} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.103889] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.103894] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513902208503808},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.103945] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.104539] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5299}, to_seq_no={branch:0, seq:3177}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203085} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.110322] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:25.110344] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:25.115718] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061845114909000, start_id=1729061845114909000, end_id=1729061845114909001) [2024-10-16 14:57:25.123705] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:25.125095] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=12] current monitor number(seq_=-1) [2024-10-16 14:57:25.128058] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=618, mem_used=2079744) [2024-10-16 14:57:25.129237] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BB3-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061845128673000, start_id=1729061845128673000, end_id=1729061845128673001) [2024-10-16 14:57:25.129424] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BB4-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061845128673000, start_id=1729061845128673000, end_id=1729061845128673001) [2024-10-16 14:57:25.129516] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BB1-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061845128673000, start_id=1729061845128673000, end_id=1729061845128673001) [2024-10-16 14:57:25.129553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BB0-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061845128673000, start_id=1729061845128673000, end_id=1729061845128673001) [2024-10-16 14:57:25.131305] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=2310, avg_group_log_size=192) [2024-10-16 14:57:25.131326] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2310) [2024-10-16 14:57:25.131533] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:25.135478] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=45] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:57:25.135493] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=14] try gc retain ctx [2024-10-16 14:57:25.135497] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:25.136102] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061855590307000, latest_log_ts:{val:1729061838929180001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:25.139958] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:25.142401] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CA1-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510313010795, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410518, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:25.154834] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061844953570000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=201258, in_cluster_service=true, cluster_version={val:1729061844635402000, v:0}, min_cluster_version={val:1729061844635402000, v:0}, max_cluster_version={val:1729061844635402000, v:0}, get_cluster_version_err=0, cluster_version_delta=519426, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244045, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=454224631, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:25.155876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061845155228000, start_id=1729061845155228000, end_id=1729061845155228001) [2024-10-16 14:57:25.155957] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061845155228000, start_id=1729061845155228000, end_id=1729061845155228001) [2024-10-16 14:57:25.155985] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=11426, avg_group_log_size=571) [2024-10-16 14:57:25.155996] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11426) [2024-10-16 14:57:25.156238] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=466, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:25.156256] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11426) [2024-10-16 14:57:25.156270] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=710, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:25.156284] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11426) [2024-10-16 14:57:25.156297] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:25.156610] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B32-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:25.156622] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B32-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=2526, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:25.156628] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B32-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=4, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=1698, avg_log_submit_to_slide_time=2491) [2024-10-16 14:57:25.159791] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eed048 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203091} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2099}, prev=this=0x2b0f22ea23d8 trans_version={val:1729061781002031000, v:0} scn={val:1729061781002031000, v:0} tx_id={txid:392202060} prev=0x2b0f22ea2298 next=(nil) modify_count=1 acc_checksum=2324586514 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:23016}, snapshot_version={val:1729061845158385000, v:0}, *this={this=0x2b0f22ea1f90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea23d8 latest_compact_node=(nil) max_trans_version={val:1729061781002031000, v:0} max_trans_id=392202060 max_elr_trans_version={val:1729061781002031000, v:0} max_elr_trans_id=392202060 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781002031000, v:0} min_modify_scn={val:1729061781002031000, v:0}}) [2024-10-16 14:57:25.159828] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=36][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eed048 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203091} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2099}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606287, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606287, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845158385000, v:0}, tx_id:{txid:392203091}, scn:{branch:0, seq:1033}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203091}, tx_desc:{this:0x2b0ec37a9fd0, tx_id:{txid:392203091}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845159451, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845157353, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2099}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.159896] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=67][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203091} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203091}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:2099}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.159914] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654910318077136896},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203091} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.159933] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.159943] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318077136896},{"BIGINT":1729058400},{"VARCHAR":"AÈ ëî", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.159998] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.161893] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4174}, to_seq_no={branch:0, seq:1034}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203091} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.171944] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501B-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:25.171963] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501B-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:25.173803] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C40E8-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=3492, avg_log_submit_to_slide_time=4093) [2024-10-16 14:57:25.175844] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.184935] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=25][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f5454e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203095} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2098}, prev=this=0x2b0f7f4bbf20 trans_version={val:1729061781039414000, v:0} scn={val:1729061781039414000, v:0} tx_id={txid:392202064} prev=0x2b0f7f4bbde0 next=(nil) modify_count=1 acc_checksum=3269986175 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:17882}, snapshot_version={val:1729061845182849000, v:0}, *this={this=0x2b0f7f4bbad8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4bbf20 latest_compact_node=(nil) max_trans_version={val:1729061781039414000, v:0} max_trans_id=392202064 max_elr_trans_version={val:1729061781039414000, v:0} max_elr_trans_id=392202064 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781039414000, v:0} min_modify_scn={val:1729061781039414000, v:0}}) [2024-10-16 14:57:25.184990] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f5454e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203095} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2098}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606300, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606300, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845182849000, v:0}, tx_id:{txid:392203095}, scn:{branch:0, seq:1038}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203095}, tx_desc:{this:0x2b0ec37d37f0, tx_id:{txid:392203095}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845183910, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845181813, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2098}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.185075] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=84][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203095} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203095}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:2098}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.185100] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=0] primary key duplication conflict(key={"BIGINT":654910318127468544},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203095} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.185124] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.185138] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318127468544},{"BIGINT":1729058400},{"VARCHAR":"@Ø", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.185216] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=36][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.185811] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=10] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3159}, to_seq_no={branch:0, seq:1039}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203095} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.188524] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31906][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB20CC6D9-0-0] [lt=22] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:25.194776] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=44] get number(ret=0, service_type_=0, range=1, base_id=1729061845194483000, start_id=1729061845194483000, end_id=1729061845194483001) [2024-10-16 14:57:25.196291] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb33c50 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203096} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5277}, prev=this=0x2b0fadac28f0 trans_version={val:1729061781066782000, v:0} scn={val:1729061781066782000, v:0} tx_id={txid:392202065} prev=0x2b0fadac27b0 next=(nil) modify_count=1 acc_checksum=3200729929 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:18231}, snapshot_version={val:1729061845194483000, v:0}, *this={this=0x2b0fadac24a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadac28f0 latest_compact_node=(nil) max_trans_version={val:1729061781066782000, v:0} max_trans_id=392202065 max_elr_trans_version={val:1729061781066782000, v:0} max_elr_trans_id=392202065 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781066782000, v:0} min_modify_scn={val:1729061781066782000, v:0}}) [2024-10-16 14:57:25.196328] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=35][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb33c50 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203096} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5277}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606279, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606279, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845194483000, v:0}, tx_id:{txid:392203096}, scn:{branch:0, seq:4217}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203096}, tx_desc:{this:0x2b0ec37b3df0, tx_id:{txid:392203096}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845195547, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845190271, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:5277}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.196392] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=64][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203096} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203096}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:5277}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.196409] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":675513902334332928},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203096} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.196424] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.196433] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513902334332928},{"BIGINT":1729058400},{"VARCHAR":"Á«¾@’", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.196495] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=34][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.196730] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6342}, to_seq_no={branch:0, seq:4218}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203096} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.196932] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.200064] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061845199750000, start_id=1729061845199750000, end_id=1729061845199750001) [2024-10-16 14:57:25.205754] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f47a0d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203098} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2248}, prev=this=0x2b0f3f404108 trans_version={val:1729061781088916000, v:0} scn={val:1729061781088916000, v:0} tx_id={txid:392202067} prev=0x2b0e5abffbc8 next=(nil) modify_count=1 acc_checksum=1241670020 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:14070}, snapshot_version={val:1729061845204145000, v:0}, *this={this=0x2b0e5abff508 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f404108 latest_compact_node=(nil) max_trans_version={val:1729061781088916000, v:0} max_trans_id=392202067 max_elr_trans_version={val:1729061781088916000, v:0} max_elr_trans_id=392202067 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781088916000, v:0} min_modify_scn={val:1729061781088916000, v:0}}) [2024-10-16 14:57:25.205806] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=52][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f47a0d8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203098} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2248}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606816, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606816, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845204145000, v:0}, tx_id:{txid:392203098}, scn:{branch:0, seq:1187}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203098}, tx_desc:{this:0x2b0ec37c1b50, tx_id:{txid:392203098}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845205206, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845202959, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2248}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.205873] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=65][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203098} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203098}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:2248}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.205906] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=0] primary key duplication conflict(key={"BIGINT":675513902359498752},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203098} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.205923] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.205931] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513902359498752},{"BIGINT":1729058400},{"VARCHAR":"@Àl", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.206003] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.206487] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3299}, to_seq_no={branch:0, seq:1188}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203098} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.212472] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5874-0-0] [lt=14] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:25.212571] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5874-0-0] [lt=17][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:25.217932] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=61817) [2024-10-16 14:57:25.218422] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:25.231414] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:25.231450] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=35] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:57:25.232034] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD028-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:25.232059] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD028-0-0] [lt=24] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1975, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:25.232064] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD028-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=1502, avg_log_submit_to_slide_time=1955) [2024-10-16 14:57:25.232260] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eed728 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203101} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4215}, prev=this=0x2b0f22ea2ad8 trans_version={val:1729061781116172000, v:0} scn={val:1729061781116172000, v:0} tx_id={txid:392202070} prev=0x2b0f22ea2998 next=(nil) modify_count=1 acc_checksum=4138543601 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:12985}, snapshot_version={val:1729061845230681000, v:0}, *this={this=0x2b0f22ea2690 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea2ad8 latest_compact_node=(nil) max_trans_version={val:1729061781116172000, v:0} max_trans_id=392202070 max_elr_trans_version={val:1729061781116172000, v:0} max_elr_trans_id=392202070 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781116172000, v:0} min_modify_scn={val:1729061781116172000, v:0}}) [2024-10-16 14:57:25.232303] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=42][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eed728 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203101} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4215}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606327, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606327, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845230681000, v:0}, tx_id:{txid:392203101}, scn:{branch:0, seq:3153}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203101}, tx_desc:{this:0x2b0ec37c7a30, tx_id:{txid:392203101}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845231741, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845227529, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4215}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.232368] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=64][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203101} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203101}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:4215}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.232383] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654910318202966016},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203101} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.232395] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.232407] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318202966016},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.232491] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.232807] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5251}, to_seq_no={branch:0, seq:3154}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203101} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.240009] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=7][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be102b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203102} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3149}, prev=this=0x2b0ec6d72a28 trans_version={val:1729061781126138000, v:0} scn={val:1729061781126138000, v:0} tx_id={txid:392202071} prev=0x2b0ec6d728e8 next=(nil) modify_count=1 acc_checksum=3577703856 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4450}, snapshot_version={val:1729061845238006000, v:0}, *this={this=0x2b0ec6d725e0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d72a28 latest_compact_node=(nil) max_trans_version={val:1729061781126138000, v:0} max_trans_id=392202071 max_elr_trans_version={val:1729061781126138000, v:0} max_elr_trans_id=392202071 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781126138000, v:0} min_modify_scn={val:1729061781126138000, v:0}}) [2024-10-16 14:57:25.240041] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be102b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203102} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3149}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606270, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606270, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845238006000, v:0}, tx_id:{txid:392203102}, scn:{branch:0, seq:2083}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203102}, tx_desc:{this:0x2b0ec37b9cd0, tx_id:{txid:392203102}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845239072, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845235924, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3149}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.240095] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203102} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203102}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:3149}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.240109] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":675513902409830400},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203102} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.240120] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.240130] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513902409830400},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.240172] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.241310] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4346}, to_seq_no={branch:0, seq:2084}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203102} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.251764] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD05-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:25.254556] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061845253904000, start_id=1729061845253904000, end_id=1729061845253904001) [2024-10-16 14:57:25.256577] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD102E-0-0] [lt=21] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:25.259986] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=102, replace_time=0, replace_start_pos=1730135, replace_num=31457) [2024-10-16 14:57:25.260004] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=102, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:25.263008] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061845262291000, start_id=1729061845262291000, end_id=1729061845262291001) [2024-10-16 14:57:25.265685] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C40F1-0-0] [lt=29] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=31, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:25.265703] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C40F1-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=31, stat_interval=1000000, avg (count/cost)=1113, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:25.267189] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061845266615000, start_id=1729061845266615000, end_id=1729061845266615001) [2024-10-16 14:57:25.271273] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=42457) [2024-10-16 14:57:25.271486] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=43959) [2024-10-16 14:57:25.275971] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.277133] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB93-0-0] [lt=16] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12287, add_timestamp:1729061845277122}) [2024-10-16 14:57:25.277147] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E16-0-0] [lt=10] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12287, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:25.277318] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E16-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12287, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061845277306, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061845277306, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061845277306, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:25.277338] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E16-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:12287, add_timestamp:1729061845277122}, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061845277306, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061845277306, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061845277306, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:25.281867] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB93-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:25.282857] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=13] start to print blacklist info [2024-10-16 14:57:25.282990] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10] ls blacklist refresh finish(cost_time=6545) [2024-10-16 14:57:25.292805] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22eede08 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203106} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3176}, prev=this=0x2b0f22ea31d8 trans_version={val:1729061781152014000, v:0} scn={val:1729061781152014000, v:0} tx_id={txid:392202075} prev=0x2b0f22ea3098 next=(nil) modify_count=1 acc_checksum=2130813449 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:12274}, snapshot_version={val:1729061845290712000, v:0}, *this={this=0x2b0f22ea2d90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ea31d8 latest_compact_node=(nil) max_trans_version={val:1729061781152014000, v:0} max_trans_id=392202075 max_elr_trans_version={val:1729061781152014000, v:0} max_elr_trans_id=392202075 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781152014000, v:0} min_modify_scn={val:1729061781152014000, v:0}}) [2024-10-16 14:57:25.292841] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=36][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22eede08 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203106} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3176}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606325, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606325, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845290712000, v:0}, tx_id:{txid:392203106}, scn:{branch:0, seq:2116}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203106}, tx_desc:{this:0x2b0ec37d1850, tx_id:{txid:392203106}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845291772, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845288597, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3176}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.292913] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=71][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203106} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203106}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:3176}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.292936] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=22][errcode=0] primary key duplication conflict(key={"BIGINT":675513902447579136},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203106} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.292953] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.292959] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":675513902447579136},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.293015] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.293817] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061845053871000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=239942, in_cluster_service=true, cluster_version={val:1729061844994045000, v:0}, min_cluster_version={val:1729061844994045000, v:0}, max_cluster_version={val:1729061844994045000, v:0}, get_cluster_version_err=0, cluster_version_delta=299768, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244296, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=219689984, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:25.295954] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6347}, to_seq_no={branch:0, seq:2117}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203106} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.297004] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.307542] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061845307147000, start_id=1729061845307147000, end_id=1729061845307147001) [2024-10-16 14:57:25.313195] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5775, mem_used=12478464) [2024-10-16 14:57:25.316095] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061845315614000, start_id=1729061845315614000, end_id=1729061845315614001) [2024-10-16 14:57:25.316397] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:25.316675] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F78-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=2531, avg_log_submit_to_slide_time=2802) [2024-10-16 14:57:25.321283] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=11] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:25.326580] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be10ca0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203109} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2088}, prev=this=0x2b0ec6d73c00 trans_version={val:1729061781177923000, v:0} scn={val:1729061781177923000, v:0} tx_id={txid:392202078} prev=0x2b0ec6d73ac0 next=(nil) modify_count=1 acc_checksum=99887162 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:11455}, snapshot_version={val:1729061845326178000, v:0}, *this={this=0x2b0ec6d737b8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d73c00 latest_compact_node=(nil) max_trans_version={val:1729061781177923000, v:0} max_trans_id=392202078 max_elr_trans_version={val:1729061781177923000, v:0} max_elr_trans_id=392202078 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781177923000, v:0} min_modify_scn={val:1729061781177923000, v:0}}) [2024-10-16 14:57:25.326623] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=42][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be10ca0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203109} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2088}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606261, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606261, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845326178000, v:0}, tx_id:{txid:392203109}, scn:{branch:0, seq:1028}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203109}, tx_desc:{this:0x2b0ec37b1e50, tx_id:{txid:392203109}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845326178, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845324091, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2088}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.326686] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203109} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203109}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:2088}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.326701] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654910318291046400},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203109} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.326714] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.326720] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318291046400},{"BIGINT":1729058400},{"VARCHAR":"Ap", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.326772] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.327056] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2091}, to_seq_no={branch:0, seq:1029}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203109} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.331707] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=323, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:25.331728] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=119, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:25.331755] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:25.331881] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD02B-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:25.331888] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD02B-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=913, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:25.340369] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1200586752, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:25.346694] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA5-0-0] [lt=17] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:25.346720] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA5-0-0] [lt=20] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=27) [2024-10-16 14:57:25.356010] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061845355922000, start_id=1729061845355922000, end_id=1729061845355922001) [2024-10-16 14:57:25.357039] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB APPEND COST TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=44, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:57:25.357051] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=2334, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:57:25.357055] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:57:25.357059] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=-554, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:57:25.357062] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=1825, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:57:25.362594] INFO [STORAGE.COMPACTION] schedule_all_tablets_minor (ob_tenant_tablet_scheduler.cpp:553) [21907][T1001_MergeLoop][T1001][YB4A0A000407-00061E997FFA3684-0-0] [lt=14] start schedule all tablet minor merge(minor_ls_tablet_iter_={ls_idx:-1, ls_ids:[{id:1}], tablet_idx:0, tablet_ids_.count():0, schedule_tablet_cnt:0, max_batch_tablet_cnt:50000}) [2024-10-16 14:57:25.368096] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107003][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=192) [2024-10-16 14:57:25.368116] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107003][][T1004][Y0-0000000000000000-0-0] [lt=17] Init thread local success [2024-10-16 14:57:25.368139] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B0B-0-0] [lt=5] init point thread id with(&point=0x55e3435c2d40, idx_=507, point=[thread id=107003, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107003) [2024-10-16 14:57:25.368242] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] worker thread created(tenant_->id()=1004, group_id_=1, token=1) [2024-10-16 14:57:25.370731] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061845369767000, start_id=1729061845369767000, end_id=1729061845369767001) [2024-10-16 14:57:25.373276] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:25.373299] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:25.373468] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=1514, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:25.373475] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1280) [2024-10-16 14:57:25.373486] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=45, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:25.373499] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1706) [2024-10-16 14:57:25.373506] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:25.373793] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C40F9-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:25.373798] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C40F9-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=7, stat_interval=1000000, avg (count/cost)=1907, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:25.375099] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B3-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:25.375120] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B3-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061845129860000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061845129860000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:25.376091] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.376477] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:85) [21907][T1001_MergeLoop][T1001][YB4A0A000407-00061E997FFA3684-0-0] [lt=22] MergeLoopTask(cost_ts=13907) [2024-10-16 14:57:25.376874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9B3-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061845376158000, start_id=1729061845376158000, end_id=1729061845376158001) [2024-10-16 14:57:25.377851] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9A279FA9B3-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.054347991943359375e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=302, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.074405359793368461e+03) [2024-10-16 14:57:25.380154] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9B3-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061845379317000, start_id=1729061845379317000, end_id=1729061845379317001) [2024-10-16 14:57:25.380576] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:25.380593] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=211) [2024-10-16 14:57:25.380818] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:25.386514] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f545bc0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203117} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2101}, prev=this=0x2b0f7f4bc620 trans_version={val:1729061781201171001, v:0} scn={val:1729061781201171001, v:0} tx_id={txid:392202086} prev=0x2b0f7f4bc4e0 next=(nil) modify_count=1 acc_checksum=450114248 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4209}, snapshot_version={val:1729061845385595000, v:0}, *this={this=0x2b0f7f4bc1d8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4bc620 latest_compact_node=(nil) max_trans_version={val:1729061781201171001, v:0} max_trans_id=392202086 max_elr_trans_version={val:1729061781201171001, v:0} max_elr_trans_id=392202086 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781201171001, v:0} min_modify_scn={val:1729061781201171001, v:0}}) [2024-10-16 14:57:25.386553] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f545bc0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203117} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2101}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606302, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606302, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845385595000, v:0}, tx_id:{txid:392203117}, scn:{branch:0, seq:2098}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203117}, tx_desc:{this:0x2b0ec37cb970, tx_id:{txid:392203117}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845385595, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845383498, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2101}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.386620] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=67][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203117} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203117}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:2101}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.386638] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654910318379126784},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203117} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.386653] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.386659] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318379126784},{"BIGINT":1729058400},{"VARCHAR":"AÐ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.386710] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.387540] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:25.387757] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4173}, to_seq_no={branch:0, seq:2099}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203117} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.387793] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15] rpc statistics(total_trans_msg_count=64, total_batch_msg_count=0) [2024-10-16 14:57:25.394732] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061845393998000, start_id=1729061845393998000, end_id=1729061845393998001) [2024-10-16 14:57:25.397058] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.415232] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get_max_applied_scn(scn={val:1729061845315614000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390059227}, palf_committed_end_scn_:{val:1729061845315614000, v:0}, last_check_scn_:{val:1729061845315614000, v:0}, max_applied_cb_scn_:{val:1729061845315614000, v:0}}) [2024-10-16 14:57:25.415266] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=29] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:25.424259] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C8C-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:25.424291] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C8C-0-0] [lt=25] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:25.424300] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C8C-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:25.426646] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=5][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be11990 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203121} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4239}, prev=this=0x2b0ec6d76060 trans_version={val:1729061781216803001, v:0} scn={val:1729061781216803001, v:0} tx_id={txid:392202090} prev=0x2b0ec6d75f20 next=(nil) modify_count=1 acc_checksum=276553281 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2100}, snapshot_version={val:1729061845423639000, v:0}, *this={this=0x2b0ec6d75c18 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6d76060 latest_compact_node=(nil) max_trans_version={val:1729061781216803001, v:0} max_trans_id=392202090 max_elr_trans_version={val:1729061781216803001, v:0} max_elr_trans_id=392202090 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061781216803001, v:0} min_modify_scn={val:1729061781216803001, v:0}}) [2024-10-16 14:57:25.426687] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=40][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be11990 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392203121} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4239}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062437606292, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062437606292, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061845423639000, v:0}, tx_id:{txid:392203121}, scn:{branch:0, seq:3177}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392203121}, tx_desc:{this:0x2b0ec37b5d90, tx_id:{txid:392203121}, state:1, addr:"10.0.4.9:2890", tenant_id:1002, session_id:3222259816, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061845425758, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061845421520, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:4239}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:25.426759] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=71][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203121} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392203121}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:4239}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:57:25.426775] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654910318416875520},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392203121} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:25.426787] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:25.426793] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910318416875520},{"BIGINT":1729058400},{"VARCHAR":"@Y", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:25.426849] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:25.428712] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6526}, to_seq_no={branch:0, seq:3178}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392203121} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:25.428827] INFO [SHARE] get_server_tenant_configs (ob_unit_getter.cpp:231) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=4] get_server_tenant_configs(unit_infos=[{unit:unit_id:1, resource_pool_id:1, unit_group_id:1, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, pool:{resource_pool_id:1, name:"sys_pool", unit_count:1, unit_config_id:1, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1, replica_type:0}}, {unit:unit_id:1001, resource_pool_id:1001, unit_group_id:1001, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1001, name:"ocp_pool", unit_count:1, unit_config_id:1001, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1002, replica_type:0}}, {unit:unit_id:1005, resource_pool_id:1003, unit_group_id:1002, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1003, name:"pool_bddb_myoceanbase1_oav", unit_count:1, unit_config_id:1003, zone_list:["myoceanbase1"], tenant_id:1004, replica_type:0}}]) [2024-10-16 14:57:25.428869] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=39] jx_debug: get tenant compatibility mode(tenant_id=1002, compat_mode=0) [2024-10-16 14:57:25.428878] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=7] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:25.428893] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=14] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:25.428909] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=15] jx_debug: get tenant compatibility mode(tenant_id=1004, compat_mode=0) [2024-10-16 14:57:25.428911] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=3] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:25.428937] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=25] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:25.430405] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:191) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] refresh resource plan success(tenant_id=1, plan_name=, directives=[]) [2024-10-16 14:57:25.431091] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=18] refresh resource user mapping rule(tenant_id=1, plan=, user_rules=[]) [2024-10-16 14:57:25.431820] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=16] refresh_resource_function_mapping_rule(tenant_id=1, plan=, rules=[]) [2024-10-16 14:57:25.431831] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=11] refresh resource mapping rule success(tenant_id=1, plan=) [2024-10-16 14:57:25.432006] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:25.432698] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=35] trace tenant dfc parameters(max_parallel_cnt_=30, max_blocked_buffer_size_=9830400, max_buffer_size_=19660800) [2024-10-16 14:57:25.432778] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=13] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:25.432789] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] tenant dfc status(ret=0, get_tenant_id()=1, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=30, get_max_blocked_buffer_size()=9830400, get_max_buffer_size()=19660800, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:25.434660] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061845434460000, start_id=1729061845434460000, end_id=1729061845434460001) [2024-10-16 14:57:25.435047] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=10] refresh resource user mapping rule(tenant_id=1001, plan=, user_rules=[]) [2024-10-16 14:57:25.435775] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=13] refresh_resource_function_mapping_rule(tenant_id=1001, plan=, rules=[]) [2024-10-16 14:57:25.435790] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=14] refresh resource mapping rule success(tenant_id=1001, plan=) [2024-10-16 14:57:25.436427] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:25.436491] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=13] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:25.436501] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=10] tenant dfc status(ret=0, get_tenant_id()=1001, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:25.438422] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=10] refresh resource user mapping rule(tenant_id=1002, plan=, user_rules=[]) [2024-10-16 14:57:25.439095] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=16] refresh_resource_function_mapping_rule(tenant_id=1002, plan=, rules=[]) [2024-10-16 14:57:25.439111] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=15] refresh resource mapping rule success(tenant_id=1002, plan=) [2024-10-16 14:57:25.439744] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:25.439808] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=13] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:25.439817] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] tenant dfc status(ret=0, get_tenant_id()=1002, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=4, get_max_size_per_channel()=655360) [2024-10-16 14:57:25.442781] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] refresh resource user mapping rule(tenant_id=1003, plan=, user_rules=[]) [2024-10-16 14:57:25.443590] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107004][][T1002][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1002, ret=0, thread_count_=191) [2024-10-16 14:57:25.443619] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107004][][T1002][Y0-0000000000000000-0-0] [lt=27] Init thread local success [2024-10-16 14:57:25.443635] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=7] worker thread created(tenant_->id()=1002, group_id_=1, token=1) [2024-10-16 14:57:25.443742] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=13] refresh_resource_function_mapping_rule(tenant_id=1003, plan=, rules=[]) [2024-10-16 14:57:25.443753] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=10] refresh resource mapping rule success(tenant_id=1003, plan=) [2024-10-16 14:57:25.444431] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:25.444494] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=14] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:25.444504] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] tenant dfc status(ret=0, get_tenant_id()=1003, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:25.446484] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=6] refresh resource user mapping rule(tenant_id=1004, plan=, user_rules=[]) [2024-10-16 14:57:25.447161] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=15] refresh_resource_function_mapping_rule(tenant_id=1004, plan=, rules=[]) [2024-10-16 14:57:25.447171] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=10] refresh resource mapping rule success(tenant_id=1004, plan=) [2024-10-16 14:57:25.447957] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:25.448024] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:25.448032] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=7] tenant dfc status(ret=0, get_tenant_id()=1004, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:25.448055] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:108) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=15] refresh tenant units(sys_unit_cnt=3, units=[{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, {tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, {tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}], ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:25.448096] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=38] finish set nway(tenant_id_=1, nway=3) [2024-10-16 14:57:25.448104] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=7] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1, nway=3, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}) [2024-10-16 14:57:25.448119] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=1.638399963378906250e+02, this=0x2b0ddc2dc170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=819, Throttle Trigger(MB)=491, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.948457122979588263e+01) [2024-10-16 14:57:25.448134] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=14] succ to set_memstore_threshold of memstore allocator(tenant_id=1, ret=0) [2024-10-16 14:57:25.448139] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=3] finish set nway(tenant_id_=1002, nway=1) [2024-10-16 14:57:25.448145] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=6] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1002, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:25.448155] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e0c298170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:25.448164] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] succ to set_memstore_threshold of memstore allocator(tenant_id=1002, ret=0) [2024-10-16 14:57:25.448169] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=3] finish set nway(tenant_id_=1001, nway=1) [2024-10-16 14:57:25.448175] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=5] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1001, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:25.448184] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0df7098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:25.448193] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] succ to set_memstore_threshold of memstore allocator(tenant_id=1001, ret=0) [2024-10-16 14:57:25.448197] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=2] finish set nway(tenant_id_=1004, nway=1) [2024-10-16 14:57:25.448200] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1004, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:25.448209] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e40098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:25.448218] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=8] succ to set_memstore_threshold of memstore allocator(tenant_id=1004, ret=0) [2024-10-16 14:57:25.448222] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=2] finish set nway(tenant_id_=1003, nway=1) [2024-10-16 14:57:25.448224] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1003, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:25.448231] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=6] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e2c698170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:25.448237] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=6] succ to set_memstore_threshold of memstore allocator(tenant_id=1003, ret=0) [2024-10-16 14:57:25.448829] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=5] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:25.448845] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=16] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1001, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:25.448850] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=4] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:25.448854] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1003, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:25.448857] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:25.448880] INFO [SERVER] cal_all_part_disk_default_percentage (ob_server_utils.cpp:301) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=6] cal_all_part_disk_default_percentage succ(data_dir="/mnt/myoceanbase/oceanbase/data//sstable", clog_dir="/mnt/myoceanbase/oceanbase/data//clog", shared_mode=true, data_disk_total_size=214429704192, data_disk_default_percentage=60, clog_disk_total_size=214429704192, clog_disk_default_percentage=30) [2024-10-16 14:57:25.448890] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:337) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=9] decide disk size finished(suggested_disk_size=32212254720, suggested_disk_percentage=0, default_disk_percentage=30, total_space=214429704192, disk_size=32212254720) [2024-10-16 14:57:25.448894] INFO [SERVER] get_log_disk_info_in_config (ob_server_utils.cpp:88) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=4] get_log_disk_info_in_config(suggested_data_disk_size=32212254720, suggested_clog_disk_size=32212254720, suggested_data_disk_percentage=0, suggested_clog_disk_percentage=0, log_disk_size=32212254720, log_disk_percentage=0, total_log_disk_size=214429704192) [2024-10-16 14:57:25.448903] INFO [CLOG] try_resize (ob_server_log_block_mgr.cpp:800) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=7] try_resize success(ret=0, log_disk_size=32212254720, total_log_disk_size=214429704192, this={dir::"/mnt/myoceanbase/oceanbase/data//clog/log_pool", dir_fd:15, meta_fd:16, log_pool_meta:{curr_total_size:32212254720, next_total_size:32212254720, status:0}, min_block_id:15037, max_block_id:15335, min_log_disk_size_for_all_tenants_:15032385536, is_inited:true}) [2024-10-16 14:57:25.448914] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:132) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0A-0-0] [lt=11] refresh tenant config(tenants=[1, 1001, 1002, 1003, 1004], ret=0) [2024-10-16 14:57:25.452698] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=31, avg_log_batch_cnt=0, total_group_log_size=48829, avg_group_log_size=1575) [2024-10-16 14:57:25.453801] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6F-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:25.453835] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A6F-0-0] [lt=32] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:25.454666] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061845454561000, start_id=1729061845454561000, end_id=1729061845454561001) [2024-10-16 14:57:25.455625] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:7668488755}, base_lsn={lsn:7582838784}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:25.455683] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=57] get_replay_process success(id={id:1}, replayed_log_size=85649971, unreplayed_log_size=0) [2024-10-16 14:57:25.455690] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:6198988121}, base_lsn={lsn:6106533888}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:25.455718] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=28] get_replay_process success(id={id:1001}, replayed_log_size=92454233, unreplayed_log_size=0) [2024-10-16 14:57:25.455724] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=8163515309, unreplayed_log_size=0) [2024-10-16 14:57:25.455730] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1003}, replayed_log_size=8111611324, unreplayed_log_size=0) [2024-10-16 14:57:25.455745] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1004, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=15691, last_replayed_log_size(MB)=15691, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:25.458940] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C40-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:25.458963] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C40-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061845452454000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061845452454000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:25.460058] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C40-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061845459772000, start_id=1729061845459772000, end_id=1729061845459772001) [2024-10-16 14:57:25.461515] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=93, replace_time=0, replace_start_pos=1761592, replace_num=31457) [2024-10-16 14:57:25.461530] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=93, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:25.462823] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061845018464002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=444353, in_cluster_service=true, cluster_version={val:1729061844518343005, v:0}, min_cluster_version={val:1729061844518343005, v:0}, max_cluster_version={val:1729061844518343005, v:0}, get_cluster_version_err=0, cluster_version_delta=944474, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228076, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=390854338, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:25.463649] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061845461926000, start_id=1729061845461926000, end_id=1729061845461926001) [2024-10-16 14:57:25.464939] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C40-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061845464821000, start_id=1729061845464821000, end_id=1729061845464821001) [2024-10-16 14:57:25.470851] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3332F18F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061845470136000, start_id=1729061845470136000, end_id=1729061845470136001) [2024-10-16 14:57:25.471974] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D1-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061845463183, keepalive_ts:-1}) [2024-10-16 14:57:25.471989] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D1-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061845463183, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:25.476217] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.483802] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:25.483812] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=11] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:25.483823] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:25.483828] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:25.483832] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:25.483835] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:25.483837] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:25.483840] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:25.483842] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:25.483844] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:25.483848] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:25.483866] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECC-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:25.483879] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECC-0-0] [lt=5] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:25.489188] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:25.489202] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:25.489215] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70514921, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:25.497120] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.505829] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] worker thread began to shrink(tenant_->id()=1004, group_id_=19, token=0) [2024-10-16 14:57:25.508711] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=9] replay status is not follower(min_unreplayed_lsn={lsn:45686897415}, base_lsn={lsn:45497032704}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:25.508766] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=54] get_replay_process success(id={id:1}, replayed_log_size=189864711, unreplayed_log_size=0) [2024-10-16 14:57:25.508785] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1001, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=181, last_replayed_log_size(MB)=181, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:25.510125] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106746][T1004_L0_G19][T1004][Y0-0000000000000000-0-0] [lt=8] worker thread exit(tenant_->id()=1004, workers_.get_size()=1) [2024-10-16 14:57:25.510143] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106746][T1004_L0_G19][T0][Y0-0000000000000000-0-0] [lt=14] tenant thread end_run(id_=1004, ret=0, thread_count_=191) [2024-10-16 14:57:25.516257] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=45] get number(ret=0, service_type_=0, range=1, base_id=1729061845514558000, start_id=1729061845514558000, end_id=1729061845514558001) [2024-10-16 14:57:25.516356] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=4515, avg_group_log_size=410) [2024-10-16 14:57:25.516610] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=236, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:25.516631] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=42, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:25.519319] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061845518506000, start_id=1729061845518506000, end_id=1729061845518506001) [2024-10-16 14:57:25.519342] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:25.519508] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061845519502], gts:1729061845518506000, latest_srr:[mts=1729061845519502]}) [2024-10-16 14:57:25.525290] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:25.525306] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:25.532046] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:25.540697] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:25.553950] INFO [SERVER.OMT] recv_group_request (ob_tenant.cpp:1419) [21189][pnio1][T0][YB4A0A000409-00061E9A3404DC3A-0-0] [lt=12] worker thread created(id()=1004, group->group_id_=19) [2024-10-16 14:57:25.553951] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107006][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=192) [2024-10-16 14:57:25.553972] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107006][][T1004][Y0-0000000000000000-0-0] [lt=19] Init thread local success [2024-10-16 14:57:25.554005] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC3A-0-0] [lt=5] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:25.554021] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC3A-0-0] [lt=10] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:25.554015] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061845166806000, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=387203, in_cluster_service=true, cluster_version={val:1729061844956765000, v:0}, min_cluster_version={val:1729061844956765000, v:0}, max_cluster_version={val:1729061844956765000, v:0}, get_cluster_version_err=0, cluster_version_delta=597244, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65243995, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=516883848, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:25.554037] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC3A-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:25.555959] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061845555796000, start_id=1729061845555796000, end_id=1729061845555796001) [2024-10-16 14:57:25.556043] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:25.556059] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:25.569595] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.569596] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.569623] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.569636] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.577403] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:25.577425] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:25.577495] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.580839] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D2-0-0] [lt=10] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061845548340, keepalive_ts:-1}) [2024-10-16 14:57:25.580857] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D2-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061845548340, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:25.581784] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1003}, {id:1001}]) [2024-10-16 14:57:25.585783] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=13] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:57:25.594936] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061845593054000, start_id=1729061845593054000, end_id=1729061845593054001) [2024-10-16 14:57:25.597149] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.597199] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:57:25.597213] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:57:25.597221] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] try to retry start_working [2024-10-16 14:57:25.597921] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061857008855000, latest_log_ts:{val:1729061840488170001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:25.598135] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4658, mem_used=8318976) [2024-10-16 14:57:25.619521] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=33] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:25.629120] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BB5-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061845628782000, start_id=1729061845628782000, end_id=1729061845628782001) [2024-10-16 14:57:25.629180] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BB7-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061845628782000, start_id=1729061845628782000, end_id=1729061845628782001) [2024-10-16 14:57:25.629347] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BB9-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061845628782000, start_id=1729061845628782000, end_id=1729061845628782001) [2024-10-16 14:57:25.629439] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BB8-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061845628782000, start_id=1729061845628782000, end_id=1729061845628782001) [2024-10-16 14:57:25.629469] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BB6-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061845628782000, start_id=1729061845628782000, end_id=1729061845628782001) [2024-10-16 14:57:25.632392] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD033-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=382, avg_log_submit_to_slide_time=676) [2024-10-16 14:57:25.638678] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [55867][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5D-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:25.638702] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [55867][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5D-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061845128673004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061845128673004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:25.642526] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=43] success to update min reserved snapshot(reserved_snapshot=1729060040187603000, duration=1800, snapshot_gc_ts=1729061840187603000) [2024-10-16 14:57:25.642755] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D5D-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061845642392000, start_id=1729061845642392000, end_id=1729061845642392001) [2024-10-16 14:57:25.643761] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4108-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=29, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=675, avg_log_submit_to_slide_time=1194) [2024-10-16 14:57:25.643782] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=13] update reserved snapshot finished(cost_ts=1258, reserved_snapshot=1729060040187603000) [2024-10-16 14:57:25.653144] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:25.654989] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061845654250000, start_id=1729061845654250000, end_id=1729061845654250001) [2024-10-16 14:57:25.655932] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get_max_applied_scn(scn={val:1729061845555796000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264563148}, palf_committed_end_scn_:{val:1729061845642392000, v:0}, last_check_scn_:{val:1729061845642392000, v:0}, max_applied_cb_scn_:{val:1729061845555796000, v:0}}) [2024-10-16 14:57:25.655963] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:25.663465] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=116, replace_time=0, replace_start_pos=1793049, replace_num=31457) [2024-10-16 14:57:25.663480] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=116, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:25.663968] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061845662166000, start_id=1729061845662166000, end_id=1729061845662166001) [2024-10-16 14:57:25.664262] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FC92-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:25.664273] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FC92-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27506, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:25.664277] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FC92-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=925, avg_log_submit_to_slide_time=27500) [2024-10-16 14:57:25.664281] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:25.664491] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E63-0-0] [lt=20] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=51, avg_log_batch_cnt=0, total_group_log_size=81312, avg_group_log_size=1594) [2024-10-16 14:57:25.664507] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E63-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497254, log_task={header:{begin_lsn:{lsn:207589526228}, end_lsn:{lsn:207589527852}, log_id:218497254, min_scn:{val:1729061845484957000, v:0}, max_scn:{val:1729061845484957000, v:0}, data_len:1568, proposal_id:81, prev_lsn:{lsn:207589524605}, prev_proposal_id:81, committed_end_lsn:{lsn:207589526228}, data_checksum:1344541774, accum_checksum:646599083, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061845486419, freeze_ts:1729061845486419, submit_ts:1729061845486425, flushed_ts:1729061845486718, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:293}, fs_cb_begin_ts=1729061845664502, log_life_time=178083) [2024-10-16 14:57:25.665708] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=7] query dest_info_ succ(dest_info=[[0, "file:///mnt/oceanbasedata/bddb/archive"]], archive_dest_changed=false) [2024-10-16 14:57:25.665726] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=16] finish to resize log external storage handler(current_ts=1729061845665724, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:25.673452] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=8, avg_log_batch_cnt=1, total_group_log_size=1392, avg_group_log_size=174) [2024-10-16 14:57:25.673475] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1392) [2024-10-16 14:57:25.677559] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.695713] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:25.697171] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.698835] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:111799216, max_lsn:17493444388, max_scn:{val:1729061845518506001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7667642653}, reuse_lsn:{lsn:17493444388}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17451501348}}, last_submit_lsn:{lsn:17493444266}, last_submit_end_lsn:{lsn:17493444388}, last_submit_log_id:111799216, last_submit_log_pid:17, max_flushed_lsn:{lsn:17493444266}, max_flushed_end_lsn:{lsn:17493444388}, max_flushed_log_pid:17, committed_end_lsn:{lsn:17493444388}, last_slide_log_id:111799216, last_slide_scn:{val:1729061845518506001, v:0}, last_slide_lsn:{lsn:17493444266}, last_slide_end_lsn:{lsn:17493444388}, last_slide_log_pid:17, last_slide_log_accum_checksum:3516732604, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431d4530}, [StateMgr]={this:0x2b0e431d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:111799217, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261853177, palf_role_change_cb:0x2b0e431df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:18}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:18}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494261753074, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494261753074, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262053331, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728249494772393002, v:0}, min_block_id:241, min_block_min_scn:{val:1728207118648006001, v:0}, base_lsn_for_block_gc:{lsn:17380134912}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:17380134912}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:757760}, readable_log_tail:{lsn:757760}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17514344448}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/meta", dir_fd:193, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66347008, block_header_serialize_buf_:0x2b0e431da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17493444388}, readable_log_tail:{lsn:17493444388}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17447239680}, min_scn:{val:1729032503563174001, v:0}, curr_block_id:260, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/log", dir_fd:195, min_block_id:241, max_block_id:261, curr_writable_block_id:260}, logical_block_size_:67104768, curr_block_writable_size_:20900060, block_header_serialize_buf_:0x2b0e431dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261852794, this:0x2b0e431d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:7668488755}, saved_end_lsn:{lsn:7668488755}, last_submit_prepare_req_time_us:1722494261853360, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431d89b0}) [2024-10-16 14:57:25.698979] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=141] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061845632137, last_flushed_end_lsn:{lsn:17493444388}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061845632313, last_flushed_end_lsn:{lsn:17493444388}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061845632420, last_flushed_end_lsn:{lsn:17493444388}}]) [2024-10-16 14:57:25.698991] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=111799217, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061845632137, last_flushed_end_lsn:{lsn:17493444388}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061845632313, last_flushed_end_lsn:{lsn:17493444388}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061845632420, last_flushed_end_lsn:{lsn:17493444388}}]) [2024-10-16 14:57:25.716299] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061845714416000, start_id=1729061845714416000, end_id=1729061845714416001) [2024-10-16 14:57:25.722843] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:25.731873] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:25.731896] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:25.733719] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15D0-0-0] [lt=8] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:25.735699] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:25.735718] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=211) [2024-10-16 14:57:25.741022] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:25.751017] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE763B-0-0] [lt=32] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:25.751033] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE763B-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=4488, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:25.751068] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=45701, hit_cnt=30, read_cnt=30, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:25.753233] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:25.753264] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=16] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:25.753275] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=10] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:25.756874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=51] get number(ret=0, service_type_=0, range=1, base_id=1729061845756862000, start_id=1729061845756862000, end_id=1729061845756862001) [2024-10-16 14:57:25.763180] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1333) [2024-10-16 14:57:25.764694] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E65-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=43, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:25.764715] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E65-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=69946) [2024-10-16 14:57:25.764906] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=43, stat_interval=1000000, avg (count/cost)=236, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:25.764925] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=43, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:25.764960] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=43, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:25.767035] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=68445, hit_cnt=43, read_cnt=43, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:25.769000] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:25.769769] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=26] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}) [2024-10-16 14:57:25.774006] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107004][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C410E-0-0] [lt=4] init point thread id with(&point=0x55e3435c2d80, idx_=508, point=[thread id=107004, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107004) [2024-10-16 14:57:25.777620] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.780770] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:25.788286] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.788289] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.788316] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.788351] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.788354] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.788363] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.788378] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.789017] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.789032] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.789372] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.789457] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:25.794901] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061845794738000, start_id=1729061845794738000, end_id=1729061845794738001) [2024-10-16 14:57:25.794977] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=25, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:25.794993] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9436, avg_group_log_size=629) [2024-10-16 14:57:25.795015] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9436) [2024-10-16 14:57:25.795026] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:25.795287] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=342, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:25.795312] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=20] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=221, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:25.795339] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:25.795911] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:25.797350] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.798899] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061841761413, last_first_register_time_us=1722494157071428, register_time_us=1729061841761413, curr_time_us=1729061845798898) [2024-10-16 14:57:25.798933] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061845798915, last_submit_register_req_time_us=1729061845798915, register_parent_reason=3) [2024-10-16 14:57:25.798940] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:25.821168] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1}, {id:1003}]) [2024-10-16 14:57:25.824779] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C94-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:25.824792] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C94-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20556, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:25.827379] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=361, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:25.827397] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:25.827412] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1258, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:25.827446] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:25.827468] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:25.827484] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:25.827495] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=52249198, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:25.832183] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=192, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:25.832200] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=159, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:25.850894] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3453, mem_used=8318976) [2024-10-16 14:57:25.854203] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=21] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1002, min_version={val:1729061845434460000, v:0}, base_version={val:1729061845330426000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061845434460001, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061845854184, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061845434460000, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061845734778, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061845504038000, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061845782649, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:25.855491] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=37] get number(ret=0, service_type_=0, range=1, base_id=1729061845855035000, start_id=1729061845855035000, end_id=1729061845855035001) [2024-10-16 14:57:25.856132] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9436) [2024-10-16 14:57:25.856175] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9436) [2024-10-16 14:57:25.856388] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93C0-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:25.856396] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93C0-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=1004, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:25.856400] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93C0-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=706, avg_log_submit_to_slide_time=971) [2024-10-16 14:57:25.864184] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061845862487000, start_id=1729061845862487000, end_id=1729061845862487001) [2024-10-16 14:57:25.865030] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=109, replace_time=0, replace_start_pos=1824506, replace_num=31457) [2024-10-16 14:57:25.865063] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=109, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:25.870794] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=6] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:25.871102] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=16] LogLoopThread round_cost_time(us)(round_cost_time=9) [2024-10-16 14:57:25.871450] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:25.871498] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=14] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061837234027000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:25.877702] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.886524] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:25.886729] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D44-0-0] [lt=16] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061845518506001, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061842876757002, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860098662000, latest_log_ts:{val:1729061843439703001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:25.886872] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D44-0-0] [lt=96] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061845518506001, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061842876757002, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:25.886939] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D44-0-0] [lt=32] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061845628782002, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061842876757002, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:25.887003] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D44-0-0] [lt=33] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061845629842002, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061842876757002, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:25.887033] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D44-0-0] [lt=29] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:25.897260] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=14] block manager free block(macro_id=[3465](ver=0,mode=0,seq=7667479), io_fd={first_id:7667479, second_id:3465, device_handle:null}) [2024-10-16 14:57:25.897285] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=22] block manager free block(macro_id=[10265](ver=0,mode=0,seq=7667480), io_fd={first_id:7667480, second_id:10265, device_handle:null}) [2024-10-16 14:57:25.897291] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=5] block manager free block(macro_id=[1025](ver=0,mode=0,seq=7667481), io_fd={first_id:7667481, second_id:1025, device_handle:null}) [2024-10-16 14:57:25.897296] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=5] block manager free block(macro_id=[9186](ver=0,mode=0,seq=7667482), io_fd={first_id:7667482, second_id:9186, device_handle:null}) [2024-10-16 14:57:25.897301] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=4] block manager free block(macro_id=[6546](ver=0,mode=0,seq=7667483), io_fd={first_id:7667483, second_id:6546, device_handle:null}) [2024-10-16 14:57:25.897305] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=4] block manager free block(macro_id=[1926](ver=0,mode=0,seq=7667484), io_fd={first_id:7667484, second_id:1926, device_handle:null}) [2024-10-16 14:57:25.897310] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[7447](ver=0,mode=0,seq=7667488), io_fd={first_id:7667488, second_id:7447, device_handle:null}) [2024-10-16 14:57:25.897314] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=4] block manager free block(macro_id=[187](ver=0,mode=0,seq=7667486), io_fd={first_id:7667486, second_id:187, device_handle:null}) [2024-10-16 14:57:25.897320] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=5] block manager free block(macro_id=[6127](ver=0,mode=0,seq=7667487), io_fd={first_id:7667487, second_id:6127, device_handle:null}) [2024-10-16 14:57:25.897324] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=4] block manager free block(macro_id=[12308](ver=0,mode=0,seq=7667489), io_fd={first_id:7667489, second_id:12308, device_handle:null}) [2024-10-16 14:57:25.897329] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=4] block manager free block(macro_id=[10988](ver=0,mode=0,seq=7667490), io_fd={first_id:7667490, second_id:10988, device_handle:null}) [2024-10-16 14:57:25.897333] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=4] block manager free block(macro_id=[12549](ver=0,mode=0,seq=7667491), io_fd={first_id:7667491, second_id:12549, device_handle:null}) [2024-10-16 14:57:25.897336] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[10569](ver=0,mode=0,seq=7667492), io_fd={first_id:7667492, second_id:10569, device_handle:null}) [2024-10-16 14:57:25.897340] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[13450](ver=0,mode=0,seq=7667493), io_fd={first_id:7667493, second_id:13450, device_handle:null}) [2024-10-16 14:57:25.897344] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[6850](ver=0,mode=0,seq=7667494), io_fd={first_id:7667494, second_id:6850, device_handle:null}) [2024-10-16 14:57:25.897347] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=2] block manager free block(macro_id=[10810](ver=0,mode=0,seq=7667495), io_fd={first_id:7667495, second_id:10810, device_handle:null}) [2024-10-16 14:57:25.897350] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[12790](ver=0,mode=0,seq=7667496), io_fd={first_id:7667496, second_id:12790, device_handle:null}) [2024-10-16 14:57:25.897354] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[4210](ver=0,mode=0,seq=7667498), io_fd={first_id:7667498, second_id:4210, device_handle:null}) [2024-10-16 14:57:25.897357] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[6190](ver=0,mode=0,seq=7667499), io_fd={first_id:7667499, second_id:6190, device_handle:null}) [2024-10-16 14:57:25.897362] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=2] block manager free block(macro_id=[9310](ver=0,mode=0,seq=7663721), io_fd={first_id:7663721, second_id:9310, device_handle:null}) [2024-10-16 14:57:25.897365] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[4690](ver=0,mode=0,seq=7663723), io_fd={first_id:7663723, second_id:4690, device_handle:null}) [2024-10-16 14:57:25.897369] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[4030](ver=0,mode=0,seq=7663728), io_fd={first_id:7663728, second_id:4030, device_handle:null}) [2024-10-16 14:57:25.897372] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[9551](ver=0,mode=0,seq=7663733), io_fd={first_id:7663733, second_id:9551, device_handle:null}) [2024-10-16 14:57:25.897376] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[2354](ver=0,mode=0,seq=7663747), io_fd={first_id:7663747, second_id:2354, device_handle:null}) [2024-10-16 14:57:25.897382] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[12220](ver=0,mode=0,seq=7667500), io_fd={first_id:7667500, second_id:12220, device_handle:null}) [2024-10-16 14:57:25.897385] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[11382](ver=0,mode=0,seq=7667456), io_fd={first_id:7667456, second_id:11382, device_handle:null}) [2024-10-16 14:57:25.897388] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[12042](ver=0,mode=0,seq=7667452), io_fd={first_id:7667452, second_id:12042, device_handle:null}) [2024-10-16 14:57:25.897398] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=9] block manager free block(macro_id=[1482](ver=0,mode=0,seq=7667453), io_fd={first_id:7667453, second_id:1482, device_handle:null}) [2024-10-16 14:57:25.897401] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[3462](ver=0,mode=0,seq=7667454), io_fd={first_id:7667454, second_id:3462, device_handle:null}) [2024-10-16 14:57:25.897405] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[9402](ver=0,mode=0,seq=7667455), io_fd={first_id:7667455, second_id:9402, device_handle:null}) [2024-10-16 14:57:25.897408] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[11749](ver=0,mode=0,seq=7667501), io_fd={first_id:7667501, second_id:11749, device_handle:null}) [2024-10-16 14:57:25.897412] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[10429](ver=0,mode=0,seq=7667502), io_fd={first_id:7667502, second_id:10429, device_handle:null}) [2024-10-16 14:57:25.897415] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=2] block manager free block(macro_id=[15290](ver=0,mode=0,seq=7667457), io_fd={first_id:7667457, second_id:15290, device_handle:null}) [2024-10-16 14:57:25.897419] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[12483](ver=0,mode=0,seq=7667458), io_fd={first_id:7667458, second_id:12483, device_handle:null}) [2024-10-16 14:57:25.897422] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[9309](ver=0,mode=0,seq=7667460), io_fd={first_id:7667460, second_id:9309, device_handle:null}) [2024-10-16 14:57:25.897426] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[9550](ver=0,mode=0,seq=7667461), io_fd={first_id:7667461, second_id:9550, device_handle:null}) [2024-10-16 14:57:25.897429] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=2] block manager free block(macro_id=[14411](ver=0,mode=0,seq=7667462), io_fd={first_id:7667462, second_id:14411, device_handle:null}) [2024-10-16 14:57:25.897432] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[1871](ver=0,mode=0,seq=7667463), io_fd={first_id:7667463, second_id:1871, device_handle:null}) [2024-10-16 14:57:25.897436] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[1274](ver=0,mode=0,seq=7667472), io_fd={first_id:7667472, second_id:1274, device_handle:null}) [2024-10-16 14:57:25.897439] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[4574](ver=0,mode=0,seq=7667465), io_fd={first_id:7667465, second_id:4574, device_handle:null}) [2024-10-16 14:57:25.897432] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=59] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:25.897443] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[8534](ver=0,mode=0,seq=7667466), io_fd={first_id:7667466, second_id:8534, device_handle:null}) [2024-10-16 14:57:25.897447] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[1934](ver=0,mode=0,seq=7667468), io_fd={first_id:7667468, second_id:1934, device_handle:null}) [2024-10-16 14:57:25.897450] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[7874](ver=0,mode=0,seq=7667469), io_fd={first_id:7667469, second_id:7874, device_handle:null}) [2024-10-16 14:57:25.897453] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[9854](ver=0,mode=0,seq=7667470), io_fd={first_id:7667470, second_id:9854, device_handle:null}) [2024-10-16 14:57:25.897451] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3658) [2024-10-16 14:57:25.897457] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[13814](ver=0,mode=0,seq=7667471), io_fd={first_id:7667471, second_id:13814, device_handle:null}) [2024-10-16 14:57:25.897460] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[12735](ver=0,mode=0,seq=7667473), io_fd={first_id:7667473, second_id:12735, device_handle:null}) [2024-10-16 14:57:25.897460] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:25.897463] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[10095](ver=0,mode=0,seq=7667503), io_fd={first_id:7667503, second_id:10095, device_handle:null}) [2024-10-16 14:57:25.897467] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[8356](ver=0,mode=0,seq=7667474), io_fd={first_id:7667474, second_id:8356, device_handle:null}) [2024-10-16 14:57:25.897470] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=2] block manager free block(macro_id=[677](ver=0,mode=0,seq=7667504), io_fd={first_id:7667504, second_id:677, device_handle:null}) [2024-10-16 14:57:25.897473] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[13458](ver=0,mode=0,seq=7667505), io_fd={first_id:7667505, second_id:13458, device_handle:null}) [2024-10-16 14:57:25.897477] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[2898](ver=0,mode=0,seq=7667506), io_fd={first_id:7667506, second_id:2898, device_handle:null}) [2024-10-16 14:57:25.897480] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[4878](ver=0,mode=0,seq=7667475), io_fd={first_id:7667475, second_id:4878, device_handle:null}) [2024-10-16 14:57:25.897484] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[8838](ver=0,mode=0,seq=7667507), io_fd={first_id:7667507, second_id:8838, device_handle:null}) [2024-10-16 14:57:25.897487] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=2] block manager free block(macro_id=[14778](ver=0,mode=0,seq=7667508), io_fd={first_id:7667508, second_id:14778, device_handle:null}) [2024-10-16 14:57:25.897490] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[4218](ver=0,mode=0,seq=7667509), io_fd={first_id:7667509, second_id:4218, device_handle:null}) [2024-10-16 14:57:25.897493] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[11059](ver=0,mode=0,seq=7667510), io_fd={first_id:7667510, second_id:11059, device_handle:null}) [2024-10-16 14:57:25.897497] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[9750](ver=0,mode=0,seq=7667476), io_fd={first_id:7667476, second_id:9750, device_handle:null}) [2024-10-16 14:57:25.897501] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[3213](ver=0,mode=0,seq=7667478), io_fd={first_id:7667478, second_id:3213, device_handle:null}) [2024-10-16 14:57:25.899925] INFO [STORAGE.BLKMGR] mark_and_sweep (ob_block_manager.cpp:1053) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] finish once mark and sweep(ret=0, alloc_num=3982210, mark_info.count()=58, marker_status={total_block_count:15360, reserved_block_count:2, linked_block_count:17, tmp_file_count:0, data_block_count:7208, shared_data_block_count:162, index_block_count:14, ids_block_count:0, disk_block_count:0, bloomfiter_count:0, hold_count:0, pending_free_count:58, free_count:7861, shared_meta_block_count:96, mark_cost_time:22873, sweep_cost_time:261, start_time:"2024-10-16 14:57:25.874381", last_end_time:"2024-10-16 14:57:25.897515", hold_info:nothing}, map_cnt=7497) [2024-10-16 14:57:25.900282] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=31] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:25.900498] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.912021] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1, io_status_sys_log=sys_group_name: SHARED_BLOCK_ReaderWriter_IO, mode: read, size: 40960.00, iops: 1.00, rt: 155.00; ) [2024-10-16 14:57:25.912048] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO STATUS CONFIG](tenant_id=1, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:3}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:25.912092] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 253.00; ) [2024-10-16 14:57:25.912101] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:25.912130] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 218.75; sys_group_name: SHARED_BLOCK_ReaderWriter_IO, mode: read, size: 8192.00, iops: 1.00, rt: 120.00; ) [2024-10-16 14:57:25.912143] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:25.912151] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:25.916502] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061845915948000, start_id=1729061845915948000, end_id=1729061845915948001) [2024-10-16 14:57:25.918862] INFO [SERVER] refresh_avail_memory (ob_table_load_assigned_memory_manager.cpp:117) [22141][T1002_TntShared][T1002][Y0-0000000000000000-0-0] [lt=19] ObTableLoadAssignedMemoryManager::refresh_avail_memory(MTL_ID()=1002, avail_memory=53687091, chunk_count_=0, avail_sort_memory_=53687091, avail_memory_=53687091) [2024-10-16 14:57:25.920761] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:25.925222] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C96-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=8, avg_log_submit_to_flush_time=1520, avg_log_submit_to_slide_time=27812) [2024-10-16 14:57:25.925238] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53C96-0-0] [lt=15][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172991, log_task={header:{begin_lsn:{lsn:14352355114}, end_lsn:{lsn:14352355236}, log_id:115172991, min_scn:{val:1729061845629842001, v:0}, max_scn:{val:1729061845629842001, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352354992}, prev_proposal_id:17, committed_end_lsn:{lsn:14352355114}, data_checksum:940009244, accum_checksum:3629699454, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061845824757, freeze_ts:1729061845824757, submit_ts:1729061845824764, flushed_ts:1729061845827417, gen_to_freeze cost time:0, gen_to_submit cost time:7, submit_to_flush cost time:2653}, fs_cb_begin_ts=1729061845925215, log_life_time=100458) [2024-10-16 14:57:25.925282] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:25.925334] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=48] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656793, miss_cnt:17, cache_read_size:8050145457, hit_ratio:"9.999997286807291719e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:25.936141] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=6] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601324, offset=9607478}, end_cursor:ObLogCursor{file_id=192, log_id=47601325, offset=9607762}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a54050, len:1738, log_data_len:284, seq:47601324, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:25.938087] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA494-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061845937408000, start_id=1729061845937408000, end_id=1729061845937408001) [2024-10-16 14:57:25.939124] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2266, mem_used=6239232) [2024-10-16 14:57:25.939480] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257690, offset=39012709}, end_cursor:ObLogCursor{file_id=197, log_id=49257691, offset=39012992}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a3c050, len:1691, log_data_len:283, seq:49257690, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:25.939696] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601325, offset=9607762}, end_cursor:ObLogCursor{file_id=192, log_id=47601326, offset=9608036}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2af4050, len:1454, log_data_len:274, seq:47601325, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:25.941411] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:25.941712] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB35F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061845941597000, start_id=1729061845941597000, end_id=1729061845941597001) [2024-10-16 14:57:25.942694] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601326, offset=9608036}, end_cursor:ObLogCursor{file_id=192, log_id=47601327, offset=9608310}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d84050, len:1180, log_data_len:274, seq:47601326, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:25.942925] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601327, offset=9608310}, end_cursor:ObLogCursor{file_id=192, log_id=47601328, offset=9608584}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b2c050, len:906, log_data_len:274, seq:47601327, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:25.942924] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257691, offset=39012992}, end_cursor:ObLogCursor{file_id=197, log_id=49257692, offset=39013274}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b10050, len:1408, log_data_len:282, seq:49257691, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:25.943056] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561121, miss_cnt:1, cache_read_size:152590656762, hit_ratio:"9.999999928346806621e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:25.943068] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598705, miss_cnt:9, cache_read_size:149777892659, hit_ratio:"9.999999345924119876e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:25.945974] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:25.945989] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:25.946004] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=9] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=146) [2024-10-16 14:57:25.946052] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257692, offset=39013274}, end_cursor:ObLogCursor{file_id=197, log_id=49257693, offset=39013556}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c34050, len:1126, log_data_len:282, seq:49257692, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:25.946181] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257693, offset=39013556}, end_cursor:ObLogCursor{file_id=197, log_id=49257694, offset=39013838}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a04050, len:844, log_data_len:282, seq:49257693, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:25.946233] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650056, miss_cnt:32, cache_read_size:8050397964, hit_ratio:"9.999994892265753332e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:25.951112] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE763F-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:25.951307] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=582476, this={dio_aligned_buf:{buf_write_offset:966, buf_padding_size:3130, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111160156, truncate_used_ts:41410904}, log_block_size:67108864, dir_fd:177, io_fd:142}, aligned_buf_len=4096, aligned_block_offset=581632, buf_len=122, total_write_size_=1317173, total_write_size_after_dio_=4591616, ob_pwrite_used_ts=196250, count=802, each_pwrite_cost=244) [2024-10-16 14:57:25.951697] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061845951696, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007615, add_bytes: 0 [2024-10-16 14:57:25.952543] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA870-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:25.952570] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA870-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061845629842002, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061845629842002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:25.953773] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA870-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061845953219000, start_id=1729061845953219000, end_id=1729061845953219001) [2024-10-16 14:57:25.957119] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061845956403000, start_id=1729061845956403000, end_id=1729061845956403001) [2024-10-16 14:57:25.958044] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1729061845662166000, v:0}, new_max_version={val:1729061845662166000, v:0}, last_min_version={val:1729061845555796000, v:0}, last_max_version={val:1729061845555796000, v:0}, total_time_us=1838, delta_us=295877) [2024-10-16 14:57:25.958069] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1729061845662166000, v:0}, min_version={val:1729061845662166000, v:0}, max_version={val:1729061845662166000, v:0}, in_service=true, leader_epoch=33576) [2024-10-16 14:57:25.958130] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA870-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061845957425000, start_id=1729061845957425000, end_id=1729061845957425001) [2024-10-16 14:57:25.958691] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:25.963096] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B17-0-0] [lt=18] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:25.963118] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B17-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178298, log_task={header:{begin_lsn:{lsn:14538468879}, end_lsn:{lsn:14538469001}, log_id:115178298, min_scn:{val:1729061845628782002, v:0}, max_scn:{val:1729061845628782002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538468757}, prev_proposal_id:23, committed_end_lsn:{lsn:14538468879}, data_checksum:1715074492, accum_checksum:936510124, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061845862997, freeze_ts:1729061845862997, submit_ts:1729061845863001, flushed_ts:1729061845865144, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:2143}, fs_cb_begin_ts=1729061845963112, log_life_time=100115) [2024-10-16 14:57:25.963321] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=494, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:25.963336] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=535, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:25.963398] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:25.967225] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061845967223, bytes: 4446807104736, bw: 0.703587 MB/s, add_ts: 1003732, add_bytes: 740518 [2024-10-16 14:57:25.973407] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=25] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:25.973660] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:25.973685] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT APPEND COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:25.973984] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=389, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:25.974002] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=31451) [2024-10-16 14:57:25.974014] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:25.974023] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=32953) [2024-10-16 14:57:25.974034] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:25.977697] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:25.977717] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:25.977723] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:25.995061] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061845993712000, start_id=1729061845993712000, end_id=1729061845993712001) [2024-10-16 14:57:25.995821] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33705, start_service_tstamp_=1722494263684720, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:26.000540] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.015036] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F0F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061846014611000, start_id=1729061846014611000, end_id=1729061846014611001) [2024-10-16 14:57:26.019781] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061846018854000, start_id=1729061846018854000, end_id=1729061846018854001) [2024-10-16 14:57:26.019807] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=17] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061846019773], gts:1729061846018854000, latest_srr:[mts=1729061846019773]}) [2024-10-16 14:57:26.024769] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061846023878000, start_id=1729061846023878000, end_id=1729061846023878001) [2024-10-16 14:57:26.031674] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=35] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:26.031985] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:26.032001] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:26.032537] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD03F-0-0] [lt=19] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:26.033066] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FC9D-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:26.052536] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=237, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:26.052555] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=33, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:26.052591] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:26.055054] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061846054015000, start_id=1729061846054015000, end_id=1729061846054015001) [2024-10-16 14:57:26.058426] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=9] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:26.059086] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=25] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:26.059136] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061840187603000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:26.059182] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=44] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061840187603000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1002, freeze_info:{latest_snapshot_gc_scn:{val:1729061840187603000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}}) [2024-10-16 14:57:26.064906] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061846063778000, start_id=1729061846063778000, end_id=1729061846063778001) [2024-10-16 14:57:26.065406] INFO [LIB] stat (utility.h:1165) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B19-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:26.065422] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106753][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B19-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:26.067246] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] Cache replace map node details(ret=0, replace_node_count=190, replace_time=0, replace_start_pos=1855963, replace_num=31457) [2024-10-16 14:57:26.067263] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=190, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:26.077757] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.082541] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:26.082725] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D42-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061846018854000, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061843668207001, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860721265000, latest_log_ts:{val:1729061844127564001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:26.082896] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D42-0-0] [lt=89] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061846018854000, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061843668207001, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:26.083044] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D42-0-0] [lt=63] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061845628782003, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061843668207001, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:26.083183] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D42-0-0] [lt=62] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061845629842002, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061843668207001, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:26.083246] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D42-0-0] [lt=62] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:26.097820] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3658) [2024-10-16 14:57:26.097873] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3658) [2024-10-16 14:57:26.098203] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F80-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:26.098213] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F80-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=926, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:26.100570] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.110405] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:26.110428] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=23][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:26.112317] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061836111381, end_time=1729061846112316, dag_status={}) [2024-10-16 14:57:26.112332] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=14] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.112339] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.112343] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.112346] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.112350] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.112367] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=18] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.112371] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.112377] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.112381] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.112384] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112392] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112404] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112415] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112418] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112425] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112429] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112436] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112440] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112447] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112454] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112457] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112467] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112474] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112478] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112485] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112489] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112495] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112499] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112506] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112510] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112516] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112521] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112527] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112531] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112537] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112547] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112551] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112557] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112563] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112567] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112571] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112577] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112581] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112588] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112592] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112598] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112611] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112615] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112621] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112625] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112632] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112636] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112642] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112646] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112653] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112656] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112666] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112672] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112676] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112680] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112687] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112691] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112697] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112701] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112708] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.112712] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:26.112718] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:26.112722] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:26.112727] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:26.112730] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:26.112732] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:26.112735] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:26.112737] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:26.112740] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:26.112742] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:26.112748] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:26.115635] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] no need to update min/max version(new_version={val:1729061845714416000, v:0}, min_version={val:1729061845714416000, v:0}, max_version={val:1729061845714416000, v:0}) [2024-10-16 14:57:26.122424] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15CA-0-0] [lt=10] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:26.122482] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=20] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:26.123730] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=18] gc stale ls task succ [2024-10-16 14:57:26.125172] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=17] current monitor number(seq_=-1) [2024-10-16 14:57:26.128159] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=697, mem_used=2079744) [2024-10-16 14:57:26.129741] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [21543][T1_CdcSrv][T1][Y0-0000000000000000-0-0] [lt=11] finish to resize log external storage handler(current_ts=1729061846129740, tenant_max_cpu=3, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:26.129819] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BBE-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061846129257000, start_id=1729061846129257000, end_id=1729061846129257001) [2024-10-16 14:57:26.129829] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BBB-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061846129257000, start_id=1729061846129257000, end_id=1729061846129257001) [2024-10-16 14:57:26.129868] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BBC-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061846129257000, start_id=1729061846129257000, end_id=1729061846129257001) [2024-10-16 14:57:26.129884] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BBD-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061846129257000, start_id=1729061846129257000, end_id=1729061846129257001) [2024-10-16 14:57:26.129936] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BBA-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1729061846129257000, start_id=1729061846129257000, end_id=1729061846129257001) [2024-10-16 14:57:26.132047] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:26.132066] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:26.132315] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:26.141717] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:26.143586] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15CE-0-0] [lt=14] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:26.145596] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CA3-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510314013988, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410519, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:26.146190] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106793][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC78-0-0] [lt=10] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:26.146219] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106793][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC78-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:26.146237] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106793][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC78-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:26.151316] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=57, stat_interval=1000000, avg (count/cost)=37, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:26.151390] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=72] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=57, stat_interval=1000000, avg (count/cost)=87, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:26.156333] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1003, min_version={val:1729061845956403000, v:0}, base_version={val:1729061845662166000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061845956403000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061846156322, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061845956403000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061846035721, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061845956403000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061846101340, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:26.157240] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061846156735000, start_id=1729061846156735000, end_id=1729061846156735001) [2024-10-16 14:57:26.157321] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9331, avg_group_log_size=666) [2024-10-16 14:57:26.157332] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9331) [2024-10-16 14:57:26.157583] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=416, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:26.157596] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9331) [2024-10-16 14:57:26.157610] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=162, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:26.157628] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9331) [2024-10-16 14:57:26.157639] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:26.158045] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD02C-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:26.158051] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD02C-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=1028, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:26.158055] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD02C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=594, avg_log_submit_to_slide_time=994) [2024-10-16 14:57:26.165715] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E6D-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=31, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:26.165728] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E6D-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=31, stat_interval=1000000, avg (count/cost)=11638, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:26.169247] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFFF-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061846160451, keepalive_ts:-1}) [2024-10-16 14:57:26.169269] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFCFFF-0-0] [lt=21] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061846160451, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:26.169759] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:26.169775] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:26.169782] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=7] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:26.169785] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:26.169788] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:26.169792] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16722) [2024-10-16 14:57:26.172745] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501C-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:26.172765] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501C-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:26.173756] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=18705, avg_group_log_size=1039) [2024-10-16 14:57:26.173769] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=20206) [2024-10-16 14:57:26.174276] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107004][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C411A-0-0] [lt=19] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=268, avg_log_submit_to_slide_time=530) [2024-10-16 14:57:26.177815] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.195257] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061846194960000, start_id=1729061846194960000, end_id=1729061846194960001) [2024-10-16 14:57:26.200619] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.216586] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061846215975000, start_id=1729061846215975000, end_id=1729061846215975001) [2024-10-16 14:57:26.223723] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:57:26.223745] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=18] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=82, cache_node_num=82) [2024-10-16 14:57:26.223758] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=12] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=82, cache_node_num=82) [2024-10-16 14:57:26.223763] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:26.225953] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=82, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}]) [2024-10-16 14:57:26.226090] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=135] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:26.230164] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21658][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3DDF-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510314098558, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410520, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:26.232140] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:26.232155] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:26.233703] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:57:26.234550] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061846232916000, start_id=1729061846232916000, end_id=1729061846232916001) [2024-10-16 14:57:26.235142] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCA3-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:26.235154] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCA3-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1429, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:26.235159] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCA3-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=518, avg_log_submit_to_slide_time=1413) [2024-10-16 14:57:26.255485] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061846255319000, start_id=1729061846255319000, end_id=1729061846255319001) [2024-10-16 14:57:26.256594] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get_max_applied_scn(scn={val:1729061846018854000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959673328}, palf_committed_end_scn_:{val:1729061846018854001, v:0}, last_check_scn_:{val:1729061846018854001, v:0}, max_applied_cb_scn_:{val:1729061846018854000, v:0}}) [2024-10-16 14:57:26.256628] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=30] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:26.256704] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=41] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:57:26.256719] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:213058217146}, scn={val:1729061846129257001, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061846151736, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058217146}, next_to_submit_scn_:{val:1729061846129257001, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058217024}, end_lsn:{lsn:213058217146}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061846129257000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061846129257000, v:0}, data_checksum:3348837161, flag:1}}, init_mode_version:0, accumulate_checksum:1099743353, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061846129257000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561124, miss_cnt:1, cache_read_size:152590657128, hit_ratio:"9.999999928346808842e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:26.265000] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061846263815000, start_id=1729061846263815000, end_id=1729061846263815001) [2024-10-16 14:57:26.266489] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=26836) [2024-10-16 14:57:26.268837] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=527, replace_time=0, replace_start_pos=1887420, replace_num=31457) [2024-10-16 14:57:26.268876] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] replace map num details(ret=0, replace_node_count=527, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:26.269306] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD000-0-0] [lt=10] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061846260500, keepalive_ts:-1}) [2024-10-16 14:57:26.269330] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD000-0-0] [lt=22] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061846260500, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:26.274568] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3F3B-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:26.274582] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3F3B-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1128, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:26.275134] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [22253][T1002_Compactio][T1002][YB4A0A000407-00061E99741D0D6E-0-0] [lt=10] InfoPoolResizeTask(cost_ts=10) [2024-10-16 14:57:26.277853] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.282950] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D35-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061846274065, keepalive_ts:-1}) [2024-10-16 14:57:26.282974] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D35-0-0] [lt=23] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061846274065, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:26.283685] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D36-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061846274106, keepalive_ts:-1}) [2024-10-16 14:57:26.283702] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D36-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061846274106, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:26.294472] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] [WRS] update tenant weak read server version(tenant_id=1001, server_version={version:{val:1729061846194960000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061846294407}, version_delta=99510) [2024-10-16 14:57:26.300652] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.311099] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:26.311121] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=15] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:26.311143] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373532188, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:26.311285] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=7] ====== traversal_flush timer task ====== [2024-10-16 14:57:26.311378] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA7-0-0] [lt=13] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061859029444000, latest_log_ts:{val:1729061842479541001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:26.311438] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA7-0-0] [lt=58] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:26.313323] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=5870, mem_used=12478464) [2024-10-16 14:57:26.321361] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=16] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:26.328814] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4889-0-0] [lt=5] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061846296357, keepalive_ts:-1}) [2024-10-16 14:57:26.328868] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4889-0-0] [lt=53] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061846296357, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:26.330434] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=24] LogLoopThread round_cost_time(us)(round_cost_time=14) [2024-10-16 14:57:26.330634] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:26.330652] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:26.330669] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:26.332410] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=400, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:26.332428] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=118, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:26.332435] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=36] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:26.332758] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD049-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:26.332766] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD049-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=801, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:26.338287] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:26.338315] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=21] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:26.338330] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=11] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:26.338353] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] TxData Memory Statistic(logstream info): (ls_id={id:1}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=10) [2024-10-16 14:57:26.338369] INFO [STORAGE] check_and_freeze_tx_data_ (ob_tenant_freezer.cpp:566) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=13] TxData Memory Statistic : (Tenant Total Memory(MB)=1024, Tenant Frozen TxData Memory(MB)=0, Tenant Active TxData Memory(MB)=10, Freeze TxData Trigger Memory(MB)=20, Total TxDataTable Hold Memory(MB)=26, Total TxDataTable Memory Limit(MB)=204) [2024-10-16 14:57:26.342027] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:26.344754] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E02-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061846344223000, start_id=1729061846344223000, end_id=1729061846344223001) [2024-10-16 14:57:26.349126] INFO [COMMON] generate_mod_stat_task (memory_dump.cpp:218) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] task info(*task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:26.349148] INFO [COMMON] handle (memory_dump.cpp:550) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=21] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:26.350038] INFO [COMMON] update_check_range (ob_sql_mem_leak_checker.cpp:62) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] update_check_range(min_check_version=652897, max_check_version=652898, global_version=652899) [2024-10-16 14:57:26.351471] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7647-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=18714) [2024-10-16 14:57:26.351797] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=17212) [2024-10-16 14:57:26.354769] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=32] get_max_applied_scn(scn={val:1729061846018854001, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023082566}, palf_committed_end_scn_:{val:1729061846018854002, v:0}, last_check_scn_:{val:1729061846018854002, v:0}, max_applied_cb_scn_:{val:1729061846018854001, v:0}}) [2024-10-16 14:57:26.354789] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:26.357410] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061846355944000, start_id=1729061846355944000, end_id=1729061846355944001) [2024-10-16 14:57:26.366078] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=11] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:57:26.366094] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=15] try gc retain ctx [2024-10-16 14:57:26.366098] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:26.366765] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061859029444000, latest_log_ts:{val:1729061842479541001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:26.370981] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:133533115, max_lsn:66390062298, max_scn:{val:1729061846215975000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:33216076378}, reuse_lsn:{lsn:66390062298}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:66348119258}}, last_submit_lsn:{lsn:66390061711}, last_submit_end_lsn:{lsn:66390062298}, last_submit_log_id:133533115, last_submit_log_pid:48, max_flushed_lsn:{lsn:66390061711}, max_flushed_end_lsn:{lsn:66390062298}, max_flushed_log_pid:48, committed_end_lsn:{lsn:66390062298}, last_slide_log_id:133533115, last_slide_scn:{val:1729061846215975000, v:0}, last_slide_lsn:{lsn:66390061711}, last_slide_end_lsn:{lsn:66390062298}, last_slide_log_pid:48, last_slide_log_accum_checksum:1150480322, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0de0acc530}, [StateMgr]={this:0x2b0de0ad0830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:133533116, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261740374, palf_role_change_cb:0x2b0de0ad7230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, reconfig_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, persistent_config_version:{proposal_id:48, config_seq:11}, ms_ack_list:0, resend_config_version:{proposal_id:48, config_seq:11}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494260838312, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494260838312, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0de0acdc70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261941355, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728741813401825001, v:0}, min_block_id:965, min_block_min_scn:{val:1728727703672931001, v:0}, base_lsn_for_block_gc:{lsn:66098196480}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, log_config_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, log_snapshot_meta:{version:1, base_lsn:{lsn:66098196480}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1634304}, readable_log_tail:{lsn:1634304}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66433720320}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/meta", dir_fd:128, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65470464, block_header_serialize_buf_:0x2b0de0ad29f8, flashback_version:0}, log_storage:{log_tail:{lsn:66390062298}, readable_log_tail:{lsn:66390062298}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66366615552}, min_scn:{val:1729057321384354001, v:0}, curr_block_id:989, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/log", dir_fd:134, min_block_id:965, max_block_id:990, curr_writable_block_id:989}, logical_block_size_:67104768, curr_block_writable_size_:43658022, block_header_serialize_buf_:0x2b0de0ad46b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261739251, this:0x2b0de0ad0f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:48, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:47, majority_max_lsn:{lsn:33216819320}, saved_end_lsn:{lsn:33216819320}, last_submit_prepare_req_time_us:1722494261740552, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0de0ad09b0}) [2024-10-16 14:57:26.371128] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=139] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061846216965, last_flushed_end_lsn:{lsn:66390062298}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061846217219, last_flushed_end_lsn:{lsn:66390062298}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061846217915, last_flushed_end_lsn:{lsn:66390062298}}]) [2024-10-16 14:57:26.371139] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=133533116, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061846216965, last_flushed_end_lsn:{lsn:66390062298}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061846217219, last_flushed_end_lsn:{lsn:66390062298}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061846217915, last_flushed_end_lsn:{lsn:66390062298}}]) [2024-10-16 14:57:26.373845] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:26.373862] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:26.374087] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=218, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:26.374098] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1210) [2024-10-16 14:57:26.374109] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=34, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:26.374137] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1210) [2024-10-16 14:57:26.374146] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:26.375375] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3F3D-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:26.375381] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3F3D-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=674, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:26.375510] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=18] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:26.377710] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106753][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=7] worker thread exit(tenant_->id()=1004, workers_.get_size()=2) [2024-10-16 14:57:26.377728] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106753][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=10] tenant thread end_run(id_=1004, ret=0, thread_count_=191) [2024-10-16 14:57:26.377932] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.382391] INFO handle (memory_dump.cpp:677) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=12] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 202 258 0 1 2 117 1 0 1 4 1 1 0 1 5 38 4 0 1 7 1 2 0 1 8 65 1 0 1 12 18 3 0 1 13 6 1 0 1 14 1 2 0 1 15 1 1 0 1 18 26 3 0 1 20 2 2 0 1 22 1 1 0 500 0 139 255 0 500 7 7 4 0 500 8 50 1 0 500 9 2 1 0 500 10 10 2 0 500 16 3 1 0 500 17 10 7 0 500 22 111 63 0 500 23 61 11 0 508 0 4 13 0 508 8 5 1 0 509 0 3 8 0 509 8 5 1 0 1001 0 129 232 0 1001 2 21 1 0 1001 4 1 1 0 1001 5 23 4 0 1001 8 44 1 0 1001 12 10 3 0 1001 13 3 1 0 1001 14 1 2 0 1001 18 14 3 0 1001 20 1 2 0 1002 0 173 258 0 1002 2 21 1 0 1002 4 2 1 0 1002 5 24 4 0 1002 8 50 1 0 1002 12 19 3 0 1002 13 7 1 0 1002 14 1 2 0 1002 18 14 3 0 1002 20 7 2 0 1003 0 141 235 0 1003 2 29 1 0 1003 4 1 1 0 1003 5 23 4 0 1003 8 45 1 0 1003 12 10 3 0 1003 13 3 1 0 1003 14 1 2 0 1003 18 13 3 0 1003 20 1 2 0 1004 0 291 256 0 1004 2 19 1 0 1004 4 2 1 0 1004 5 34 4 0 1004 8 52 1 0 1004 12 19 3 0 1004 13 10 1 0 1004 14 2 2 0 1004 15 2 1 0 1004 18 25 3 0 1004 20 11 2 0 cost_time: 32382 [2024-10-16 14:57:26.382426] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=13] [MEMORY] tenant: 1, limit: 2,147,483,648 hold: 1,978,109,952 rpc_hold: 0 cache_hold: 861,929,472 cache_used: 861,929,472 cache_item_count: 411 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 535,248,896 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MEMSTORE_CTX_ID hold_bytes= 245,366,784 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TRANS_CTX_MGR_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PLAN_CACHE_CTX_ID hold_bytes= 79,691,776 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 136,314,880 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= META_OBJ_CTX_ID hold_bytes= 37,748,736 limit= 429,496,720 [MEMORY] ctx_id= TX_CALLBACK_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOB_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PS_CACHE_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TX_DATA_TABLE hold_bytes= 54,546,432 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MDS_DATA_ID hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:26.382693] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=11] [MEMORY] tenant_id= 1 ctx_id= DEFAULT_CTX_ID hold= 535,248,896 used= 422,562,304 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 32 [MEMORY] hold= 56,180,736 used= 56,153,088 count= 27 avg_used= 2,079,744 block_cnt= 27 chunk_cnt= 27 mod=CACHE_MAP_NODE [MEMORY] hold= 41,963,520 used= 41,943,040 count= 1 avg_used= 41,943,040 block_cnt= 1 chunk_cnt= 1 mod=LogGroupBuffer [MEMORY] hold= 30,965,760 used= 27,525,120 count= 420 avg_used= 65,536 block_cnt= 420 chunk_cnt= 50 mod=MinorMergeMgr [MEMORY] hold= 23,511,040 used= 23,433,792 count= 8 avg_used= 2,929,224 block_cnt= 8 chunk_cnt= 7 mod=IoControl [MEMORY] hold= 22,880,256 used= 21,568,040 count= 161 avg_used= 133,962 block_cnt= 161 chunk_cnt= 38 mod=SqlPlanMon [MEMORY] hold= 18,874,368 used= 16,967,424 count= 256 avg_used= 66,279 block_cnt= 256 chunk_cnt= 63 mod=SqlDtlBuf [MEMORY] hold= 17,694,720 used= 16,942,080 count= 240 avg_used= 70,592 block_cnt= 240 chunk_cnt= 30 mod=[T]ObSessionDIB [MEMORY] hold= 16,797,696 used= 16,777,336 count= 1 avg_used= 16,777,336 block_cnt= 1 chunk_cnt= 1 mod=MrgMemPoolChk [MEMORY] hold= 15,034,848 used= 14,886,881 count= 98 avg_used= 151,906 block_cnt= 31 chunk_cnt= 7 mod=OmtTenant [MEMORY] hold= 11,771,904 used= 9,838,592 count= 236 avg_used= 41,688 block_cnt= 236 chunk_cnt= 61 mod=[T]char [MEMORY] hold= 10,911,744 used= 9,699,328 count= 148 avg_used= 65,536 block_cnt= 148 chunk_cnt= 38 mod=MajorMergeMgr [MEMORY] hold= 8,777,728 used= 8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv [MEMORY] hold= 8,409,088 used= 8,388,640 count= 1 avg_used= 8,388,640 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 6,094,848 used= 6,033,408 count= 8 avg_used= 754,176 block_cnt= 8 chunk_cnt= 7 mod=ResourceGroup [MEMORY] hold= 4,964,352 used= 4,959,488 count= 3 avg_used= 1,653,162 block_cnt= 3 chunk_cnt= 3 mod=SqlFltSpanRec [MEMORY] hold= 4,943,872 used= 4,915,456 count= 2 avg_used= 2,457,728 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe [MEMORY] hold= 4,235,264 used= 4,194,304 count= 2 avg_used= 2,097,152 block_cnt= 2 chunk_cnt= 2 mod=SHARE_BLOCK [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMem [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 12 mod=CkptDgnMemCU [MEMORY] hold= 3,772,416 used= 3,751,936 count= 1 avg_used= 3,751,936 block_cnt= 1 chunk_cnt= 1 mod=DiskIteratorSto [MEMORY] hold= 3,723,264 used= 3,698,688 count= 2 avg_used= 1,849,344 block_cnt= 2 chunk_cnt= 2 mod=LogDIOAligned [MEMORY] hold= 3,162,112 used= 3,145,808 count= 2 avg_used= 1,572,904 block_cnt= 2 chunk_cnt= 2 mod=TenantDLMgr [MEMORY] hold= 3,145,728 used= 2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 5 mod=SqlDtlQueue [MEMORY] hold= 2,899,968 used= 2,890,688 count= 3 avg_used= 963,562 block_cnt= 3 chunk_cnt= 3 mod=MysqlRequesReco [MEMORY] hold= 2,828,928 used= 2,708,352 count= 604 avg_used= 4,484 block_cnt= 576 chunk_cnt= 72 mod=WorkerMap [MEMORY] hold= 2,777,088 used= 2,760,192 count= 1 avg_used= 2,760,192 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo [MEMORY] hold= 2,415,648 used= 2,387,720 count= 104 avg_used= 22,958 block_cnt= 104 chunk_cnt= 4 mod=TabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=FastFrezCkr [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap [MEMORY] hold= 2,375,680 used= 2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta [MEMORY] hold= 2,248,704 used= 2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb [MEMORY] hold= 2,198,864 used= 416,700 count= 25,277 avg_used= 16 block_cnt= 270 chunk_cnt= 2 mod=Number [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=LogReplayTask [MEMORY] hold= 1,843,200 used= 1,770,696 count= 9 avg_used= 196,744 block_cnt= 9 chunk_cnt= 2 mod=DagMap [MEMORY] hold= 1,777,664 used= 1,770,696 count= 1 avg_used= 1,770,696 block_cnt= 1 chunk_cnt= 1 mod=TableProc [MEMORY] hold= 1,548,288 used= 1,507,118 count= 5 avg_used= 301,423 block_cnt= 5 chunk_cnt= 1 mod=LDIOSetup [MEMORY] hold= 1,376,256 used= 1,322,752 count= 8 avg_used= 165,344 block_cnt= 8 chunk_cnt= 7 mod=SQLSessionInfo [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=TransTimeWheel [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=XATimeWheel [MEMORY] hold= 1,261,568 used= 681,912 count= 77 avg_used= 8,856 block_cnt= 77 chunk_cnt= 31 mod=[T]ObDSActionAr [MEMORY] hold= 1,196,032 used= 1,182,720 count= 2 avg_used= 591,360 block_cnt= 2 chunk_cnt= 2 mod=ServerObjecPool [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsInfo [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=RewriteRuleMap [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsCache [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPlanCac [MEMORY] hold= 1,024,000 used= 1,015,808 count= 1 avg_used= 1,015,808 block_cnt= 1 chunk_cnt= 1 mod=CommitVersions [MEMORY] hold= 983,488 used= 960,256 count= 121 avg_used= 7,936 block_cnt= 121 chunk_cnt= 9 mod=TltMemtablMgr [MEMORY] hold= 958,464 used= 858,728 count= 13 avg_used= 66,056 block_cnt= 13 chunk_cnt= 13 mod=CommSysVarFac [MEMORY] hold= 958,464 used= 950,272 count= 1 avg_used= 950,272 block_cnt= 1 chunk_cnt= 1 mod=IOWorkerLQ [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=ArcSenderQueue [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=TransService [MEMORY] hold= 802,816 used= 394,352 count= 49 avg_used= 8,048 block_cnt= 49 chunk_cnt= 22 mod=[T]ObPerfEventR [MEMORY] hold= 798,400 used= 779,200 count= 100 avg_used= 7,792 block_cnt= 100 chunk_cnt= 20 mod=CkptDgnMemNode [MEMORY] hold= 798,112 used= 789,624 count= 3 avg_used= 263,208 block_cnt= 2 chunk_cnt= 2 mod=PartitTableTask [MEMORY] hold= 786,688 used= 524,352 count= 33 avg_used= 15,889 block_cnt= 33 chunk_cnt= 4 mod=LogAlloc [MEMORY] hold= 765,440 used= 730,112 count= 184 avg_used= 3,968 block_cnt= 184 chunk_cnt= 44 mod=CkpDgn [MEMORY] hold= 663,552 used= 655,360 count= 1 avg_used= 655,360 block_cnt= 1 chunk_cnt= 1 mod=FetchLog [MEMORY] hold= 663,552 used= 659,200 count= 1 avg_used= 659,200 block_cnt= 1 chunk_cnt= 1 mod=MulLevelQueue [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=ElectTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTF [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DetectorTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=FrzTrigger [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTR [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MultiVersionGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=OBJLockGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DupTbLease [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MdsT [MEMORY] hold= 598,016 used= 590,232 count= 1 avg_used= 590,232 block_cnt= 1 chunk_cnt= 1 mod=DagNetIdMap [MEMORY] hold= 458,752 used= 440,928 count= 3 avg_used= 146,976 block_cnt= 3 chunk_cnt= 3 mod=bf_queue [MEMORY] hold= 421,504 used= 404,416 count= 89 avg_used= 4,544 block_cnt= 89 chunk_cnt= 31 mod=[T]ObTraceEvent [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ApplySrv [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ReplaySrv [MEMORY] hold= 406,400 used= 396,800 count= 50 avg_used= 7,936 block_cnt= 50 chunk_cnt= 29 mod=LinearHashMapNo [MEMORY] hold= 399,200 used= 389,600 count= 50 avg_used= 7,792 block_cnt= 50 chunk_cnt= 32 mod=CkptDgnCUNode [MEMORY] hold= 385,024 used= 377,120 count= 1 avg_used= 377,120 block_cnt= 1 chunk_cnt= 1 mod=ClogGe [MEMORY] hold= 338,112 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 44 chunk_cnt= 2 mod=LSMap [MEMORY] hold= 311,296 used= 262,144 count= 256 avg_used= 1,024 block_cnt= 44 chunk_cnt= 1 mod=TabletToLS [MEMORY] hold= 311,104 used= 303,904 count= 6 avg_used= 50,650 block_cnt= 6 chunk_cnt= 2 mod=PoolFreeList [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompStMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=DmlStatHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompactMap [MEMORY] hold= 294,912 used= 261,632 count= 4 avg_used= 65,408 block_cnt= 4 chunk_cnt= 3 mod=MemTblObj [MEMORY] hold= 268,224 used= 261,888 count= 33 avg_used= 7,936 block_cnt= 33 chunk_cnt= 18 mod=CtxBlockPl [MEMORY] hold= 260,224 used= 253,952 count= 31 avg_used= 8,192 block_cnt= 31 chunk_cnt= 22 mod=ParamCharsType [MEMORY] hold= 212,992 used= 212,160 count= 1 avg_used= 212,160 block_cnt= 1 chunk_cnt= 1 mod=MetaMemMgr [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 19 mod=QueryAllocator [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 20 mod=SqlSession [MEMORY] hold= 204,800 used= 196,744 count= 1 avg_used= 196,744 block_cnt= 1 chunk_cnt= 1 mod=DagNetMap [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=T3MBucket [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=ResourMapLock [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=TenDLBucket [MEMORY] hold= 163,776 used= 156,741 count= 2 avg_used= 78,370 block_cnt= 2 chunk_cnt= 2 mod=CsSstableReader [MEMORY] hold= 147,456 used= 139,264 count= 1 avg_used= 139,264 block_cnt= 1 chunk_cnt= 1 mod=RFLTaskQueue [MEMORY] hold= 147,456 used= 138,240 count= 2 avg_used= 69,120 block_cnt= 2 chunk_cnt= 1 mod=LDBlockBitMap [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 11 mod=RawPl [MEMORY] hold= 128,576 used= 33,168 count= 471 avg_used= 70 block_cnt= 175 chunk_cnt= 33 mod=Coro [MEMORY] hold= 128,000 used= 124,928 count= 16 avg_used= 7,808 block_cnt= 16 chunk_cnt= 2 mod=TabletIDSetNode [MEMORY] hold= 122,880 used= 110,608 count= 2 avg_used= 55,304 block_cnt= 2 chunk_cnt= 1 mod=TabletIDSetBkt [MEMORY] hold= 114,688 used= 106,496 count= 1 avg_used= 106,496 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlMgr [MEMORY] hold= 114,688 used= 111,096 count= 1 avg_used= 111,096 block_cnt= 1 chunk_cnt= 1 mod=AsyncFreezeTabl [MEMORY] hold= 106,496 used= 98,432 count= 1 avg_used= 98,432 block_cnt= 1 chunk_cnt= 1 mod=tenant_mf_mgr [MEMORY] hold= 106,496 used= 98,528 count= 1 avg_used= 98,528 block_cnt= 1 chunk_cnt= 1 mod=LSSvr [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=PlanVaIdx [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 10 mod=SqlSessiVarMap [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=DictPl [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=RlePl [MEMORY] hold= 81,920 used= 73,896 count= 1 avg_used= 73,896 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairCache [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 10 mod=ConstPl [MEMORY] hold= 77,824 used= 65,536 count= 64 avg_used= 1,024 block_cnt= 12 chunk_cnt= 3 mod=TxCtxMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=DEVICE_MANAGER [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=StorMetaCacheIO [MEMORY] hold= 73,728 used= 66,176 count= 1 avg_used= 66,176 block_cnt= 1 chunk_cnt= 1 mod=Rpc [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxDataMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxCtxMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LockMemObj [MEMORY] hold= 73,728 used= 72,736 count= 1 avg_used= 72,736 block_cnt= 1 chunk_cnt= 1 mod=LogSharedQueueT [MEMORY] hold= 73,728 used= 65,536 count= 1 avg_used= 65,536 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushLog [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushMeta [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 9 mod=IntDiffPl [MEMORY] hold= 72,240 used= 1,024 count= 896 avg_used= 1 block_cnt= 10 chunk_cnt= 1 mod=CharsetUtil [MEMORY] hold= 65,216 used= 60,712 count= 4 avg_used= 15,178 block_cnt= 4 chunk_cnt= 4 mod=HB_SERVICE [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 8 mod=MdsTableMgr [MEMORY] hold= 64,000 used= 62,464 count= 8 avg_used= 7,808 block_cnt= 8 chunk_cnt= 8 mod=HashNodeLCSta [MEMORY] hold= 58,400 used= 55,296 count= 16 avg_used= 3,456 block_cnt= 11 chunk_cnt= 8 mod=CACHE_INST [MEMORY] hold= 57,344 used= 50,496 count= 1 avg_used= 50,496 block_cnt= 1 chunk_cnt= 1 mod=palf_env [MEMORY] hold= 54,000 used= 5,760 count= 240 avg_used= 24 block_cnt= 146 chunk_cnt= 27 mod=[T]MemoryContex [MEMORY] hold= 48,960 used= 44,840 count= 2 avg_used= 22,420 block_cnt= 2 chunk_cnt= 2 mod=DynamicFactor [MEMORY] hold= 48,880 used= 44,760 count= 2 avg_used= 22,380 block_cnt= 2 chunk_cnt= 2 mod=Autoincrement [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 4 mod=HexStrPl [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 5 mod=ColEqualPl [MEMORY] hold= 45,792 used= 35,616 count= 53 avg_used= 672 block_cnt= 51 chunk_cnt= 23 mod=[T]ObWarningBuf [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DagWarnHisBkt [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 1 mod=ObDMMDL [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=StrDiffPl [MEMORY] hold= 32,912 used= 22,464 count= 54 avg_used= 416 block_cnt= 23 chunk_cnt= 7 mod=OMT_Worker [MEMORY] hold= 32,768 used= 17,408 count= 2 avg_used= 8,704 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTTP [MEMORY] hold= 32,768 used= 25,664 count= 1 avg_used= 25,664 block_cnt= 1 chunk_cnt= 1 mod=TSQLSessionMgr [MEMORY] hold= 32,768 used= 24,688 count= 2 avg_used= 12,344 block_cnt= 2 chunk_cnt= 2 mod=TLD_AssignedMgr [MEMORY] hold= 32,704 used= 26,079 count= 2 avg_used= 13,039 block_cnt= 2 chunk_cnt= 2 mod=CGReadInfoMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 3 mod=StrPrefixPl [MEMORY] hold= 32,000 used= 31,232 count= 4 avg_used= 7,808 block_cnt= 4 chunk_cnt= 4 mod=HashNodePlanCac [MEMORY] hold= 24,576 used= 16,384 count= 1 avg_used= 16,384 block_cnt= 1 chunk_cnt= 1 mod=SlogWriteBuffer [MEMORY] hold= 24,576 used= 22,592 count= 1 avg_used= 22,592 block_cnt= 1 chunk_cnt= 1 mod=IO_MGR [MEMORY] hold= 24,368 used= 20,136 count= 2 avg_used= 10,068 block_cnt= 2 chunk_cnt= 2 mod=TabletStats [MEMORY] hold= 24,336 used= 20,104 count= 2 avg_used= 10,052 block_cnt= 2 chunk_cnt= 2 mod=TLD_ResourceMgr [MEMORY] hold= 21,152 used= 16,896 count= 22 avg_used= 768 block_cnt= 15 chunk_cnt= 12 mod=TGTimer [MEMORY] hold= 17,056 used= 8,448 count= 3 avg_used= 2,816 block_cnt= 2 chunk_cnt= 2 mod=BaseLogWriter [MEMORY] hold= 16,384 used= 13,824 count= 1 avg_used= 13,824 block_cnt= 1 chunk_cnt= 1 mod=LogReplayStatus [MEMORY] hold= 16,384 used= 11,584 count= 1 avg_used= 11,584 block_cnt= 1 chunk_cnt= 1 mod=LogApplyStatus [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=SlogNopLog [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MedCheckSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_ClientTask [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_TableCtxMgr [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=DiaTabletMap [MEMORY] hold= 16,384 used= 8,448 count= 1 avg_used= 8,448 block_cnt= 1 chunk_cnt= 1 mod=PalfEnv [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TabletSvrBucket [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=ColSubStrPl [MEMORY] hold= 15,648 used= 15,264 count= 2 avg_used= 7,632 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoNode [MEMORY] hold= 11,984 used= 11,592 count= 2 avg_used= 5,796 block_cnt= 2 chunk_cnt= 1 mod=StopWordSet [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=Scheduler [MEMORY] hold= 9,376 used= 8,984 count= 2 avg_used= 4,492 block_cnt= 2 chunk_cnt= 1 mod=FTPluginMap [MEMORY] hold= 9,248 used= 6,732 count= 13 avg_used= 517 block_cnt= 13 chunk_cnt= 12 mod=SqlSessiQuerSql [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapCn [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapDi [MEMORY] hold= 8,192 used= 7,760 count= 2 avg_used= 3,880 block_cnt= 1 chunk_cnt= 1 mod=HashBucPsSessIn [MEMORY] hold= 8,160 used= 5,512 count= 13 avg_used= 424 block_cnt= 12 chunk_cnt= 7 mod=PackStateMap [MEMORY] hold= 8,144 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 9 mod=SequenceIdMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=PurgeThrottle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ls_id [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FetchLogTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LsElectRefInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LockWaitMgr [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=APPLY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=REPLAY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSIter [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MatchLsnMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MdsDebugMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=HandleSubmit [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=IORunners [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LCLSender [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairs [MEMORY] hold= 8,128 used= 5,512 count= 13 avg_used= 424 block_cnt= 12 chunk_cnt= 8 mod=SequenceMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompCkmItems [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ShareBlksMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FTPluginHandle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=DASIDCache [MEMORY] hold= 8,112 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 8 mod=ContextsMap [MEMORY] hold= 8,096 used= 7,904 count= 1 avg_used= 7,904 block_cnt= 1 chunk_cnt= 1 mod=PsSessionInfo [MEMORY] hold= 8,000 used= 7,808 count= 1 avg_used= 7,808 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsInfo [MEMORY] hold= 7,984 used= 7,792 count= 1 avg_used= 7,792 block_cnt= 1 chunk_cnt= 1 mod=ColUsagNode [MEMORY] hold= 7,920 used= 7,728 count= 1 avg_used= 7,728 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsCache [MEMORY] hold= 7,904 used= 7,712 count= 1 avg_used= 7,712 block_cnt= 1 chunk_cnt= 1 mod=DmlStatNode [MEMORY] hold= 7,824 used= 7,632 count= 1 avg_used= 7,632 block_cnt= 1 chunk_cnt= 1 mod=CheckMapNode [MEMORY] hold= 7,136 used= 66 count= 33 avg_used= 2 block_cnt= 31 chunk_cnt= 11 mod=[T]TSIUseWeak [MEMORY] hold= 6,800 used= 6,216 count= 3 avg_used= 2,072 block_cnt= 2 chunk_cnt= 2 mod=DetectManager [MEMORY] hold= 5,056 used= 4,656 count= 2 avg_used= 2,328 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResGrpIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResRuleIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=PxPoolBkt [MEMORY] hold= 4,288 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 4,160 used= 3,968 count= 1 avg_used= 3,968 block_cnt= 1 chunk_cnt= 1 mod=TinyAllocator [MEMORY] hold= 3,328 used= 256 count= 16 avg_used= 16 block_cnt= 16 chunk_cnt= 14 mod=[T]TSILastOper [MEMORY] hold= 3,328 used= 1,792 count= 8 avg_used= 224 block_cnt= 5 chunk_cnt= 2 mod=LogIOTask [MEMORY] hold= 3,200 used= 3,008 count= 1 avg_used= 3,008 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlDfc [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=IRMMemHashBuck [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=MedCheckMap [MEMORY] hold= 2,320 used= 1,536 count= 4 avg_used= 384 block_cnt= 4 chunk_cnt= 3 mod=TableLockNode [MEMORY] hold= 2,048 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=LogIOWS [MEMORY] hold= 2,000 used= 1,800 count= 1 avg_used= 1,800 block_cnt= 1 chunk_cnt= 1 mod=PxResMgr [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=MSTXCTX [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TXBFRetryId [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TabStatMgrLock [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GROUP_INDEX_MAP [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GCMemtableMap [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=DUP_LS_SET [MEMORY] hold= 1,344 used= 960 count= 2 avg_used= 480 block_cnt= 1 chunk_cnt= 1 mod=PoolArenaArray [MEMORY] hold= 1,280 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=ModuleInitCtx [MEMORY] hold= 1,248 used= 1,056 count= 1 avg_used= 1,056 block_cnt= 1 chunk_cnt= 1 mod=LOG_HASH_MAP [MEMORY] hold= 1,216 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=LockMemtableMgr [MEMORY] hold= 1,152 used= 960 count= 1 avg_used= 960 block_cnt= 1 chunk_cnt= 1 mod=FTParserMgr [MEMORY] hold= 1,120 used= 120 count= 5 avg_used= 24 block_cnt= 2 chunk_cnt= 1 mod=FreezeTask [MEMORY] hold= 976 used= 776 count= 1 avg_used= 776 block_cnt= 1 chunk_cnt= 1 mod=DepTabletSet [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=CheckPointTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=FlushTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TLD_TIMER [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TabShellTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=DiskUsageTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=tenant_rm_mgr [MEMORY] hold= 848 used= 648 count= 1 avg_used= 648 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfo [MEMORY] hold= 752 used= 552 count= 1 avg_used= 552 block_cnt= 1 chunk_cnt= 1 mod=LSFreeze [MEMORY] hold= 672 used= 288 count= 2 avg_used= 144 block_cnt= 2 chunk_cnt= 2 mod=ObTxLSLogCb [MEMORY] hold= 672 used= 472 count= 1 avg_used= 472 block_cnt= 1 chunk_cnt= 1 mod=Election [MEMORY] hold= 576 used= 384 count= 1 avg_used= 384 block_cnt= 1 chunk_cnt= 1 mod=HAScheduler [MEMORY] hold= 544 used= 144 count= 2 avg_used= 72 block_cnt= 1 chunk_cnt= 1 mod=TntSrvObjPool [MEMORY] hold= 512 used= 120 count= 2 avg_used= 60 block_cnt= 2 chunk_cnt= 2 mod=UserResourceMgr [MEMORY] hold= 448 used= 64 count= 2 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfFSCbNode [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTHandler [MEMORY] hold= 400 used= 208 count= 1 avg_used= 208 block_cnt= 1 chunk_cnt= 1 mod=[T]ObMacroBlock [MEMORY] hold= 320 used= 102 count= 1 avg_used= 102 block_cnt= 1 chunk_cnt= 1 mod=TxLSLogBuf [MEMORY] hold= 304 used= 112 count= 1 avg_used= 112 block_cnt= 1 chunk_cnt= 1 mod=Coordinator [MEMORY] hold= 304 used= 104 count= 1 avg_used= 104 block_cnt= 1 chunk_cnt= 1 mod=RedoLogBuckLock [MEMORY] hold= 256 used= 48 count= 1 avg_used= 48 block_cnt= 1 chunk_cnt= 1 mod=[T]TSITabletTab [MEMORY] hold= 256 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfoMap [MEMORY] hold= 256 used= 64 count= 1 avg_used= 64 block_cnt= 1 chunk_cnt= 1 mod=NegParamIdex [MEMORY] hold= 240 used= 41 count= 1 avg_used= 41 block_cnt= 1 chunk_cnt= 1 mod=KeepAliveBuf [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfRCCbNode [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=RebuildCbNode [MEMORY] hold= 208 used= 16 count= 1 avg_used= 16 block_cnt= 1 chunk_cnt= 1 mod=logservice [MEMORY] hold= 422,562,304 used= 405,378,852 count= 33,075 avg_used= 12,256 mod=SUMMARY [2024-10-16 14:57:26.382800] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=95] [MEMORY] tenant_id= 1 ctx_id= MEMSTORE_CTX_ID hold= 245,366,784 used= 243,449,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 block_cnt= 117 chunk_cnt= 117 mod=Memstore [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 mod=SUMMARY [2024-10-16 14:57:26.382814] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= TRANS_CTX_MGR_ID hold= 2,097,152 used= 729,088 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 block_cnt= 1 chunk_cnt= 1 mod=PartTranCtxMgr [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 mod=SUMMARY [2024-10-16 14:57:26.382839] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=9] [MEMORY] tenant_id= 1 ctx_id= PLAN_CACHE_CTX_ID hold= 79,691,776 used= 73,331,136 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 68,663,616 used= 62,143,518 count= 2,832 avg_used= 21,943 block_cnt= 2,832 chunk_cnt= 37 mod=SqlPhyPlan [MEMORY] hold= 3,414,336 used= 3,146,720 count= 326 avg_used= 9,652 block_cnt= 326 chunk_cnt= 37 mod=SqlPlanCache [MEMORY] hold= 1,251,200 used= 1,140,800 count= 575 avg_used= 1,984 block_cnt= 192 chunk_cnt= 37 mod=CreateContext [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 73,331,136 used= 66,432,638 count= 3,735 avg_used= 17,786 mod=SUMMARY [2024-10-16 14:57:26.382860] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=10] [MEMORY] tenant_id= 1 ctx_id= GLIBC hold= 2,097,152 used= 81,040 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 80,944 used= 59,018 count= 263 avg_used= 224 block_cnt= 20 chunk_cnt= 1 mod=PlJit [MEMORY] hold= 96 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PlCodeGen [MEMORY] hold= 81,040 used= 59,050 count= 264 avg_used= 223 mod=SUMMARY [2024-10-16 14:57:26.382875] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=9] [MEMORY] tenant_id= 1 ctx_id= CO_STACK hold= 136,314,880 used= 133,668,864 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 block_cnt= 259 chunk_cnt= 65 mod=CoStack [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 mod=SUMMARY [2024-10-16 14:57:26.382891] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= KVSTORE_CACHE_ID hold= 861,929,472 used= 861,929,472 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 mod=SUMMARY [2024-10-16 14:57:26.382906] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=10] [MEMORY] tenant_id= 1 ctx_id= META_OBJ_CTX_ID hold= 37,748,736 used= 36,290,496 limit= 429,496,720 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 29,491,200 used= 26,236,400 count= 400 avg_used= 65,591 block_cnt= 400 chunk_cnt= 18 mod=L_TabletPool [MEMORY] hold= 6,709,248 used= 5,952,128 count= 91 avg_used= 65,408 block_cnt= 91 chunk_cnt= 15 mod=N_TabletPool [MEMORY] hold= 90,048 used= 84,768 count= 2 avg_used= 42,384 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 36,290,496 used= 32,273,296 count= 493 avg_used= 65,463 mod=SUMMARY [2024-10-16 14:57:26.382917] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= TX_CALLBACK_CTX_ID hold= 12,582,912 used= 6,234,176 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 block_cnt= 767 chunk_cnt= 6 mod=MemtableCallbac [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:26.382934] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=8] [MEMORY] tenant_id= 1 ctx_id= LOB_CTX_ID hold= 2,097,152 used= 81,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LobAllocator [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 81,856 used= 73,344 count= 2 avg_used= 36,672 mod=SUMMARY [2024-10-16 14:57:26.382942] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= PS_CACHE_CTX_ID hold= 2,097,152 used= 16,288 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 block_cnt= 4 chunk_cnt= 1 mod=SqlPsCache [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 mod=SUMMARY [2024-10-16 14:57:26.382959] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= TX_DATA_TABLE hold= 54,546,432 used= 52,755,200 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 49,564,544 used= 48,393,728 count= 6,098 avg_used= 7,936 block_cnt= 6,098 chunk_cnt= 25 mod=TX_DATA_SLICE [MEMORY] hold= 3,182,528 used= 3,153,728 count= 3 avg_used= 1,051,242 block_cnt= 3 chunk_cnt= 3 mod=MEMTABLE_ARENA [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 52,755,200 used= 51,555,392 count= 6,102 avg_used= 8,448 mod=SUMMARY [2024-10-16 14:57:26.382974] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= MDS_DATA_ID hold= 4,194,304 used= 398,272 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 260,096 used= 253,952 count= 32 avg_used= 7,936 block_cnt= 32 chunk_cnt= 2 mod=MdsTable [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 2 mod=TX_OP [MEMORY] hold= 398,272 used= 388,864 count= 49 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:26.382986] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= SCHEMA_SERVICE hold= 2,097,152 used= 8,128 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=NonPartTenCac [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:26.383005] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,082,318,848 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 371,380,224 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 14,680,064 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 104,857,600 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 20,971,520 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= RPC_CTX_ID hold_bytes= 6,291,456 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 26,357,760 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 237,785,088 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= UNEXPECTED_IN_500 hold_bytes= 295,800,832 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:26.383222] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 371,380,224 used= 311,732,240 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 66,584,784 used= 66,551,824 count= 33 avg_used= 2,016,721 block_cnt= 33 chunk_cnt= 33 mod=Compressor [MEMORY] hold= 31,477,760 used= 31,458,304 count= 1 avg_used= 31,458,304 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 23,584,768 used= 23,499,200 count= 9 avg_used= 2,611,022 block_cnt= 9 chunk_cnt= 8 mod=IoControl [MEMORY] hold= 15,925,248 used= 15,247,872 count= 216 avg_used= 70,592 block_cnt= 216 chunk_cnt= 42 mod=[T]ObSessionDIB [MEMORY] hold= 12,873,728 used= 12,856,352 count= 1 avg_used= 12,856,352 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 9,457,664 used= 9,437,784 count= 1 avg_used= 9,437,784 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 5,505,024 used= 5,238,784 count= 32 avg_used= 163,712 block_cnt= 32 chunk_cnt= 22 mod=Iteratorid()=1002, group_id_=1, token=0) [2024-10-16 14:57:26.447504] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107004][T1002_L0_G1][T1002][Y0-0000000000000000-0-0] [lt=15] worker thread exit(tenant_->id()=1002, workers_.get_size()=2) [2024-10-16 14:57:26.447522] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107004][T1002_L0_G1][T0][Y0-0000000000000000-0-0] [lt=13] tenant thread end_run(id_=1002, ret=0, thread_count_=190) [2024-10-16 14:57:26.454817] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A70-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:26.454841] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A70-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:26.455588] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1729061846453784000, start_id=1729061846453784000, end_id=1729061846453784001) [2024-10-16 14:57:26.461712] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:57:26.461728] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:26.461735] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:26.461739] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:26.464007] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:57:26.464146] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=136] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:26.465079] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061846463968000, start_id=1729061846463968000, end_id=1729061846463968001) [2024-10-16 14:57:26.466418] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E73-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497262, log_task={header:{begin_lsn:{lsn:207589528706}, end_lsn:{lsn:207589528828}, log_id:218497262, min_scn:{val:1729061846130310002, v:0}, max_scn:{val:1729061846130310002, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589528584}, prev_proposal_id:81, committed_end_lsn:{lsn:207589528706}, data_checksum:940009244, accum_checksum:4100204612, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061846365989, freeze_ts:1729061846365990, submit_ts:1729061846365994, flushed_ts:1729061846366802, gen_to_freeze cost time:1, gen_to_submit cost time:5, submit_to_flush cost time:808}, fs_cb_begin_ts=1729061846466412, log_life_time=100423) [2024-10-16 14:57:26.468092] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C41-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:26.468114] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C41-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061846129257003, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061846129257003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:26.469495] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C41-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061846468451000, start_id=1729061846468451000, end_id=1729061846468451001) [2024-10-16 14:57:26.470611] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=8] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=1918877, replace_num=31457) [2024-10-16 14:57:26.470628] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:26.472992] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C41-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061846472810000, start_id=1729061846472810000, end_id=1729061846472810001) [2024-10-16 14:57:26.477967] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.500835] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.520159] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061846519305000, start_id=1729061846519305000, end_id=1729061846519305001) [2024-10-16 14:57:26.520183] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=19] ts response task statistics(alloc_count=261363826, free_count=130681912) [2024-10-16 14:57:26.520194] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:26.531997] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:26.532011] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:26.532027] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444444726, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:26.533173] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:26.542326] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:26.551946] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE764B-0-0] [lt=31] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=14, avg_log_batch_cnt=0, total_group_log_size=7714, avg_group_log_size=551) [2024-10-16 14:57:26.557560] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061846556553000, start_id=1729061846556553000, end_id=1729061846556553001) [2024-10-16 14:57:26.557640] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:26.557657] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:26.568749] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22697][T1004_TntTransf][T1004][YB4A0A000407-00061E9ABD5C8935-0-0] [lt=17] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:57:26.569692] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.570664] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.570681] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.570693] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.575034] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:14538469733}], scn_array=[cnt:1, {val:1729061846129257004, v:0}], curr_size=122, accum_size=976, time_cost=11524) [2024-10-16 14:57:26.575198] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:26.575209] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:26.578029] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.581896] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1002}, {id:1001}]) [2024-10-16 14:57:26.586519] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:26.586535] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:26.597862] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3193, avg_group_log_size=290) [2024-10-16 14:57:26.598143] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=222, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:26.598159] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=43, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:26.598245] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4677, mem_used=8318976) [2024-10-16 14:57:26.599608] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061846598404000, start_id=1729061846598404000, end_id=1729061846598404001) [2024-10-16 14:57:26.600954] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.616815] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061846615375000, start_id=1729061846615375000, end_id=1729061846615375001) [2024-10-16 14:57:26.629970] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BC0-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061846629776000, start_id=1729061846629776000, end_id=1729061846629776001) [2024-10-16 14:57:26.630168] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BC1-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061846629776000, start_id=1729061846629776000, end_id=1729061846629776001) [2024-10-16 14:57:26.630231] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BC3-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061846629776000, start_id=1729061846629776000, end_id=1729061846629776001) [2024-10-16 14:57:26.630397] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BC2-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061846629776000, start_id=1729061846629776000, end_id=1729061846629776001) [2024-10-16 14:57:26.630707] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BBF-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061846629776000, start_id=1729061846629776000, end_id=1729061846629776001) [2024-10-16 14:57:26.633224] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCAF-0-0] [lt=21] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=633, avg_log_submit_to_slide_time=803) [2024-10-16 14:57:26.648822] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21660][T1_L0_G0][T1][YB4A0A000407-00061E9978AD3A2F-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510314517215, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410521, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:26.649417] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [62903][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5E-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:26.649441] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [62903][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5E-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061846129257004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061846129257004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:26.654003] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:26.655063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D5E-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061846654563000, start_id=1729061846654563000, end_id=1729061846654563001) [2024-10-16 14:57:26.656572] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061846654563000, start_id=1729061846654563000, end_id=1729061846654563001) [2024-10-16 14:57:26.657866] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CA8-0-0] [lt=20][errcode=-4201] hashtable not init [2024-10-16 14:57:26.657877] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CA8-0-0] [lt=10][errcode=-4016] get cursor info failed(cursor_id=1110, get_sessid()=3221565766) [2024-10-16 14:57:26.657952] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061846654563000, start_id=1729061846654563000, end_id=1729061846654563001) [2024-10-16 14:57:26.665593] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061846663822000, start_id=1729061846663822000, end_id=1729061846663822001) [2024-10-16 14:57:26.666053] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCB0-0-0] [lt=30] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:26.666065] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCB0-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21625, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:26.666070] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCB0-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=1663, avg_log_submit_to_slide_time=21619) [2024-10-16 14:57:26.666096] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:26.666775] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E77-0-0] [lt=24] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:26.671201] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:26.673984] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:57:26.673994] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:57:26.673998] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:26.674744] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061843821398000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860098662000, latest_log_ts:{val:1729061843439703001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:26.674845] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=60] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:26.674854] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:26.675455] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061843821398000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:26.676126] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=28] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061843821398000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:26.676809] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=35] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061843821398000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:26.678089] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.678354] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=484, avg_log_submit_to_slide_time=1149) [2024-10-16 14:57:26.696310] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:26.696919] INFO [STORAGE] runTimerTask (ob_locality_manager.cpp:708) [21260][LocalityReload][T0][Y0-0000000000000000-0-0] [lt=11] runTimer to refresh locality_info(ret=0) [2024-10-16 14:57:26.697954] INFO [SHARE] set_server_locality_array (ob_server_locality_cache.cpp:346) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=14] set_server_locality_array success(server_locality_array=[{inited:true, addr:"10.0.4.7:2890", zone:"myoceanbase1", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494256558000, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.8:2890", zone:"myoceanbase2", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494430723273, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.9:2890", zone:"myoceanbase3", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494593087951, server_stop_time:0, server_status:1}], has_readonly_zone=false) [2024-10-16 14:57:26.697981] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=21] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1, primary_zone=RANDOM) [2024-10-16 14:57:26.697986] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=5] add_locality_zone success(locality_zone={tenant_id:1, region_priority:18446744073709551615}) [2024-10-16 14:57:26.698001] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=5] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1001, primary_zone=RANDOM) [2024-10-16 14:57:26.698004] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=4] add_locality_zone success(locality_zone={tenant_id:1001, region_priority:18446744073709551615}) [2024-10-16 14:57:26.698011] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1002, primary_zone=RANDOM) [2024-10-16 14:57:26.698014] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] add_locality_zone success(locality_zone={tenant_id:1002, region_priority:18446744073709551615}) [2024-10-16 14:57:26.698018] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1003, primary_zone=RANDOM) [2024-10-16 14:57:26.698020] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1003, region_priority:18446744073709551615}) [2024-10-16 14:57:26.698024] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1004, primary_zone=RANDOM) [2024-10-16 14:57:26.698026] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1004, region_priority:18446744073709551615}) [2024-10-16 14:57:26.698031] INFO [SHARE] load_region (ob_locality_table_operator.cpp:373) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] load region(ret=0, locality_info={version:0, local_region:"sys_region", local_zone:"myoceanbase1", local_idc:"", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["myoceanbase1", "myoceanbase2", "myoceanbase3"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:18446744073709551615}, {tenant_id:1002, region_priority:18446744073709551615}, {tenant_id:1003, region_priority:18446744073709551615}, {tenant_id:1004, region_priority:18446744073709551615}]}) [2024-10-16 14:57:26.701443] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.722955] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:26.726080] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CA6-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115172999, log_task={header:{begin_lsn:{lsn:14352356090}, end_lsn:{lsn:14352356212}, log_id:115172999, min_scn:{val:1729061846130310004, v:0}, max_scn:{val:1729061846130310004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352355968}, prev_proposal_id:17, committed_end_lsn:{lsn:14352356090}, data_checksum:940009244, accum_checksum:1972330651, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061846625916, freeze_ts:1729061846625917, submit_ts:1729061846625922, flushed_ts:1729061846627851, gen_to_freeze cost time:1, gen_to_submit cost time:6, submit_to_flush cost time:1929}, fs_cb_begin_ts=1729061846726071, log_life_time=100155) [2024-10-16 14:57:26.732457] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:26.732476] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:26.742626] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:26.751648] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE764F-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:26.751664] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE764F-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30060, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:26.752315] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:26.755145] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=2] gc stale ls task succ [2024-10-16 14:57:26.755171] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:26.755179] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:26.757754] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061846757458000, start_id=1729061846757458000, end_id=1729061846757458001) [2024-10-16 14:57:26.764450] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:26.766878] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E79-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:26.766897] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E79-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:26.768528] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=40] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=194, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:26.768548] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=227, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:26.768579] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:26.770386] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:26.771299] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15D2-0-0] [lt=20] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:26.771402] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15D4-0-0] [lt=9] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:26.778500] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.780871] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=17] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:26.787665] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.787693] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.787705] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.787716] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.787800] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.787814] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.789016] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.789030] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.789042] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.789053] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.790792] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:26.801134] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:26.802278] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061846801880000, start_id=1729061846801880000, end_id=1729061846801880001) [2024-10-16 14:57:26.802350] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=26, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:26.802374] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=10423, avg_group_log_size=651) [2024-10-16 14:57:26.802387] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10423) [2024-10-16 14:57:26.802397] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:26.802581] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.803739] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=27] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=406, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:26.803755] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=195, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:26.805060] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:26.806972] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC584-0-0] [lt=21] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:57:26.807648] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC584-0-0] [lt=11] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1002) [2024-10-16 14:57:26.807658] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC584-0-0] [lt=9] no transfer partition job [2024-10-16 14:57:26.807670] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC584-0-0] [lt=10] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:57:26.808898] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC584-0-0] [lt=7] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1002, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:57:26.808932] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC584-0-0] [lt=13] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:57:26.810095] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:500) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC584-0-0] [lt=12][errcode=-4152] get obj error(ret=-4152) [2024-10-16 14:57:26.810132] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC584-0-0] [lt=9] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1001, status:0}], ls_array_=[{tenant_id:1002, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729057174083520, last_statistic_bg_stat_time=1729008957751717, last_statistic_schema_version=1729008953019688, last_statistic_max_transfer_task_id={id:-1}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:57:26.815549] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=17] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:26.816871] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061846815282000, start_id=1729061846815282000, end_id=1729061846815282001) [2024-10-16 14:57:26.823658] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:26.823860] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1002}, {id:1}]) [2024-10-16 14:57:26.824239] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:26.824296] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061844396711000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:26.829249] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD056-0-0] [lt=33] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:26.829262] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD056-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=46059, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:26.832970] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=497, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:26.832985] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=758, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:26.833926] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107054][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=192) [2024-10-16 14:57:26.833944] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107054][][T1004][Y0-0000000000000000-0-0] [lt=16] Init thread local success [2024-10-16 14:57:26.833964] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A35B5986C-0-0] [lt=4] init point thread id with(&point=0x55e3435c3a00, idx_=558, point=[thread id=107054, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107054) [2024-10-16 14:57:26.835307] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107055][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=193) [2024-10-16 14:57:26.835332] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=18] worker thread created(tenant_->id()=1004, group_id_=1, token=1) [2024-10-16 14:57:26.835350] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107055][][T1004][Y0-0000000000000000-0-0] [lt=41] Init thread local success [2024-10-16 14:57:26.844090] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:26.851014] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=23] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3512, mem_used=8318976) [2024-10-16 14:57:26.856789] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061846855378000, start_id=1729061846855378000, end_id=1729061846855378001) [2024-10-16 14:57:26.857161] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10424) [2024-10-16 14:57:26.857740] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10423) [2024-10-16 14:57:26.857793] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93D0-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=6, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:26.857801] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93D0-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=1701, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:26.857804] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93D0-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=720, avg_log_submit_to_slide_time=1667) [2024-10-16 14:57:26.860074] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22703][T1004_TntShared][T1004][Y0-0000000000000000-0-0] [lt=11] clear dtl interm result cost(us)(clear_cost=905, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:26.868274] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=5] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:26.868386] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107055][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B29-0-0] [lt=5] init point thread id with(&point=0x55e3435c3a40, idx_=559, point=[thread id=107055, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107055) [2024-10-16 14:57:26.868841] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E7B-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497266, log_task={header:{begin_lsn:{lsn:207589529194}, end_lsn:{lsn:207589529316}, log_id:218497266, min_scn:{val:1729061846630807001, v:0}, max_scn:{val:1729061846630807001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589529072}, prev_proposal_id:81, committed_end_lsn:{lsn:207589529194}, data_checksum:940009244, accum_checksum:4195324513, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061846766873, freeze_ts:1729061846766873, submit_ts:1729061846766893, flushed_ts:1729061846768558, gen_to_freeze cost time:0, gen_to_submit cost time:20, submit_to_flush cost time:1665}, fs_cb_begin_ts=1729061846868834, log_life_time=101961) [2024-10-16 14:57:26.868984] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:26.869034] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061841826930000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:26.870077] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061846865304000, start_id=1729061846865304000, end_id=1729061846865304001) [2024-10-16 14:57:26.870391] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:26.871264] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:26.878684] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=48][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.892410] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] new mysql sessid created(conn.sessid_=3221737991, support_ssl=false) [2024-10-16 14:57:26.892447] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] sm conn init succ(conn.sessid_=3221737991, sess.client_addr_="10.0.4.7:53362") [2024-10-16 14:57:26.892455] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] accept one succ(*s={this:0x2b0f27406030, session_id:3221737991, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:26.892612] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=17] username and tenantname(user_name=root, tenant_name=) [2024-10-16 14:57:26.892627] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=6] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:26.892701] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A8-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:26.893174] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A8-0-0] [lt=6] MySQL LOGIN(direct_client_ip="10.0.4.7", client_ip=10.0.4.7, tenant_name=sys, tenant_id=1, user_name=root, host_name=%, sessid=3221737991, proxy_sessid=0, client_sessid=3221737991, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=3908101, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:26.896709] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55C5-0-0] [lt=2][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510314765102, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410522, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:26.900357] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=8] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:26.902658] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.907605] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD3-0-0] [lt=22] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061846875304, keepalive_ts:-1}) [2024-10-16 14:57:26.907619] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD3-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061846875304, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:26.913990] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 152.00; ) [2024-10-16 14:57:26.914011] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:26.914062] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=29] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 180.75; ) [2024-10-16 14:57:26.914070] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:26.914092] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:26.915271] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=19][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:26.920093] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=10] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:26.920111] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=210) [2024-10-16 14:57:26.920852] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:26.920964] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DD6-0-0] [lt=10] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:57:26.926368] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=449, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:26.926385] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:26.926397] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=256, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:26.926430] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=30, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:26.926681] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B8FF7-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=8, avg_log_submit_to_flush_time=727, avg_log_submit_to_slide_time=36969) [2024-10-16 14:57:26.926693] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:26.926707] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656804, miss_cnt:17, cache_read_size:8050146799, hit_ratio:"9.999997286807768004e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:26.939235] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2307, mem_used=6239232) [2024-10-16 14:57:26.942909] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:26.946041] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Cache wash clean map node details(ret=0, clean_node_count=29, clean_time=0, clean_start_pos=566226, clean_num=62914) [2024-10-16 14:57:26.947987] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601328, offset=9608584}, end_cursor:ObLogCursor{file_id=192, log_id=47601329, offset=9608868}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2aa4050, len:632, log_data_len:284, seq:47601328, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:26.949230] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257694, offset=39013838}, end_cursor:ObLogCursor{file_id=197, log_id=49257695, offset=39014121}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b6c050, len:562, log_data_len:283, seq:49257694, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:26.949293] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] [COMPACTION DAG STATUS] (start_time=1729061836948334, end_time=1729061846949291, dag_status={}) [2024-10-16 14:57:26.949308] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.949317] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.949321] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.949325] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.949328] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.949331] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.949335] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.949338] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.949341] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:26.949345] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949352] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949356] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949360] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949364] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949368] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949372] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949376] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949380] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949384] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949388] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949392] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949404] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949407] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949412] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949416] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949419] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949423] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949427] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949431] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949435] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949438] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949442] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949446] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949450] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949454] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949457] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949461] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949465] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949469] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949473] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949477] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949481] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949485] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949489] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949493] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949496] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949500] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949504] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949508] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949512] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949516] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949520] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949523] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949527] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949536] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949540] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949544] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949547] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949551] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949555] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949559] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949562] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949566] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949570] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949574] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:26.949578] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:26.949582] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:26.949586] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:26.949588] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:26.949591] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:26.949593] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:26.949595] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:26.949598] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:26.949605] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:26.949608] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:26.949611] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:26.950919] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA495-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061846948524000, start_id=1729061846948524000, end_id=1729061846948524001) [2024-10-16 14:57:26.951628] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] [APPLY STAT CB APPEND COST TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=46, this=0x2b0e07718280, extra_info="") [2024-10-16 14:57:26.951639] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=2399, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:57:26.951642] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:57:26.951646] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=-559, this=0x2b0e07718910, extra_info="") [2024-10-16 14:57:26.951649] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=1885, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:57:26.951901] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601329, offset=9608868}, end_cursor:ObLogCursor{file_id=192, log_id=47601330, offset=9609142}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a0c050, len:348, log_data_len:274, seq:47601329, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:26.952060] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7653-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:26.952108] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561130, miss_cnt:1, cache_read_size:152590657982, hit_ratio:"9.999999928346812172e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:26.954256] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601330, offset=9609142}, end_cursor:ObLogCursor{file_id=192, log_id=47601331, offset=9609416}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c14050, len:4170, log_data_len:274, seq:47601330, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:26.954418] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601331, offset=9609416}, end_cursor:ObLogCursor{file_id=192, log_id=47601332, offset=9609690}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b34050, len:3896, log_data_len:274, seq:47601331, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:26.954480] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598714, miss_cnt:9, cache_read_size:149777893757, hit_ratio:"9.999999345924162064e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:26.957423] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB360-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061846955376000, start_id=1729061846955376000, end_id=1729061846955376001) [2024-10-16 14:57:26.957867] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBA-0-0] [lt=15] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:26.957883] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBA-0-0] [lt=13] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:26.958490] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061846957824000, start_id=1729061846957824000, end_id=1729061846957824001) [2024-10-16 14:57:26.959065] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1729061846663822000, v:0}, new_max_version={val:1729061846663822000, v:0}, last_min_version={val:1729061846532044001, v:0}, last_max_version={val:1729061846532044001, v:0}, total_time_us=1320, delta_us=295241) [2024-10-16 14:57:26.959084] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1729061846663822000, v:0}, min_version={val:1729061846663822000, v:0}, max_version={val:1729061846663822000, v:0}, in_service=true, leader_epoch=33576) [2024-10-16 14:57:26.959178] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257695, offset=39014121}, end_cursor:ObLogCursor{file_id=197, log_id=49257696, offset=39014403}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c08050, len:4375, log_data_len:282, seq:49257695, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:26.959315] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061846959314, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007618, add_bytes: 0 [2024-10-16 14:57:26.961371] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257696, offset=39014403}, end_cursor:ObLogCursor{file_id=197, log_id=49257697, offset=39014685}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cd0050, len:4093, log_data_len:282, seq:49257696, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:26.961673] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257697, offset=39014685}, end_cursor:ObLogCursor{file_id=197, log_id=49257698, offset=39014967}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cc4050, len:3811, log_data_len:282, seq:49257697, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:26.961733] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650066, miss_cnt:32, cache_read_size:8050399184, hit_ratio:"9.999994892266569346e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:26.962354] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA871-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:26.962373] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA871-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061846630807001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061846630807001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:26.963664] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B2B-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:26.963842] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1375, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:26.963861] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=172, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:26.963895] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:26.964012] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA871-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061846963118000, start_id=1729061846963118000, end_id=1729061846963118001) [2024-10-16 14:57:26.967748] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA871-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061846967325000, start_id=1729061846967325000, end_id=1729061846967325001) [2024-10-16 14:57:26.967811] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:142) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA871-0-0] [lt=12] transaction rpc statistics(total_rt=3550, total_process=39, avg_rt=88) [2024-10-16 14:57:26.967866] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061846967865, bytes: 4446807502848, bw: 0.379426 MB/s, add_ts: 1000642, add_bytes: 398112 [2024-10-16 14:57:26.968401] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:26.974231] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=33] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:26.974247] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:26.974470] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=316, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:26.974478] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:26.974489] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=338, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:26.974513] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:26.974520] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:26.978745] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:26.979336] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:26.979359] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:57:26.979370] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:26.990943] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=9] ====== tenant freeze timer task ====== [2024-10-16 14:57:27.000398] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=26, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:27.000421] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4223) [2024-10-16 14:57:27.000430] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:27.002495] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061847002282000, start_id=1729061847002282000, end_id=1729061847002282001) [2024-10-16 14:57:27.002740] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.009749] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=4] Garbage Collector is running(seq_=656697, gc_interval=10000000) [2024-10-16 14:57:27.009779] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=13] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:27.009790] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=28) [2024-10-16 14:57:27.009796] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:27.010555] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:27.010570] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=11] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:27.010576] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] execute_gc cost time(ret=0, time_us=7) [2024-10-16 14:57:27.010580] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:27.016981] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061847015769000, start_id=1729061847015769000, end_id=1729061847015769001) [2024-10-16 14:57:27.020526] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061847020316000, start_id=1729061847020316000, end_id=1729061847020316001) [2024-10-16 14:57:27.020558] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061847020555], gts:1729061847020316000, latest_srr:[mts=1729061847020555]}) [2024-10-16 14:57:27.023839] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F21-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061847023497000, start_id=1729061847023497000, end_id=1729061847023497001) [2024-10-16 14:57:27.032838] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:27.032859] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:27.033387] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD05D-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:27.033639] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD05E-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:27.036901] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:27.036909] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:27.036921] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=291869210, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:27.037394] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:27.037411] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=152203660, tenant_id=1) [2024-10-16 14:57:27.037423] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:27.055978] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=752, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:27.055995] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=700, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:27.056030] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:27.056762] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061847056312000, start_id=1729061847056312000, end_id=1729061847056312001) [2024-10-16 14:57:27.057737] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=246, start_service_tstamp_=1722494263598349, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:27.065581] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=12][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:27.070097] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061847069089000, start_id=1729061847069089000, end_id=1729061847069089001) [2024-10-16 14:57:27.071338] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:27.078813] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.083163] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480A-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061847050882, keepalive_ts:-1}) [2024-10-16 14:57:27.083186] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480A-0-0] [lt=21] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061847050882, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:27.102800] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.110504] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:27.110519] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:27.123792] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:27.128259] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=753, mem_used=2079744) [2024-10-16 14:57:27.130247] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BC5-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061847125566000, start_id=1729061847125566000, end_id=1729061847125566001) [2024-10-16 14:57:27.130338] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BC7-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061847125566000, start_id=1729061847125566000, end_id=1729061847125566001) [2024-10-16 14:57:27.130411] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BC8-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061847125566000, start_id=1729061847125566000, end_id=1729061847125566001) [2024-10-16 14:57:27.130794] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26170][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BC4-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061847125566000, start_id=1729061847125566000, end_id=1729061847125566001) [2024-10-16 14:57:27.133198] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B939-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061847125566000, start_id=1729061847125566001, end_id=1729061847125566002) [2024-10-16 14:57:27.133562] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B93B-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061847125566000, start_id=1729061847125566000, end_id=1729061847125566001) [2024-10-16 14:57:27.135914] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:27.148834] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21662][T1_L0_G0][T1][YB4A0A000407-00061E9978ED0934-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510315017227, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410523, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:27.150003] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:27.151957] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7657-0-0] [lt=16][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938122, log_task={header:{begin_lsn:{lsn:213058218122}, end_lsn:{lsn:213058218244}, log_id:220938122, min_scn:{val:1729061846629776004, v:0}, max_scn:{val:1729061846629776004, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058218000}, prev_proposal_id:88, committed_end_lsn:{lsn:213058218122}, data_checksum:1715074492, accum_checksum:3983134436, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061847051826, freeze_ts:1729061847051826, submit_ts:1729061847051831, flushed_ts:1729061847056006, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:4175}, fs_cb_begin_ts=1729061847151950, log_life_time=100124) [2024-10-16 14:57:27.152325] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=77, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:27.152337] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:27.158901] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061847158590000, start_id=1729061847158590000, end_id=1729061847158590001) [2024-10-16 14:57:27.159001] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10209, avg_group_log_size=600) [2024-10-16 14:57:27.159013] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10209) [2024-10-16 14:57:27.159438] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=259, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:27.159453] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10209) [2024-10-16 14:57:27.159471] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=589, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:27.159678] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10209) [2024-10-16 14:57:27.159697] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:27.159805] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD03D-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:27.159813] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD03D-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1382, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:27.159818] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD03D-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=885, avg_log_submit_to_slide_time=1348) [2024-10-16 14:57:27.163846] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B2F-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:27.163867] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B2F-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:27.169427] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5285-0-0] [lt=27] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:27.169442] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5285-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=36888, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:27.173661] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501D-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:27.173678] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501D-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:27.174368] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:27.174381] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:27.174851] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4137-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=753, avg_log_submit_to_slide_time=1215) [2024-10-16 14:57:27.178883] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.196554] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:27.196703] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D82-0-0] [lt=16] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061857008855000, latest_log_ts:{val:1729061840488170001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:27.196763] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D82-0-0] [lt=56] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:27.197208] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=48] success to update min reserved snapshot(reserved_snapshot=1729060044396711000, duration=1800, snapshot_gc_ts=1729061844396711000) [2024-10-16 14:57:27.197532] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4213) [2024-10-16 14:57:27.197579] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4223) [2024-10-16 14:57:27.198086] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F8C-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:27.198098] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F8C-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1167, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:27.198265] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=13] update reserved snapshot finished(cost_ts=1058, reserved_snapshot=1729060044396711000) [2024-10-16 14:57:27.202376] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061847201755000, start_id=1729061847201755000, end_id=1729061847201755001) [2024-10-16 14:57:27.202856] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.217123] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061847215932000, start_id=1729061847215932000, end_id=1729061847215932001) [2024-10-16 14:57:27.233010] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:27.233059] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=45] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1342) [2024-10-16 14:57:27.233129] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:27.233149] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:27.249884] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=39] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:27.251992] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7659-0-0] [lt=15][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938123, log_task={header:{begin_lsn:{lsn:213058218244}, end_lsn:{lsn:213058218366}, log_id:220938123, min_scn:{val:1729061847136758000, v:0}, max_scn:{val:1729061847136758000, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058218122}, prev_proposal_id:88, committed_end_lsn:{lsn:213058218244}, data_checksum:1715074492, accum_checksum:2795188896, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061847151934, freeze_ts:1729061847151934, submit_ts:1729061847151941, flushed_ts:1729061847152587, gen_to_freeze cost time:0, gen_to_submit cost time:7, submit_to_flush cost time:646}, fs_cb_begin_ts=1729061847251983, log_life_time=100049) [2024-10-16 14:57:27.257401] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061847256466000, start_id=1729061847256466000, end_id=1729061847256466001) [2024-10-16 14:57:27.259300] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21531][T1_TntSharedTim][T1][Y0-0000000000000000-0-0] [lt=12] clear dtl interm result cost(us)(clear_cost=1806, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:27.261132] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD102F-0-0] [lt=8] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:27.270716] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061847269205000, start_id=1729061847269205000, end_id=1729061847269205001) [2024-10-16 14:57:27.270994] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:27.271424] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:27.277136] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:27.277152] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1669, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:27.279873] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.302919] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.313398] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5949, mem_used=12478464) [2024-10-16 14:57:27.321425] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=18] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:27.326186] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=38] success to update min reserved snapshot(reserved_snapshot=1729060038714540000, duration=1800, snapshot_gc_ts=1729061838714540000) [2024-10-16 14:57:27.327599] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=16] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060038714540000) [2024-10-16 14:57:27.327638] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=38] update reserved snapshot finished(cost_ts=1457, reserved_snapshot=1729060038714540000) [2024-10-16 14:57:27.333361] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=404, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:27.333391] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=769, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:27.333413] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:27.333726] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD066-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:27.333738] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD066-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1681, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:27.333743] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD066-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=1311, avg_log_submit_to_slide_time=1662) [2024-10-16 14:57:27.333945] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD067-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:27.333957] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD067-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1632, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:27.350322] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:27.350542] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA6-0-0] [lt=9] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:27.350562] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA6-0-0] [lt=13] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=19) [2024-10-16 14:57:27.351314] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=14] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:27.351329] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:27.351336] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=5] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:27.351339] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:27.351345] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=5] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:27.351349] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17112) [2024-10-16 14:57:27.351979] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE765B-0-0] [lt=35] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:27.352254] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:27.360725] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061847360317000, start_id=1729061847360317000, end_id=1729061847360317001) [2024-10-16 14:57:27.372890] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=8] blacklist_loop exec finished(cost_time=15, is_enabled=true, send_cnt=0) [2024-10-16 14:57:27.374454] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:27.374490] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:27.374667] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=467, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:27.374679] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:27.374693] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=439, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:27.374702] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:27.374738] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=34] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:27.374971] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=78, stat_interval=5000000, avg (count/cost)=41, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:57:27.374979] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=8] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=78, stat_interval=5000000, avg (count/cost)=1172, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:57:27.374982] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=78, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:57:27.374986] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=78, stat_interval=5000000, avg (count/cost)=-568, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:57:27.374989] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=78, stat_interval=5000000, avg (count/cost)=645, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:57:27.379892] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.387965] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:27.395478] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0A-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061847386586, keepalive_ts:-1}) [2024-10-16 14:57:27.395500] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0A-0-0] [lt=20] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061847386586, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:27.395994] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=9] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:27.398593] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B5-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:27.398630] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B5-0-0] [lt=24] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061847125566002, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061847125566002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:27.400231] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9B5-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061847399295000, start_id=1729061847399295000, end_id=1729061847399295001) [2024-10-16 14:57:27.400956] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=12, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:27.401128] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F8F-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=909, avg_log_submit_to_slide_time=1220) [2024-10-16 14:57:27.402358] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF098-0-0] [lt=15] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.139135742187500000e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=306, Throttle Trigger(MB)=184, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.963039971287522121e+03) [2024-10-16 14:57:27.402513] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061847402474000, start_id=1729061847402474000, end_id=1729061847402474001) [2024-10-16 14:57:27.402975] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.404551] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9B5-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061847403530000, start_id=1729061847403530000, end_id=1729061847403530001) [2024-10-16 14:57:27.405360] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:27.417293] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061847416705000, start_id=1729061847416705000, end_id=1729061847416705001) [2024-10-16 14:57:27.427058] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CB4-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:27.427085] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CB4-0-0] [lt=21] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:27.427095] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CB4-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:27.433340] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:27.452596] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=4] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:27.453234] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=32] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:27.453283] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061838714540000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:27.455559] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A71-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:27.455589] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A71-0-0] [lt=29] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:27.457482] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061847457027000, start_id=1729061847457027000, end_id=1729061847457027001) [2024-10-16 14:57:27.464210] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=43845039, this={dio_aligned_buf:{buf_write_offset:1577, buf_padding_size:2519, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35369025, truncate_used_ts:14641021}, log_block_size:67108864, dir_fd:201, io_fd:197}, aligned_buf_len=4096, aligned_block_offset=43843584, buf_len=122, total_write_size_=12060, total_write_size_after_dio_=417792, ob_pwrite_used_ts=62163, count=99, each_pwrite_cost=627) [2024-10-16 14:57:27.470369] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061847470123000, start_id=1729061847470123000, end_id=1729061847470123001) [2024-10-16 14:57:27.471505] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:27.476628] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4141-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:27.476639] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4141-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1310, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:27.479951] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.484002] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C42-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:27.484031] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C42-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061847136758002, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061847136758002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:27.484270] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:27.484279] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:27.484289] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:27.484294] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:27.484303] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:27.484305] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:27.484308] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:27.484310] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:27.484313] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:27.484300] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECD-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:27.484315] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:27.484319] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:27.484320] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECD-0-0] [lt=17] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:27.485835] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C42-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061847484867000, start_id=1729061847484867000, end_id=1729061847484867001) [2024-10-16 14:57:27.488476] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C42-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061847488021000, start_id=1729061847488021000, end_id=1729061847488021001) [2024-10-16 14:57:27.489282] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:27.489298] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:27.489311] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70534782, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:27.491495] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=20) [2024-10-16 14:57:27.503010] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.516975] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=42] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a81850, tx_id:{txid:199664003}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061847515935, active_ts:1729061847515935, commit_ts:-1, finish_ts:1729061847515935, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061877515930, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061847511679, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:27.517802] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=45] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a6bc70, tx_id:{txid:199664004}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061847516993, active_ts:1729061847516993, commit_ts:-1, finish_ts:1729061847516993, timeout_us:29999996, lock_timeout_us:-1, expire_ts:1729061877516989, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061847511679, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:27.520942] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061847520135000, start_id=1729061847520135000, end_id=1729061847520135001) [2024-10-16 14:57:27.520961] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:27.520973] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061847520968], gts:1729061847520135000, latest_srr:[mts=1729061847520968]}) [2024-10-16 14:57:27.539043] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=904810496, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=168931328, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:27.539103] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=26] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:27.539112] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:27.539146] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=969699328, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=104042496, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:27.539189] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:27.539194] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:27.539222] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=940462080, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=133279744, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:27.539255] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:27.539259] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011856, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:27.539288] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=958939136, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=114802688, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:27.539320] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:27.539324] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:27.545681] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCA7D-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061847544672000, start_id=1729061847544672000, end_id=1729061847544672001) [2024-10-16 14:57:27.550628] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:27.560789] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061847560461000, start_id=1729061847560461000, end_id=1729061847560461001) [2024-10-16 14:57:27.560863] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:27.560885] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:27.568194] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.568223] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.569599] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.569622] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.579825] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [22451][T1003_ObXAHbWor][T1003][Y0-0000000000000000-0-0] [lt=23] XA scheduler heartbeat task statistics(avg_time=0) [2024-10-16 14:57:27.580012] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.582000] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1}, {id:1002}]) [2024-10-16 14:57:27.586853] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:27.586871] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:27.598351] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4696, mem_used=8318976) [2024-10-16 14:57:27.600791] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=4213, avg_group_log_size=351) [2024-10-16 14:57:27.601064] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=291, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:27.601083] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=189, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:27.602739] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061847602053000, start_id=1729061847602053000, end_id=1729061847602053001) [2024-10-16 14:57:27.603065] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:27.603087] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:27.603093] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.617479] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061847616970000, start_id=1729061847616970000, end_id=1729061847616970001) [2024-10-16 14:57:27.630664] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BCA-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061847629883000, start_id=1729061847629883000, end_id=1729061847629883001) [2024-10-16 14:57:27.630690] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BCC-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061847629883000, start_id=1729061847629883000, end_id=1729061847629883001) [2024-10-16 14:57:27.631138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BC9-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061847630943000, start_id=1729061847630943000, end_id=1729061847630943001) [2024-10-16 14:57:27.631153] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BCD-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061847630943000, start_id=1729061847630943000, end_id=1729061847630943001) [2024-10-16 14:57:27.633467] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:27.633723] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD06F-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=1092, avg_log_submit_to_slide_time=1520) [2024-10-16 14:57:27.637626] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=16] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:57:27.653112] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=10] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:57:27.653135] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=19] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:27.653142] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:27.653147] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:27.654555] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:27.655295] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=111, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:57:27.655444] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=147] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:27.655473] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:27.657840] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061847657614000, start_id=1729061847657614000, end_id=1729061847657614001) [2024-10-16 14:57:27.660627] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:57:27.661498] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061847657614000, start_id=1729061847657614000, end_id=1729061847657614001) [2024-10-16 14:57:27.666852] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22826][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5F-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:27.666874] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22826][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D5F-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061847125566004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061847125566004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:27.668115] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9006-0-0] [lt=27] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:27.668127] INFO [LIB] stat (utility.h:1165) [107003][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9006-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10456, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:27.668132] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107003][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9006-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=418, avg_log_submit_to_slide_time=10450) [2024-10-16 14:57:27.668159] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:27.668191] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:27.668196] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=5] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:27.669543] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E8B-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:27.670336] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061847670139000, start_id=1729061847670139001, end_id=1729061847670139002) [2024-10-16 14:57:27.671585] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:27.671941] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D5F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061847671213000, start_id=1729061847671213000, end_id=1729061847671213001) [2024-10-16 14:57:27.675225] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:27.675240] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:27.680617] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.692272] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=9] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:27.696338] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:27.703115] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.723060] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=21] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:27.727227] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107003][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CBA-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173009, log_task={header:{begin_lsn:{lsn:14352357310}, end_lsn:{lsn:14352357432}, log_id:115173009, min_scn:{val:1729061847125566005, v:0}, max_scn:{val:1729061847125566005, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352357188}, prev_proposal_id:17, committed_end_lsn:{lsn:14352357310}, data_checksum:940009244, accum_checksum:111165959, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061847627165, freeze_ts:1729061847627165, submit_ts:1729061847627171, flushed_ts:1729061847627831, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:660}, fs_cb_begin_ts=1729061847727220, log_life_time=100055) [2024-10-16 14:57:27.733566] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:27.733583] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:27.750939] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:27.752357] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE765F-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1939, avg_group_log_size=176) [2024-10-16 14:57:27.752869] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D528F-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:27.752882] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D528F-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=39756, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:27.755200] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:27.755235] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=19] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:27.755243] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:27.756938] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=32] get_min_unreplayed_log_info(lsn={lsn:207589530414}, scn={val:1729061847632006001, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061847669975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589530414}, next_to_submit_scn_:{val:1729061847632006001, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589530292}, end_lsn:{lsn:207589530414}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061847632006000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061847632006000, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:2195193459, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061847632006000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598723, miss_cnt:9, cache_read_size:149777894855, hit_ratio:"9.999999345924205363e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:57:27.764381] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:27.769537] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E8D-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:27.769558] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E8D-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:27.769785] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=352, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:27.769798] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=207, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:27.769834] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:27.770448] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:27.771048] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=21] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:57:27.775734] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4149-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=645, avg_log_submit_to_slide_time=963) [2024-10-16 14:57:27.779615] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:27.779632] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:27.779640] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=48) [2024-10-16 14:57:27.780650] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.780967] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=13] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:27.788626] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.788625] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.788642] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.788680] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.788695] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.788723] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.788745] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.789001] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.789014] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.789057] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.789065] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:27.802706] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061847801890000, start_id=1729061847801890000, end_id=1729061847801890001) [2024-10-16 14:57:27.802768] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:27.802783] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9440, avg_group_log_size=629) [2024-10-16 14:57:27.802801] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9440) [2024-10-16 14:57:27.802808] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:27.803182] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.803754] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=22] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:27.816657] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] no need to update min/max version(new_version={val:1729061847215932001, v:0}, min_version={val:1729061847215932001, v:0}, max_version={val:1729061847215932001, v:0}) [2024-10-16 14:57:27.823967] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1002}, {id:1}]) [2024-10-16 14:57:27.827525] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:27.827543] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:27.827595] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52270625, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:27.834805] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=216, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:27.834821] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=426, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:27.841149] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=7] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:27.844352] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107003][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=29] worker thread exit(tenant_->id()=1004, workers_.get_size()=3) [2024-10-16 14:57:27.844370] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107003][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=13] tenant thread end_run(id_=1004, ret=0, thread_count_=192) [2024-10-16 14:57:27.851105] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3578, mem_used=8318976) [2024-10-16 14:57:27.852833] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1817, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:27.857431] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22136][T1002_TntTransf][T1002][YB4A0A000407-00061E9AB24CC6BB-0-0] [lt=12] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:57:27.858252] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=51] get number(ret=0, service_type_=0, range=1, base_id=1729061847857232000, start_id=1729061847857232000, end_id=1729061847857232001) [2024-10-16 14:57:27.858543] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=223, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:27.858566] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=22] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9440) [2024-10-16 14:57:27.858579] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=357, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:27.858617] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9440) [2024-10-16 14:57:27.858635] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=10, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:27.858817] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93DF-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:27.858840] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93DF-0-0] [lt=23] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=1015, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:27.858845] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93DF-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=643, avg_log_submit_to_slide_time=982) [2024-10-16 14:57:27.862824] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061847862677000, start_id=1729061847862677000, end_id=1729061847862677001) [2024-10-16 14:57:27.870574] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061847869119000, start_id=1729061847869119000, end_id=1729061847869119001) [2024-10-16 14:57:27.871632] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:27.880710] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.891747] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061843679355, last_first_register_time_us=1722494150998907, register_time_us=1729061843679355, curr_time_us=1729061847891745) [2024-10-16 14:57:27.891785] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061847891766, last_submit_register_req_time_us=1729061847891766, register_parent_reason=3) [2024-10-16 14:57:27.891798] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:27.900435] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=16] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:27.903223] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.904758] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=25] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:27.909857] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:27.910305] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:207589530714}, scn={val:1729061847632006003, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:27.910321] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:3, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061847910263, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589530770}, next_to_submit_scn_:{val:1729061847632006004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589530658}, end_lsn:{lsn:207589530770}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061847632006003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061847632006003, v:0}, data_checksum:442016515, flag:0}}, init_mode_version:0, accumulate_checksum:3645375752, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061847632006003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598726, miss_cnt:9, cache_read_size:149777895211, hit_ratio:"9.999999345924219796e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589530714}, scn_:{val:1729061847632006003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061847877593, first_handle_ts_:1729061847910319, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f38a577c0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:27.910477] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060040187603000) [2024-10-16 14:57:27.910487] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589530714}, scn_:{val:1729061847632006003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061847877593, first_handle_ts_:1729061847910464, replay_cost_:22, retry_cost_:-1, read_log_buf_:0x2b0f38a577d0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061847910263, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589530770}, next_to_submit_scn_:{val:1729061847632006004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589530658}, end_lsn:{lsn:207589530770}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061847632006003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061847632006003, v:0}, data_checksum:442016515, flag:0}}, init_mode_version:0, accumulate_checksum:3645375752, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061847632006003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598726, miss_cnt:9, cache_read_size:149777895211, hit_ratio:"9.999999345924219796e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_queue_idx=25, ret=0) [2024-10-16 14:57:27.916568] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 147.50; ) [2024-10-16 14:57:27.916586] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:27.916631] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 168.00; ) [2024-10-16 14:57:27.916640] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:27.916651] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:27.920939] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:27.921466] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15CE-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:27.925206] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=17] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:27.927502] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=267, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:27.927518] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:27.927528] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:27.927556] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:27.927732] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B900B-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:27.927740] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B900B-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=17758, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:27.927743] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107054][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B900B-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=8, avg_log_submit_to_flush_time=299, avg_log_submit_to_slide_time=19470) [2024-10-16 14:57:27.927768] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:27.927782] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656814, miss_cnt:17, cache_read_size:8050148019, hit_ratio:"9.999997286808200991e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:27.935806] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:57:27.935821] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:27.935828] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:27.935832] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:27.938044] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:57:27.938386] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=340] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:27.939328] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2340, mem_used=6239232) [2024-10-16 14:57:27.950208] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=16] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:27.950367] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=23] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:27.951241] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:27.952371] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7663-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:27.952916] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561140, miss_cnt:1, cache_read_size:152590659799, hit_ratio:"9.999999928346816613e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:27.955267] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601332, offset=9609690}, end_cursor:ObLogCursor{file_id=192, log_id=47601333, offset=9609974}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ba8050, len:3622, log_data_len:284, seq:47601332, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:27.957019] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA496-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061847955741000, start_id=1729061847955741000, end_id=1729061847955741001) [2024-10-16 14:57:27.957252] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:27.957267] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:27.957274] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=5] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=156) [2024-10-16 14:57:27.958109] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601333, offset=9609974}, end_cursor:ObLogCursor{file_id=192, log_id=47601334, offset=9610248}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ba4050, len:3338, log_data_len:274, seq:47601333, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:27.959932] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601334, offset=9610248}, end_cursor:ObLogCursor{file_id=192, log_id=47601335, offset=9610522}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c88050, len:3064, log_data_len:274, seq:47601334, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:27.960061] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601335, offset=9610522}, end_cursor:ObLogCursor{file_id=192, log_id=47601336, offset=9610796}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ba0050, len:2790, log_data_len:274, seq:47601335, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:27.960109] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=43] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598726, miss_cnt:9, cache_read_size:149777895211, hit_ratio:"9.999999345924219796e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:27.964193] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B3F-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:27.964372] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=267, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:27.964394] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:27.964434] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:27.964661] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650078, miss_cnt:32, cache_read_size:8050400648, hit_ratio:"9.999994892267547453e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:27.965798] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257698, offset=39014967}, end_cursor:ObLogCursor{file_id=197, log_id=49257699, offset=39015250}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b9c050, len:3529, log_data_len:283, seq:49257698, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:27.967631] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061847967629, bytes: 0, bw: 0.000000 MB/s, add_ts: 1008315, add_bytes: 0 [2024-10-16 14:57:27.967782] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB361-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061847965120000, start_id=1729061847965120000, end_id=1729061847965120001) [2024-10-16 14:57:27.969060] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO [ratelimit] time: 1729061847969059, bytes: 4446807861197, bw: 0.341341 MB/s, add_ts: 1001194, add_bytes: 358349 [2024-10-16 14:57:27.969351] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257699, offset=39015250}, end_cursor:ObLogCursor{file_id=197, log_id=49257700, offset=39015532}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a38050, len:3246, log_data_len:282, seq:49257699, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:27.970876] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA872-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:27.970895] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA872-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061847632006003, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061847632006003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:27.972224] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA872-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061847972140000, start_id=1729061847972140000, end_id=1729061847972140001) [2024-10-16 14:57:27.973621] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257700, offset=39015532}, end_cursor:ObLogCursor{file_id=197, log_id=49257701, offset=39015814}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b8c050, len:2964, log_data_len:282, seq:49257700, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:27.974104] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257701, offset=39015814}, end_cursor:ObLogCursor{file_id=197, log_id=49257702, offset=39016096}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2da4050, len:2682, log_data_len:282, seq:49257701, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:27.975037] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D2-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061847965900, keepalive_ts:-1}) [2024-10-16 14:57:27.975051] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D2-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061847965900, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:27.975257] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA872-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061847974821000, start_id=1729061847974821000, end_id=1729061847974821001) [2024-10-16 14:57:27.975395] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:27.975407] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:27.975985] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=376, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:27.975995] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:27.976003] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=93, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:27.976011] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:27.976018] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:27.976106] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:27.980640] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=24] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:27.980669] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:27.981467] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:27.981491] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=24] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:27.981509] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:27.981573] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=12] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061859029444000, latest_log_ts:{val:1729061842479541001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:27.981621] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:27.981638] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:57:27.981645] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061842479541001, v:0}) [2024-10-16 14:57:27.981649] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:27.981652] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:27.981658] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:27.981667] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:57:27.981674] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:27.981679] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:27.981908] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:27.993536] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:28.001162] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:28.001182] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3192) [2024-10-16 14:57:28.001190] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:28.003272] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.003337] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=18) [2024-10-16 14:57:28.017553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061848016084000, start_id=1729061848016084000, end_id=1729061848016084001) [2024-10-16 14:57:28.018238] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061848017726000, start_id=1729061848017726000, end_id=1729061848017726001) [2024-10-16 14:57:28.018380] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061847616970000, v:0}, new_max_version={val:1729061847616970000, v:0}, last_min_version={val:1729061847215932001, v:0}, last_max_version={val:1729061847215932001, v:0}, total_time_us=1635, delta_us=401409) [2024-10-16 14:57:28.018394] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061847616970000, v:0}, min_version={val:1729061847616970000, v:0}, max_version={val:1729061847616970000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:28.022638] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061848021925000, start_id=1729061848021925000, end_id=1729061848021925001) [2024-10-16 14:57:28.022657] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=562, get_gts_with_stc_cnt=31, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=29, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:28.022674] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=567, get_gts_with_stc_cnt=153, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=87, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:28.022679] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=4] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061848022672], gts:1729061848021925000, latest_srr:[mts=1729061848022672]}) [2024-10-16 14:57:28.031408] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F34-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061848031178000, start_id=1729061848031178000, end_id=1729061848031178001) [2024-10-16 14:57:28.033656] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:28.033677] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:28.033953] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:28.033981] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:28.055743] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:28.058427] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061848057047000, start_id=1729061848057047000, end_id=1729061848057047001) [2024-10-16 14:57:28.063446] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061848062479000, start_id=1729061848062479000, end_id=1729061848062479001) [2024-10-16 14:57:28.070711] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061848069921000, start_id=1729061848069921000, end_id=1729061848069921001) [2024-10-16 14:57:28.071713] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:28.081967] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.086043] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [22469][T1003_SSTableGC][T1003][YB4A0A000407-00061E997A7CB786-0-0] [lt=9] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60476}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015306454154001}, old_min_reserved_snapshot=1729060037234027000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060037234027000) [2024-10-16 14:57:28.088955] INFO [STORAGE] refresh_queuing_mode (ob_tenant_tablet_stat_mgr.cpp:1160) [22321][T1003_TabletSta][T1003][Y0-0000000000000000-0-0] [lt=15] refresh queuing mode(ret=0, tenant_id=1003, stream_cnt=35, update_schema_cnt=35, cost_time=46) [2024-10-16 14:57:28.088968] INFO [STORAGE] runTimerTask (ob_tenant_tablet_stat_mgr.cpp:1210) [22321][T1003_TabletSta][T1003][Y0-0000000000000000-0-0] [lt=11] TenantTabletStatMgr refresh all tablet stream(MTL_ID()=1003, interval_step=1, global_iter_pool={is_inited:false, is_washing:false, is_disabled:false, get_cnt:0, tenant_id:1003, bucket_cnt:0, tenant_mem_user_limit:0, tenant_mem_user_hold:0, cached_node_array:0x2b0e2cbb6360}) [2024-10-16 14:57:28.091861] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109202943, max_lsn:17959675637, max_scn:{val:1729061848031178000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7385580775}, reuse_lsn:{lsn:17959675637}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17917732597}}, last_submit_lsn:{lsn:17959675515}, last_submit_end_lsn:{lsn:17959675637}, last_submit_log_id:109202943, last_submit_log_pid:246, max_flushed_lsn:{lsn:17959675515}, max_flushed_end_lsn:{lsn:17959675637}, max_flushed_log_pid:246, committed_end_lsn:{lsn:17959675637}, last_slide_log_id:109202943, last_slide_scn:{val:1729061848031178000, v:0}, last_slide_lsn:{lsn:17959675515}, last_slide_end_lsn:{lsn:17959675637}, last_slide_log_pid:246, last_slide_log_accum_checksum:3938450268, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1d4530}, [StateMgr]={this:0x2b0e0f1d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109202944, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261839590, palf_role_change_cb:0x2b0e0f1df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, reconfig_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, persistent_config_version:{proposal_id:246, config_seq:121}, ms_ack_list:0, resend_config_version:{proposal_id:246, config_seq:121}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258131745, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258131745, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262039736, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728994126835081001, v:0}, min_block_id:265, min_block_min_scn:{val:1728952113212346001, v:0}, base_lsn_for_block_gc:{lsn:17849868288}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, log_config_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, log_snapshot_meta:{version:1, base_lsn:{lsn:17849868288}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1798144}, readable_log_tail:{lsn:1798144}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17984077824}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/meta", dir_fd:167, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65306624, block_header_serialize_buf_:0x2b0e0f1da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17959675637}, readable_log_tail:{lsn:17959675637}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17916973056}, min_scn:{val:1729035023717936004, v:0}, curr_block_id:267, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/log", dir_fd:169, min_block_id:265, max_block_id:268, curr_writable_block_id:267}, logical_block_size_:67104768, curr_block_writable_size_:24402187, block_header_serialize_buf_:0x2b0e0f1dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261838799, this:0x2b0e0f1d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:246, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:245, majority_max_lsn:{lsn:7385763025}, saved_end_lsn:{lsn:7385763025}, last_submit_prepare_req_time_us:1722494261839744, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1d89b0}) [2024-10-16 14:57:28.092022] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=158] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848075679, last_flushed_end_lsn:{lsn:17959675637}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848075874, last_flushed_end_lsn:{lsn:17959675637}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848075912, last_flushed_end_lsn:{lsn:17959675637}}]) [2024-10-16 14:57:28.092035] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=12] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=109202944, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848075679, last_flushed_end_lsn:{lsn:17959675637}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848075874, last_flushed_end_lsn:{lsn:17959675637}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848075912, last_flushed_end_lsn:{lsn:17959675637}}]) [2024-10-16 14:57:28.097635] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [22469][T1003_SSTableGC][T1003][YB4A0A000407-00061E997A7CB786-0-0] [lt=53] SSTableGCTask(cost_ts=11754) [2024-10-16 14:57:28.103301] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.105743] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061848104822000, start_id=1729061848104822000, end_id=1729061848104822001) [2024-10-16 14:57:28.110503] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:28.110520] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:28.122324] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=13] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97836134400) [2024-10-16 14:57:28.123848] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=16] gc stale ls task succ [2024-10-16 14:57:28.125313] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=12] current monitor number(seq_=-1) [2024-10-16 14:57:28.128355] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=849, mem_used=2079744) [2024-10-16 14:57:28.131131] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BCE-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061848130493000, start_id=1729061848130493000, end_id=1729061848130493001) [2024-10-16 14:57:28.131138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BD1-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061848130493000, start_id=1729061848130493000, end_id=1729061848130493001) [2024-10-16 14:57:28.131182] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BD0-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061848130493000, start_id=1729061848130493000, end_id=1729061848130493001) [2024-10-16 14:57:28.131398] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BD2-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061848130493000, start_id=1729061848130493000, end_id=1729061848130493001) [2024-10-16 14:57:28.131577] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BCF-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061848131553000, start_id=1729061848131553000, end_id=1729061848131553001) [2024-10-16 14:57:28.134294] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=26, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:57:28.134312] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=17] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=2298, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:57:28.134316] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:57:28.134319] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=-1116, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:57:28.134322] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=1208, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:57:28.151831] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:28.152448] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7667-0-0] [lt=15][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938131, log_task={header:{begin_lsn:{lsn:213058219817}, end_lsn:{lsn:213058219939}, log_id:220938131, min_scn:{val:1729061847630943003, v:0}, max_scn:{val:1729061847630943003, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058219695}, prev_proposal_id:88, committed_end_lsn:{lsn:213058219817}, data_checksum:1715074492, accum_checksum:1114255995, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061848052376, freeze_ts:1729061848052376, submit_ts:1729061848052381, flushed_ts:1729061848053218, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:837}, fs_cb_begin_ts=1729061848152443, log_life_time=100067) [2024-10-16 14:57:28.152711] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=31] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=299, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:28.152726] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=244, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:28.152760] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=33, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:28.152984] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CAB-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510316021376, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410524, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:28.153064] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=50, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:28.153076] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=50, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:28.164287] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B43-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:28.169805] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E95-0-0] [lt=22] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:28.169817] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E95-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24634, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:28.169822] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E95-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497280, log_task={header:{begin_lsn:{lsn:207589530892}, end_lsn:{lsn:207589531014}, log_id:218497280, min_scn:{val:1729061847632006005, v:0}, max_scn:{val:1729061847632006005, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589530770}, prev_proposal_id:81, committed_end_lsn:{lsn:207589530892}, data_checksum:940009244, accum_checksum:1182093637, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061848069714, freeze_ts:1729061848069714, submit_ts:1729061848069718, flushed_ts:1729061848070958, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1240}, fs_cb_begin_ts=1729061848169799, log_life_time=100085) [2024-10-16 14:57:28.170916] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=90] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=11339, avg_group_log_size=629) [2024-10-16 14:57:28.170935] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11810) [2024-10-16 14:57:28.171080] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=217, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:28.171095] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11339) [2024-10-16 14:57:28.171109] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=488, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:28.171147] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11810) [2024-10-16 14:57:28.171159] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:28.171382] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B63-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:28.171388] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B63-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=1431, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:28.171398] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B63-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=293, avg_log_gen_to_submit_time=322, avg_log_submit_to_flush_time=744, avg_log_submit_to_slide_time=1108) [2024-10-16 14:57:28.174575] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501E-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:28.174599] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501E-0-0] [lt=23] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:28.175539] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:28.175563] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:28.175993] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4156-0-0] [lt=19] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=520, avg_log_submit_to_slide_time=749) [2024-10-16 14:57:28.181982] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.201506] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3202) [2024-10-16 14:57:28.201576] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3192) [2024-10-16 14:57:28.201813] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B60-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:28.201825] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B60-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1167, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:28.203707] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.210615] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5875-0-0] [lt=13] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:28.210709] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5875-0-0] [lt=14][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:28.217643] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061848216588000, start_id=1729061848216588000, end_id=1729061848216588001) [2024-10-16 14:57:28.233786] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:28.233811] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:28.233837] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:28.233843] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:28.234073] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:28.236331] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:213058220117}, scn={val:1729061848130493001, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:28.236426] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060040187603000) [2024-10-16 14:57:28.236440] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:213058220117}, scn_:{val:1729061848130493001, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061848227125, first_handle_ts_:1729061848236414, replay_cost_:24, retry_cost_:-1, read_log_buf_:0x2b0f38a59560, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061848236286, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058220173}, next_to_submit_scn_:{val:1729061848130493002, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058220061}, end_lsn:{lsn:213058220173}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061848130493001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061848130493001, v:0}, data_checksum:3126288599, flag:0}}, init_mode_version:0, accumulate_checksum:2846047155, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061848130493001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561143, miss_cnt:1, cache_read_size:152590660155, hit_ratio:"9.999999928346818834e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}, replay_queue_idx=21, ret=0) [2024-10-16 14:57:28.247710] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD06-0-0] [lt=10][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:28.253015] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] succ to submit log task to replay service(to_submit_lsn={lsn:213058220229}, to_submit_scn={val:1729061848130493002, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061848252986, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058220295}, next_to_submit_scn_:{val:1729061848130493003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058220173}, end_lsn:{lsn:213058220295}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061848130493002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061848130493002, v:0}, data_checksum:3348837161, flag:1}}, init_mode_version:0, accumulate_checksum:1595731025, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061848130493002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561144, miss_cnt:1, cache_read_size:152590660277, hit_ratio:"9.999999928346818834e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:28.258677] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061848258351000, start_id=1729061848258351000, end_id=1729061848258351001) [2024-10-16 14:57:28.259368] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=246, start_service_tstamp_=1722494263598349, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:28.264380] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B45-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:28.264540] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061848263934000, start_id=1729061848263934000, end_id=1729061848263934001) [2024-10-16 14:57:28.270709] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061848270309000, start_id=1729061848270309000, end_id=1729061848270309001) [2024-10-16 14:57:28.271749] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:28.281556] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB94-0-0] [lt=14][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:28.282060] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.283280] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=12] ls blacklist refresh finish(cost_time=6742) [2024-10-16 14:57:28.295530] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [22470][T1003_Compactio][T1003][YB4A0A000407-00061E99742D0D6E-0-0] [lt=14] InfoPoolResizeTask(cost_ts=12) [2024-10-16 14:57:28.303765] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.306589] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061848303955000, start_id=1729061848303955000, end_id=1729061848303955001) [2024-10-16 14:57:28.311188] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:57:28.311204] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:28.311220] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373541948, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:28.313512] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=6020, mem_used=12478464) [2024-10-16 14:57:28.321509] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=18] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:28.325860] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:28.325922] WDIAG read_multi_row (ob_table_access_helper.h:238) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:28.326669] INFO delete_row (ob_table_access_helper.h:312) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=15] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1002, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061848325938) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061848325938) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:28.326880] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=15] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:28.327088] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=11] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:28.327259] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=6] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:28.334171] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=210, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:28.334201] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=158, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:28.334213] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:28.334449] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD084-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:28.334457] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD084-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=649, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:28.334645] INFO [LIB] stat (utility.h:1165) [107055][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCE2-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:28.334652] INFO [LIB] stat (utility.h:1165) [107055][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCE2-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=644, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:28.334656] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107055][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FCE2-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=354, avg_log_submit_to_slide_time=627) [2024-10-16 14:57:28.336308] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:28.336324] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:28.336342] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:28.346980] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=11] ====== tenant freeze timer task ====== [2024-10-16 14:57:28.346993] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:28.347005] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:28.347663] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=12] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:28.352058] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:28.352490] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE766B-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1807) [2024-10-16 14:57:28.352704] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1807) [2024-10-16 14:57:28.357928] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:28.370108] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1454) [2024-10-16 14:57:28.375646] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:28.375675] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:28.375921] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=227, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:28.375930] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:28.375941] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=98, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:28.375948] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:28.375965] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:28.376054] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C415C-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:28.376061] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C415C-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=531, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:28.382093] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.388919] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:28.401654] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:28.401966] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0F99-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=791, avg_log_submit_to_slide_time=1105) [2024-10-16 14:57:28.403828] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.409697] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B6-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:28.409731] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B6-0-0] [lt=22] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061848131553001, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061848131553001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:28.410771] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9B6-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061848410291000, start_id=1729061848410291000, end_id=1729061848410291001) [2024-10-16 14:57:28.411800] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9A279FA9B6-0-0] [lt=23] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.054650688171386719e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=302, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.073993929144144658e+03) [2024-10-16 14:57:28.413303] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9B6-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061848413120000, start_id=1729061848413120000, end_id=1729061848413120001) [2024-10-16 14:57:28.413965] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:28.417778] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061848417347000, start_id=1729061848417347000, end_id=1729061848417347001) [2024-10-16 14:57:28.427568] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CC8-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:28.427587] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CC8-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:28.427597] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CC8-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:28.434231] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:28.456399] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A72-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:28.456416] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A72-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:28.457334] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:28.458733] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061848457488000, start_id=1729061848457488000, end_id=1729061848457488001) [2024-10-16 14:57:28.464739] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061848464122000, start_id=1729061848464122000, end_id=1729061848464122001) [2024-10-16 14:57:28.470986] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061848469393000, start_id=1729061848469393000, end_id=1729061848469393001) [2024-10-16 14:57:28.471542] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=13] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:28.471566] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=22] get_min_unreplayed_log_info(lsn={lsn:14538472173}, scn={val:1729061848130493004, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061848464972, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538472173}, next_to_submit_scn_:{val:1729061848130493004, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538472051}, end_lsn:{lsn:14538472173}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061848130493003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061848130493003, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:1093088813, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061848130493003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650083, miss_cnt:32, cache_read_size:8050401258, hit_ratio:"9.999994892267954905e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:28.471628] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=57] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:28.471831] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:28.481717] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106793][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8E8-0-0] [lt=7] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:28.481736] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106793][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8E8-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:28.481753] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106793][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8E8-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:28.482126] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.491011] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C43-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:28.491044] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C43-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061848130493004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061848130493004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:28.491931] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C43-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061848491190000, start_id=1729061848491190000, end_id=1729061848491190001) [2024-10-16 14:57:28.492081] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:220938136, max_lsn:213058220539, max_scn:{val:1729061848130493004, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:60464363030}, reuse_lsn:{lsn:213058220539}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:213016277499}}, last_submit_lsn:{lsn:213058220417}, last_submit_end_lsn:{lsn:213058220539}, last_submit_log_id:220938136, last_submit_log_pid:88, max_flushed_lsn:{lsn:213058220417}, max_flushed_end_lsn:{lsn:213058220539}, max_flushed_log_pid:88, committed_end_lsn:{lsn:213058220539}, last_slide_log_id:220938136, last_slide_scn:{val:1729061848130493004, v:0}, last_slide_lsn:{lsn:213058220417}, last_slide_end_lsn:{lsn:213058220539}, last_slide_log_pid:88, last_slide_log_accum_checksum:290025678, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0dac0530}, [StateMgr]={this:0x2b0e0dac4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, leader:"10.0.4.8:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:60468525403}, scan_disk_log_finished:true, last_check_start_id:220938137, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261639483, palf_role_change_cb:0x2b0e0dacb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, reconfig_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, persistent_config_version:{proposal_id:88, config_seq:21}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061847891766, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061847891766, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0dac1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, accepted_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1729009096591759001, v:0}, min_block_id:3154, min_block_min_scn:{val:1729006455168279002, v:0}, base_lsn_for_block_gc:{lsn:212587905024}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, log_config_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, log_snapshot_meta:{version:1, base_lsn:{lsn:212587905024}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}}, log_meta_storage:{log_tail:{lsn:5767168}, readable_log_tail:{lsn:5767168}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213124743168}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/meta", dir_fd:175, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61337600, block_header_serialize_buf_:0x2b0e0dac69f8, flashback_version:0}, log_storage:{log_tail:{lsn:213058220539}, readable_log_tail:{lsn:213058220539}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213057638400}, min_scn:{val:1729061839666529002, v:0}, curr_block_id:3175, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/log", dir_fd:177, min_block_id:3154, max_block_id:3176, curr_writable_block_id:3175}, logical_block_size_:67104768, curr_block_writable_size_:66522629, block_header_serialize_buf_:0x2b0e0dac86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261637707, this:0x2b0e0dac4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:87, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:86, majority_max_lsn:{lsn:60466038076}, saved_end_lsn:{lsn:60466038076}, last_submit_prepare_req_time_us:1722494261639634, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0dac49b0}) [2024-10-16 14:57:28.492221] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=136] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:28.492233] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=220938137, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:28.492254] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:211147168, max_lsn:204023085241, max_scn:{val:1729061848031178004, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57922170954}, reuse_lsn:{lsn:204023085241}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:203981142201}}, last_submit_lsn:{lsn:204023085119}, last_submit_end_lsn:{lsn:204023085241}, last_submit_log_id:211147168, last_submit_log_pid:130, max_flushed_lsn:{lsn:204023085119}, max_flushed_end_lsn:{lsn:204023085241}, max_flushed_log_pid:130, committed_end_lsn:{lsn:204023085241}, last_slide_log_id:211147168, last_slide_scn:{val:1729061848031178004, v:0}, last_slide_lsn:{lsn:204023085119}, last_slide_end_lsn:{lsn:204023085241}, last_slide_log_pid:130, last_slide_log_accum_checksum:2843010749, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1ec530}, [StateMgr]={this:0x2b0e0f1f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:211147169, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261539368, palf_role_change_cb:0x2b0e0f1f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, reconfig_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, persistent_config_version:{proposal_id:130, config_seq:17}, ms_ack_list:0, resend_config_version:{proposal_id:130, config_seq:17}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258231773, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258231773, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, accepted_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261739861, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1729008439619722001, v:0}, min_block_id:3020, min_block_min_scn:{val:1729005678665693002, v:0}, base_lsn_for_block_gc:{lsn:203595866112}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, log_config_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, log_snapshot_meta:{version:1, base_lsn:{lsn:203595866112}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}}, log_meta_storage:{log_tail:{lsn:6397952}, readable_log_tail:{lsn:6397952}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:204065599488}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/meta", dir_fd:145, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:60706816, block_header_serialize_buf_:0x2b0e0f1f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:204023085241}, readable_log_tail:{lsn:204023085241}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:203998494720}, min_scn:{val:1729060878194301003, v:0}, curr_block_id:3040, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/log", dir_fd:173, min_block_id:3020, max_block_id:3041, curr_writable_block_id:3040}, logical_block_size_:67104768, curr_block_writable_size_:42514247, block_header_serialize_buf_:0x2b0e0f1f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261539352, this:0x2b0e0f1f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:130, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:129, majority_max_lsn:{lsn:57923794664}, saved_end_lsn:{lsn:57923794664}, last_submit_prepare_req_time_us:1722494261539574, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1f09b0}) [2024-10-16 14:57:28.492349] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=94] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848475959, last_flushed_end_lsn:{lsn:204023085241}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848476168, last_flushed_end_lsn:{lsn:204023085241}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848476243, last_flushed_end_lsn:{lsn:204023085241}}]) [2024-10-16 14:57:28.492357] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=211147169, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848475959, last_flushed_end_lsn:{lsn:204023085241}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848476168, last_flushed_end_lsn:{lsn:204023085241}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848476243, last_flushed_end_lsn:{lsn:204023085241}}]) [2024-10-16 14:57:28.492371] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:218497284, max_lsn:207589531502, max_scn:{val:1729061848132634003, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57807896114}, reuse_lsn:{lsn:207589531502}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:207547588462}}, last_submit_lsn:{lsn:207589531380}, last_submit_end_lsn:{lsn:207589531502}, last_submit_log_id:218497284, last_submit_log_pid:81, max_flushed_lsn:{lsn:207589531380}, max_flushed_end_lsn:{lsn:207589531502}, max_flushed_log_pid:81, committed_end_lsn:{lsn:207589531380}, last_slide_log_id:218497283, last_slide_scn:{val:1729061848132634002, v:0}, last_slide_lsn:{lsn:207589531258}, last_slide_end_lsn:{lsn:207589531380}, last_slide_log_pid:81, last_slide_log_accum_checksum:2474327865, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0db32530}, [StateMgr]={this:0x2b0e0db36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, leader:"10.0.4.9:2890", leader_epoch:7, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:57814812323}, scan_disk_log_finished:true, last_check_start_id:218497284, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433991656, palf_role_change_cb:0x2b0e0db3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, reconfig_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, persistent_config_version:{proposal_id:81, config_seq:20}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061844681942, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061844681942, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0db33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, accepted_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1729009518284639001, v:0}, min_block_id:3073, min_block_min_scn:{val:1729006819670466001, v:0}, base_lsn_for_block_gc:{lsn:207152418816}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, log_config_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, log_snapshot_meta:{version:1, base_lsn:{lsn:207152418816}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}}, log_meta_storage:{log_tail:{lsn:5660672}, readable_log_tail:{lsn:5660672}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207622152192}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/meta", dir_fd:180, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61444096, block_header_serialize_buf_:0x2b0e0db389f8, flashback_version:0}, log_storage:{log_tail:{lsn:207589531502}, readable_log_tail:{lsn:207589531502}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207555047424}, min_scn:{val:1729060516440786001, v:0}, curr_block_id:3093, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/log", dir_fd:182, min_block_id:3073, max_block_id:3094, curr_writable_block_id:3093}, logical_block_size_:67104768, curr_block_writable_size_:32620690, block_header_serialize_buf_:0x2b0e0db3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433990608, this:0x2b0e0db36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:80, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:79, majority_max_lsn:{lsn:57811903042}, saved_end_lsn:{lsn:57811903042}, last_submit_prepare_req_time_us:1722494433991756, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0db369b0}) [2024-10-16 14:57:28.492464] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=92] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:28.492472] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=218497284, log_task={header:{begin_lsn:{lsn:207589531380}, end_lsn:{lsn:207589531502}, log_id:218497284, min_scn:{val:1729061848132634003, v:0}, max_scn:{val:1729061848132634003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589531258}, prev_proposal_id:81, committed_end_lsn:{lsn:207589531380}, data_checksum:940009244, accum_checksum:2897285890, is_padding_log:false, is_raw_write:false}, state_map:{val:26}, ref_cnt:0, gen_ts:1729061848469894, freeze_ts:1729061848469894, submit_ts:1729061848469897, flushed_ts:1729061848471221, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:1324}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:28.494364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C43-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061848493927000, start_id=1729061848493927000, end_id=1729061848493927001) [2024-10-16 14:57:28.503685] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115178324, max_lsn:14538472173, max_scn:{val:1729061848130493003, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6487749074}, reuse_lsn:{lsn:14538472173}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14496529133}}, last_submit_lsn:{lsn:14538472051}, last_submit_end_lsn:{lsn:14538472173}, last_submit_log_id:115178324, last_submit_log_pid:23, max_flushed_lsn:{lsn:14538472051}, max_flushed_end_lsn:{lsn:14538472173}, max_flushed_log_pid:23, committed_end_lsn:{lsn:14538472173}, last_slide_log_id:115178324, last_slide_scn:{val:1729061848130493003, v:0}, last_slide_lsn:{lsn:14538472051}, last_slide_end_lsn:{lsn:14538472173}, last_slide_log_pid:23, last_slide_log_accum_checksum:1093088813, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43cc0530}, [StateMgr]={this:0x2b0e43cc4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, leader:"10.0.4.8:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6488080380}, scan_disk_log_finished:true, last_check_start_id:115178325, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261653045, palf_role_change_cb:0x2b0e43ccb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, reconfig_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, persistent_config_version:{proposal_id:23, config_seq:34}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061845798915, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061845798915, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43cc1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, accepted_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1728259580444569001, v:0}, min_block_id:201, min_block_min_scn:{val:1728204893605525001, v:0}, base_lsn_for_block_gc:{lsn:14427525120}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, log_config_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, log_snapshot_meta:{version:1, base_lsn:{lsn:14427525120}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}}, log_meta_storage:{log_tail:{lsn:1134592}, readable_log_tail:{lsn:1134592}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14561734656}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/meta", dir_fd:166, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65970176, block_header_serialize_buf_:0x2b0e43cc69f8, flashback_version:0}, log_storage:{log_tail:{lsn:14538472173}, readable_log_tail:{lsn:14538472173}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14494629888}, min_scn:{val:1729026040007117003, v:0}, curr_block_id:216, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/log", dir_fd:201, min_block_id:201, max_block_id:217, curr_writable_block_id:216}, logical_block_size_:67104768, curr_block_writable_size_:23262483, block_header_serialize_buf_:0x2b0e43cc86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261652788, this:0x2b0e43cc4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:22, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:21, majority_max_lsn:{lsn:6487893593}, saved_end_lsn:{lsn:6487893593}, last_submit_prepare_req_time_us:1722494261653224, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43cc49b0}) [2024-10-16 14:57:28.503824] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=135] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:28.503835] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=115178325, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:28.503886] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:28.503890] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.504625] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:28.504650] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=24] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:28.504669] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:28.504738] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061855590307000, latest_log_ts:{val:1729061838929180001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:28.504781] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=40] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:28.504798] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:28.504805] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061838929180001, v:0}) [2024-10-16 14:57:28.504808] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:28.504812] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:28.504817] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:28.504833] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=10][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:28.504840] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:28.504845] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:28.506644] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061848506627000, start_id=1729061848506627000, end_id=1729061848506627001) [2024-10-16 14:57:28.523003] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061848522922000, start_id=1729061848522922000, end_id=1729061848522922001) [2024-10-16 14:57:28.523025] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:28.523182] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061848523177], gts:1729061848522922000, latest_srr:[mts=1729061848523177]}) [2024-10-16 14:57:28.532103] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:57:28.532127] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=17] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:28.532148] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444455045, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:28.552360] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:28.569138] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.569138] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.569614] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.569620] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.570213] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E9D-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497284, log_task={header:{begin_lsn:{lsn:207589531380}, end_lsn:{lsn:207589531502}, log_id:218497284, min_scn:{val:1729061848132634003, v:0}, max_scn:{val:1729061848132634003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589531258}, prev_proposal_id:81, committed_end_lsn:{lsn:207589531380}, data_checksum:940009244, accum_checksum:2897285890, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061848469894, freeze_ts:1729061848469894, submit_ts:1729061848469897, flushed_ts:1729061848471221, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:1324}, fs_cb_begin_ts=1729061848570196, log_life_time=100302) [2024-10-16 14:57:28.576255] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4162-0-0] [lt=34] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:28.576268] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4162-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=527, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:28.582109] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1002}, {id:1003}]) [2024-10-16 14:57:28.582190] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.586665] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:28.586681] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:28.598444] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4711, mem_used=8318976) [2024-10-16 14:57:28.601798] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3079, avg_group_log_size=307) [2024-10-16 14:57:28.602136] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=327, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:28.602158] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=426, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:28.603749] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109593595, max_lsn:13645403413, max_scn:{val:1729061848522922000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6198843846}, reuse_lsn:{lsn:13645403413}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:13603460373}}, last_submit_lsn:{lsn:13645403291}, last_submit_end_lsn:{lsn:13645403413}, last_submit_log_id:109593595, last_submit_log_pid:17, max_flushed_lsn:{lsn:13645403291}, max_flushed_end_lsn:{lsn:13645403413}, max_flushed_log_pid:17, committed_end_lsn:{lsn:13645403413}, last_slide_log_id:109593595, last_slide_scn:{val:1729061848522922000, v:0}, last_slide_lsn:{lsn:13645403291}, last_slide_end_lsn:{lsn:13645403413}, last_slide_log_pid:17, last_slide_log_accum_checksum:3470640320, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431ec530}, [StateMgr]={this:0x2b0e431f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109593596, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261552988, palf_role_change_cb:0x2b0e431f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:14, lock_type:0, added_list:0, removed_list:0, type:"UNLOCK_CONFIG_CHANGE"}, state:0, checking_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:30}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:30}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494257649102, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494257649102, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261753456, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1728296971089337004, v:0}, min_block_id:189, min_block_min_scn:{val:1728240898542283004, v:0}, base_lsn_for_block_gc:{lsn:13555163136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:13555163136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}}, log_meta_storage:{log_tail:{lsn:1011712}, readable_log_tail:{lsn:1011712}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13689372672}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/meta", dir_fd:198, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66093056, block_header_serialize_buf_:0x2b0e431f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:13645403413}, readable_log_tail:{lsn:13645403413}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13622267904}, min_scn:{val:1729042537979987002, v:0}, curr_block_id:203, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/log", dir_fd:200, min_block_id:189, max_block_id:204, curr_writable_block_id:203}, logical_block_size_:67104768, curr_block_writable_size_:43969259, block_header_serialize_buf_:0x2b0e431f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261552391, this:0x2b0e431f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:6198988121}, saved_end_lsn:{lsn:6198988121}, last_submit_prepare_req_time_us:1722494261553147, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431f09b0}) [2024-10-16 14:57:28.603886] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=135] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848534367, last_flushed_end_lsn:{lsn:13645403413}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848534549, last_flushed_end_lsn:{lsn:13645403413}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848534700, last_flushed_end_lsn:{lsn:13645403413}}]) [2024-10-16 14:57:28.603898] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=109593596, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848534367, last_flushed_end_lsn:{lsn:13645403413}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848534549, last_flushed_end_lsn:{lsn:13645403413}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061848534700, last_flushed_end_lsn:{lsn:13645403413}}]) [2024-10-16 14:57:28.603937] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.617898] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061848616684000, start_id=1729061848616684000, end_id=1729061848616684001) [2024-10-16 14:57:28.631616] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BD3-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061848630722000, start_id=1729061848630722000, end_id=1729061848630722001) [2024-10-16 14:57:28.631634] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BD6-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061848630722000, start_id=1729061848630722000, end_id=1729061848630722001) [2024-10-16 14:57:28.631775] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BD7-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061848630722000, start_id=1729061848630722000, end_id=1729061848630722001) [2024-10-16 14:57:28.632254] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BD4-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061848631779000, start_id=1729061848631779000, end_id=1729061848631779001) [2024-10-16 14:57:28.632323] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BD5-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061848631779000, start_id=1729061848631779000, end_id=1729061848631779001) [2024-10-16 14:57:28.634059] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:28.634073] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] try gc retain ctx [2024-10-16 14:57:28.634076] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:28.634094] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] get_max_applied_scn(scn={val:1729061848522922000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493447926}, palf_committed_end_scn_:{val:1729061848522922000, v:0}, last_check_scn_:{val:1729061848522922000, v:0}, max_applied_cb_scn_:{val:1729061848522922000, v:0}}) [2024-10-16 14:57:28.634122] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:28.634782] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=53] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061845518506004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860721265000, latest_log_ts:{val:1729061844127564001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:28.634880] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=43] get_max_applied_scn(scn={val:1729061848522922000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645403413}, palf_committed_end_scn_:{val:1729061848522922000, v:0}, last_check_scn_:{val:1729061848522922000, v:0}, max_applied_cb_scn_:{val:1729061848522922000, v:0}}) [2024-10-16 14:57:28.634892] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:28.635519] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=35] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061845518506004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:28.636177] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=33] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061845518506004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:28.636212] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=31] get_min_unreplayed_log_info(lsn={lsn:14352358643}, scn={val:1729061848131553005, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061848629011, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352358643}, next_to_submit_scn_:{val:1729061848131553005, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352358521}, end_lsn:{lsn:14352358643}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061848131553004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061848131553004, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:1344531859, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061848131553004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656821, miss_cnt:17, cache_read_size:8050148864, hit_ratio:"9.999997286808504082e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:28.636858] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=30] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061845518506004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:28.637115] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:28.637177] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=601, avg_log_submit_to_slide_time=785) [2024-10-16 14:57:28.653578] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:28.653595] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:28.653613] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=16] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:28.653618] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:28.655730] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=104, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}]) [2024-10-16 14:57:28.655867] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=135] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:28.655901] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:28.659031] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061848658281000, start_id=1729061848658281000, end_id=1729061848658281001) [2024-10-16 14:57:28.664760] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061848663595000, start_id=1729061848663595000, end_id=1729061848663595001) [2024-10-16 14:57:28.664823] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:28.664836] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:28.670048] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84E9F-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:28.671157] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061848671146000, start_id=1729061848671146000, end_id=1729061848671146001) [2024-10-16 14:57:28.671301] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=14] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:28.672151] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061848671146000, start_id=1729061848671146000, end_id=1729061848671146001) [2024-10-16 14:57:28.675594] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=1950334, replace_num=31457) [2024-10-16 14:57:28.675616] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:28.675658] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D60-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:28.675681] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D60-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061848130493004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061848130493004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:28.675863] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:28.675890] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:28.680190] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D60-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061848679148000, start_id=1729061848679148000, end_id=1729061848679148001) [2024-10-16 14:57:28.682281] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.696404] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:28.698342] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=11] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:28.703999] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.706754] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061848705731000, start_id=1729061848705731000, end_id=1729061848705731001) [2024-10-16 14:57:28.722089] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=3445577, this={dio_aligned_buf:{buf_write_offset:1659, buf_padding_size:2437, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80962589, truncate_used_ts:29163428}, log_block_size:67108864, dir_fd:148, io_fd:171}, aligned_buf_len=4096, aligned_block_offset=3444736, buf_len=818, total_write_size_=102620, total_write_size_after_dio_=823296, ob_pwrite_used_ts=112814, count=176, each_pwrite_cost=640) [2024-10-16 14:57:28.723165] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:28.728251] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:28.728266] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:28.734178] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:28.734197] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:28.738249] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=15] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:28.738286] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15D2-0-0] [lt=11] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:28.738368] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15D4-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:28.749161] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=11][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:28.750487] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9] new mysql sessid created(conn.sessid_=3221734324, support_ssl=false) [2024-10-16 14:57:28.750518] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13] sm conn init succ(conn.sessid_=3221734324, sess.client_addr_="10.0.4.7:53366") [2024-10-16 14:57:28.750527] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] accept one succ(*s={this:0x2b0f27404030, session_id:3221734324, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:28.750592] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=11] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:28.751258] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37A9-0-0] [lt=22] MySQL LOGIN(direct_client_ip="10.0.4.7", client_ip=10.0.4.7, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=%, sessid=3221734324, proxy_sessid=13882479996272574487, client_sessid=2107375639, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=4062023, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:28.751416] INFO [SERVER] process (obmp_init_db.cpp:53) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CAD-0-0] [lt=13] init db(db_name=oceanbase) [2024-10-16 14:57:28.752654] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:28.753675] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3F84-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:28.753686] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3F84-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9563, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:28.753920] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=12] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:28.753935] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=14] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.753948] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=13] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.753952] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.753958] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754648] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:28.754675] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=13] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:28.754685] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:28.754693] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:28.754700] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:28.754727] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:28.754733] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754738] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754744] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754747] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754758] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:28.754763] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754766] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754769] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754773] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754778] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:28.754781] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754785] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754788] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.754791] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB1-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:28.755274] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:28.755298] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:28.755306] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:28.758181] INFO [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:194) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] get wrs ts(ls_id={id:1}, delta=234259, log_service_ts=1729061848522922, min_tx_service_ts=4611686018427387, timestamp={val:1729061848522922000, v:0}) [2024-10-16 14:57:28.758196] INFO [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:214) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=0, Near_To_GTS_Cnt=30, Other_Error_Cnt=0, Submit_Succ_Cnt=70, last_scn="{val:1729061846519305001, v:0}", last_lsn={lsn:17959673872}, last_gts={val:1729061848522922000, v:0}, min_start_scn="{val:0, v:0}", min_start_status=1, sys_ls_end_scn={val:0, v:0}) [2024-10-16 14:57:28.764621] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B4F-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:28.764633] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B4F-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10024, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:28.764637] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B4F-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=220, avg_log_submit_to_slide_time=10018) [2024-10-16 14:57:28.764670] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:28.764808] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:28.770071] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EA1-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:28.770094] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EA1-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:28.770306] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=458, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:28.770324] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=114, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:28.770353] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:28.770510] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:28.776471] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3F86-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=303, avg_log_submit_to_slide_time=549) [2024-10-16 14:57:28.776475] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB APPEND COST TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=46, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:57:28.776487] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=1687, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:57:28.776491] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:57:28.776494] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=-681, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:57:28.776497] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=2] [APPLY STAT CB TOTAL TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=1052, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:57:28.780937] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=17] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:28.782325] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.788514] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.788518] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.788532] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.788616] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=34] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.788619] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.788629] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.788641] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.789015] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.789035] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.789051] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.789066] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:28.803867] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=21] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061844798044, last_first_register_time_us=1722494157071428, register_time_us=1729061844798044, curr_time_us=1729061848803866) [2024-10-16 14:57:28.803904] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061848803887, last_submit_register_req_time_us=1729061848803887, register_parent_reason=3) [2024-10-16 14:57:28.803913] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:28.804061] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.805318] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:28.818012] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061848817459000, start_id=1729061848817459000, end_id=1729061848817459001) [2024-10-16 14:57:28.824071] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1003}, {id:1}]) [2024-10-16 14:57:28.841814] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=48] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:28.841829] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=11150, avg_group_log_size=557) [2024-10-16 14:57:28.841846] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11620) [2024-10-16 14:57:28.841853] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:28.848832] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:28.851200] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3647, mem_used=8318976) [2024-10-16 14:57:28.852770] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7675-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=12, avg_log_batch_cnt=0, total_group_log_size=1454, avg_group_log_size=121) [2024-10-16 14:57:28.853262] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:28.854869] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107055][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=7] worker thread exit(tenant_->id()=1004, workers_.get_size()=2) [2024-10-16 14:57:28.854886] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107055][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=12] tenant thread end_run(id_=1004, ret=0, thread_count_=191) [2024-10-16 14:57:28.859198] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061848858808000, start_id=1729061848858808000, end_id=1729061848858808001) [2024-10-16 14:57:28.859564] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=27] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=221, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:28.859577] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11150) [2024-10-16 14:57:28.859589] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=168, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:28.859617] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11150) [2024-10-16 14:57:28.859633] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:28.859706] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:28.859787] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93F3-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:28.859795] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93F3-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=735, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:28.859813] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F93F3-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=424, avg_log_submit_to_slide_time=703) [2024-10-16 14:57:28.864868] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061848864168000, start_id=1729061848864168000, end_id=1729061848864168001) [2024-10-16 14:57:28.868481] INFO [SQL] check_session_leak (ob_sql_session_mgr.cpp:652) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7] get current session count(used_session_count=23, hold_session_count=23, session_leak_count_threshold=100) [2024-10-16 14:57:28.871427] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1729061848870525000, start_id=1729061848870525000, end_id=1729061848870525001) [2024-10-16 14:57:28.874342] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=42] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:28.874981] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:28.875029] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061847238487000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:28.875668] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:28.882393] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.892951] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=12] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061844681942, last_first_register_time_us=1722494150998907, register_time_us=1729061844681942, curr_time_us=1729061848892949) [2024-10-16 14:57:28.892989] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061848892970, last_submit_register_req_time_us=1729061848892970, register_parent_reason=3) [2024-10-16 14:57:28.892998] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:28.900494] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=14] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:28.903928] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115173021, max_lsn:14352358887, max_scn:{val:1729061848632862001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6301879685}, reuse_lsn:{lsn:14352358887}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14310415847}}, last_submit_lsn:{lsn:14352358765}, last_submit_end_lsn:{lsn:14352358887}, last_submit_log_id:115173021, last_submit_log_pid:17, max_flushed_lsn:{lsn:14352358765}, max_flushed_end_lsn:{lsn:14352358887}, max_flushed_log_pid:17, committed_end_lsn:{lsn:14352358887}, last_slide_log_id:115173021, last_slide_scn:{val:1729061848632862001, v:0}, last_slide_lsn:{lsn:14352358765}, last_slide_end_lsn:{lsn:14352358887}, last_slide_log_pid:17, last_slide_log_accum_checksum:673658520, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43d32530}, [StateMgr]={this:0x2b0e43d36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.9:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6302436797}, scan_disk_log_finished:true, last_check_start_id:115173022, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433837169, palf_role_change_cb:0x2b0e43d3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, reconfig_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, persistent_config_version:{proposal_id:17, config_seq:26}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061848803887, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061848803887, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43d33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1728301961314447004, v:0}, min_block_id:199, min_block_min_scn:{val:1728247158357854003, v:0}, base_lsn_for_block_gc:{lsn:14293315584}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:14293315584}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}}, log_meta_storage:{log_tail:{lsn:1064960}, readable_log_tail:{lsn:1064960}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14360420352}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/meta", dir_fd:192, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66039808, block_header_serialize_buf_:0x2b0e43d389f8, flashback_version:0}, log_storage:{log_tail:{lsn:14352358887}, readable_log_tail:{lsn:14352358887}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14293315584}, min_scn:{val:1729013741169122001, v:0}, curr_block_id:213, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/log", dir_fd:207, min_block_id:199, max_block_id:214, curr_writable_block_id:213}, logical_block_size_:67104768, curr_block_writable_size_:8061465, block_header_serialize_buf_:0x2b0e43d3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433836510, this:0x2b0e43d36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:16, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:15, majority_max_lsn:{lsn:6302238429}, saved_end_lsn:{lsn:6302238429}, last_submit_prepare_req_time_us:1722494433837290, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43d369b0}) [2024-10-16 14:57:28.904075] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=143] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:28.904087] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=115173022, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:28.904110] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.906998] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061848905983000, start_id=1729061848905983000, end_id=1729061848905983001) [2024-10-16 14:57:28.909459] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD4-0-0] [lt=21] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061848877185, keepalive_ts:-1}) [2024-10-16 14:57:28.909477] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD4-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061848877185, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:28.919335] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 118.00; ) [2024-10-16 14:57:28.919359] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:28.919398] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 239.50; ) [2024-10-16 14:57:28.919403] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:28.919412] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:28.921007] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:28.928086] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=208, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:28.928101] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:28.928116] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:28.928149] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:28.928412] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B901F-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:28.928423] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B901F-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=514, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:28.928428] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107054][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B901F-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=246, avg_log_submit_to_slide_time=508) [2024-10-16 14:57:28.928425] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:28.928442] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656824, miss_cnt:17, cache_read_size:8050149230, hit_ratio:"9.999997286808633978e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:28.934667] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=201, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:28.934684] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:28.939437] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2375, mem_used=6239232) [2024-10-16 14:57:28.952938] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:28.955915] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Cache wash clean map node details(ret=0, clean_node_count=31, clean_time=0, clean_start_pos=629140, clean_num=62914) [2024-10-16 14:57:28.956093] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7677-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:28.962381] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601336, offset=9610796}, end_cursor:ObLogCursor{file_id=192, log_id=47601337, offset=9611080}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b38050, len:2516, log_data_len:284, seq:47601336, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:28.964252] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA497-0-0] [lt=18] submit log success(service_type_=0, last_id=1729061865590307000, limited_id=1729061865590307000) [2024-10-16 14:57:28.964262] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA497-0-0] [lt=10] submit log with lock success(service_type_=0, last_id=1729061865590307000, limited_id=1729061865590307000) [2024-10-16 14:57:28.964266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA497-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061848964072000, start_id=1729061848964072000, end_id=1729061848964072001) [2024-10-16 14:57:28.964753] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B53-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:28.964815] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650087, miss_cnt:32, cache_read_size:8050401746, hit_ratio:"9.999994892268281310e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:28.964941] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=175, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:28.964960] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:28.964995] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:28.965004] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] update id meta(ret=0, service_type=0, limited_id=1729061865590307000, latest_scn={val:1729061848964072001, v:0}, *this={tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061865590307000, latest_log_ts:{val:1729061848964072001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:28.965035] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=32] handle submit callback(service_type_=0, success=true, used_ts=784, log_ts={val:1729061848964072001, v:0}, limited_id=1729061865590307000) [2024-10-16 14:57:28.965048] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=13] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:28.965053] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:28.965120] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061848964072000, start_id=1729061848964072001, end_id=1729061848964072002) [2024-10-16 14:57:28.965140] INFO [STORAGE.TRANS] wait_gts_elapse (ob_gts_source.cpp:434) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=19] wait queue push task success(task=0x2b0df701e570) [2024-10-16 14:57:28.965152] INFO [STORAGE.TRANS] wait_gts_elapse_commit_version_ (ob_trans_part_ctx.cpp:3072) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] need wait gts elapse(ret=0, ret="OB_SUCCESS", *this={this:0x2b0df701d0d0, ref:3, trans_id:{txid:193381284}, tenant_id:1001, is_exiting:false, trans_expired_time:1729061878963027, cluster_version:17180066304, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=1729061848964199], ctx_create_time:1729061848963035}{self_ls_id:{id:1}, session_id:1, part_trans_action:3, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x2b0e08204030, tx_data_guard_:{tx_data:{tx_id:{txid:193381284}, ref_cnt:1, state:"ELR_COMMIT", commit_version:{val:1729061848964072002, v:0}, start_scn:{val:1729061848964072002, v:0}, end_scn:{val:1729061848964072002, v:0}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:263984595184547857, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:1729061848964072002, v:0}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1729061848964072, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1722494638569576 max_table_version=1722494638569576 trans_version={val:1729061848964072000, v:0} commit_version={val:1729061848964072002, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:193381284} ls_id=1 row_callback[alloc:4, free:0, unsubmit:0] redo[fill:6,sync_succ:6, sync_fail:0] main_list_len=6 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,6,6,6,6,0,0,0)|]}} -{busy_cbs_cnt:1, oldest_busy_cb:{ObTxBaseLogCb:{base_ts:{val:1729061848964072000, v:0}, log_ts:{val:1729061848964072002, v:0}, lsn:{lsn:114618388051}, submit_ts:1729061848964289}, this:0x2b0df7020a40, is_inited_:true, trans_id:{txid:193381284}, ls_id:{id:1}, ctx:0x2b0df701d0d0, tx_data_guard:{tx_data:NULL}, is_callbacked_:false, is_dynamic_:false, mds_range_:{range_array_.count():0, range_array_:[]}, cb_arg_array_:[{log_type:0x1, arg:null}, {log_type:0x40, arg:null}, {log_type:0x100, arg:null}], first_part_scn_:{val:18446744073709551615, v:3}, extra_cb_:null, need_free_extra_cb_:false, callbacks_.count():1}, final_log_cb:{ObTxBaseLogCb:{base_ts:{val:1729061848964072000, v:0}, log_ts:{val:1729061848964072002, v:0}, lsn:{lsn:114618388051}, submit_ts:1729061848964289}, this:0x2b0df7020a40, is_inited_:true, trans_id:{txid:193381284}, ls_id:{id:1}, ctx:0x2b0df701d0d0, tx_data_guard:{tx_data:NULL}, is_callbacked_:false, is_dynamic_:false, mds_range_:{range_array_.count():0, range_array_:[]}, cb_arg_array_:[{log_type:0x1, arg:null}, {log_type:0x40, arg:null}, {log_type:0x100, arg:null}], first_part_scn_:{val:18446744073709551615, v:3}, extra_cb_:null, need_free_extra_cb_:false, callbacks_.count():1}, -{info_log_submitted:1 gts_waiting:1 state_log_submitted:1 }-, -{downstream_state:"REDO_COMPLETE", upstream:{id:-1}, participants:[{id:1}], redo_log_no:0, scheduler:"10.0.4.7:2890", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:1, max_applied_log_ts:{val:1729061848964072002, v:0}, max_appling_log_ts:{val:1729061848964072002, v:0}, max_applying_part_log_no:0, max_submitted_seq_no:{branch:0, seq:9}, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, max_durable_lsn:{lsn:114618388051}, commit_parts:[{ls_id:{id:1}, exec_epoch:263984595184547857, transfer_epoch:-1}], }- }-}) [2024-10-16 14:57:28.965585] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601337, offset=9611080}, end_cursor:ObLogCursor{file_id=192, log_id=47601338, offset=9611354}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a98050, len:2232, log_data_len:274, seq:47601337, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:28.967609] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=27] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601338, offset=9611354}, end_cursor:ObLogCursor{file_id=192, log_id=47601339, offset=9611628}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ab4050, len:1958, log_data_len:274, seq:47601338, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:28.967783] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601339, offset=9611628}, end_cursor:ObLogCursor{file_id=192, log_id=47601340, offset=9611902}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bc8050, len:1684, log_data_len:274, seq:47601339, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:28.967832] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=69] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561150, miss_cnt:1, cache_read_size:152590661009, hit_ratio:"9.999999928346822164e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:28.967849] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=62] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598735, miss_cnt:9, cache_read_size:149777896309, hit_ratio:"9.999999345924261984e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:28.970130] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EA5-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497288, log_task={header:{begin_lsn:{lsn:207589531868}, end_lsn:{lsn:207589531990}, log_id:218497288, min_scn:{val:1729061848631779003, v:0}, max_scn:{val:1729061848631779003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589531746}, prev_proposal_id:81, committed_end_lsn:{lsn:207589531868}, data_checksum:940009244, accum_checksum:2273839550, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061848870113, freeze_ts:1729061848870113, submit_ts:1729061848870117, flushed_ts:1729061848875083, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:4966}, fs_cb_begin_ts=1729061848970126, log_life_time=100013) [2024-10-16 14:57:28.970184] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:28.970517] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061848970516, bytes: 4446808276649, bw: 0.395629 MB/s, add_ts: 1001457, add_bytes: 415452 [2024-10-16 14:57:28.974544] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061848974544, bytes: 0, bw: 0.000000 MB/s, add_ts: 1006915, add_bytes: 0 [2024-10-16 14:57:28.975063] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257702, offset=39016096}, end_cursor:ObLogCursor{file_id=197, log_id=49257703, offset=39016379}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c3c050, len:2400, log_data_len:283, seq:49257702, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:28.976072] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:28.976086] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:28.976268] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=210, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:28.976287] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=28, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:28.976314] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:28.979253] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C416F-0-0] [lt=21] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:28.980691] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB362-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061848979178000, start_id=1729061848979178000, end_id=1729061848979178001) [2024-10-16 14:57:28.980856] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA873-0-0] [lt=10] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:28.980875] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA873-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061848631779004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061848631779004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:28.981913] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:28.981925] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27628320, tenant_id=1004) [2024-10-16 14:57:28.981934] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:28.982064] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA873-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061848981998000, start_id=1729061848981998000, end_id=1729061848981998001) [2024-10-16 14:57:28.982344] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257703, offset=39016379}, end_cursor:ObLogCursor{file_id=197, log_id=49257704, offset=39016661}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b50050, len:2117, log_data_len:282, seq:49257703, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:28.983133] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:28.983151] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:28.983158] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:28.985365] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA873-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061848985160000, start_id=1729061848985160000, end_id=1729061848985160001) [2024-10-16 14:57:28.985929] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=104] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:28.985996] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257704, offset=39016661}, end_cursor:ObLogCursor{file_id=197, log_id=49257705, offset=39016943}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2be4050, len:1835, log_data_len:282, seq:49257704, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:28.986226] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257705, offset=39016943}, end_cursor:ObLogCursor{file_id=197, log_id=49257706, offset=39017225}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c70050, len:1553, log_data_len:282, seq:49257705, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:28.992238] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E9E-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061848959999, keepalive_ts:-1}) [2024-10-16 14:57:28.992254] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E9E-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061848959999, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:28.992331] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E9F-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061848960057, keepalive_ts:-1}) [2024-10-16 14:57:28.992342] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4E9F-0-0] [lt=11] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061848960057, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:28.992591] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:28.992617] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=16] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=209) [2024-10-16 14:57:28.998893] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=12] ====== tenant freeze timer task ====== [2024-10-16 14:57:29.002095] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=21] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:29.002145] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3544) [2024-10-16 14:57:29.002158] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:29.004173] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.013998] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:29.014750] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=14] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.014777] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.014797] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:29.014935] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846519305003, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860098662000, latest_log_ts:{val:1729061843439703001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:29.014980] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=42] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846519305003, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860098662000, latest_log_ts:{val:1729061843439703001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:29.015010] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:29.015017] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061843439703001, v:0}) [2024-10-16 14:57:29.015021] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:29.015028] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:29.015038] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:29.015045] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:29.015752] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.015766] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=13] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.015777] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:29.015816] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846519305003, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:29.015845] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846519305003, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:29.015867] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:29.015873] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:29.015879] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:29.015884] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:29.016590] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.016610] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=20] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.016619] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:29.016653] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846519305003, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:29.016684] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846519305003, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:29.016707] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:29.016712] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:29.016718] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:29.016723] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:29.017426] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.017441] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.017451] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:29.017486] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846519305003, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:29.017514] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846519305003, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:29.017537] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:29.017543] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:29.017549] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:29.017553] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:29.017557] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:29.018063] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=11] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:29.018229] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061849017364000, start_id=1729061849017364000, end_id=1729061849017364001) [2024-10-16 14:57:29.018869] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061848817459000, v:0}, new_max_version={val:1729061848817459000, v:0}, last_min_version={val:1729061848616684000, v:0}, last_max_version={val:1729061848616684000, v:0}, total_time_us=1476, delta_us=201409) [2024-10-16 14:57:29.018886] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061848817459000, v:0}, min_version={val:1729061848817459000, v:0}, max_version={val:1729061848817459000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:29.023378] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061849022611000, start_id=1729061849022611000, end_id=1729061849022611001) [2024-10-16 14:57:29.023397] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061849023373], gts:1729061849022611000, latest_srr:[mts=1729061849023373]}) [2024-10-16 14:57:29.034505] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:29.034538] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:29.034770] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:29.034789] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1636) [2024-10-16 14:57:29.036989] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:29.037002] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:29.037014] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291875945, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:29.038737] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:29.038751] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=152203660, tenant_id=1) [2024-10-16 14:57:29.038764] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:29.039038] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F47-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061849038601000, start_id=1729061849038601000, end_id=1729061849038601001) [2024-10-16 14:57:29.059454] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061849058618000, start_id=1729061849058618000, end_id=1729061849058618001) [2024-10-16 14:57:29.062167] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=64] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:29.062752] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=19] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:29.062795] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061840187603000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:29.065151] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061849064094000, start_id=1729061849064094000, end_id=1729061849064094001) [2024-10-16 14:57:29.071245] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061849069480000, start_id=1729061849069480000, end_id=1729061849069480001) [2024-10-16 14:57:29.075741] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:29.076358] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:29.083198] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.104223] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.106986] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061849106092000, start_id=1729061849106092000, end_id=1729061849106092001) [2024-10-16 14:57:29.110480] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:29.110495] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:29.127007] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15CC-0-0] [lt=15] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:29.127057] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:29.128472] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=900, mem_used=2079744) [2024-10-16 14:57:29.131980] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BD8-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061849131137000, start_id=1729061849131137000, end_id=1729061849131137001) [2024-10-16 14:57:29.132006] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BD9-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061849131137000, start_id=1729061849131137000, end_id=1729061849131137001) [2024-10-16 14:57:29.132066] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BDA-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061849131137000, start_id=1729061849131137000, end_id=1729061849131137001) [2024-10-16 14:57:29.132148] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BDC-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061849131137000, start_id=1729061849131137000, end_id=1729061849131137001) [2024-10-16 14:57:29.132362] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BDB-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061849132194000, start_id=1729061849132194000, end_id=1729061849132194001) [2024-10-16 14:57:29.153071] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:29.153092] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:29.153134] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:29.153365] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=53, stat_interval=1000000, avg (count/cost)=31, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:29.153372] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=53, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:29.155522] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15D0-0-0] [lt=19] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:29.156092] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21667][T1_L0_G0][T1][YB4A0A000407-00061E99796CFF95-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317024485, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410525, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:29.156212] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:29.163945] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=16][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:29.164870] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B57-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:29.170312] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EA9-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:29.170325] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EA9-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40815, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:29.170332] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EA9-0-0] [lt=5][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497290, log_task={header:{begin_lsn:{lsn:207589532112}, end_lsn:{lsn:207589532234}, log_id:218497290, min_scn:{val:1729061848631779005, v:0}, max_scn:{val:1729061848631779005, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589531990}, prev_proposal_id:81, committed_end_lsn:{lsn:207589532112}, data_checksum:940009244, accum_checksum:3124798803, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061849070256, freeze_ts:1729061849070256, submit_ts:1729061849070260, flushed_ts:1729061849071143, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:883}, fs_cb_begin_ts=1729061849170305, log_life_time=100049) [2024-10-16 14:57:29.174895] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10338, avg_group_log_size=574) [2024-10-16 14:57:29.174909] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10338) [2024-10-16 14:57:29.175570] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501F-0-0] [lt=22] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:29.175583] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD501F-0-0] [lt=12] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:29.175667] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD061-0-0] [lt=39] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10338) [2024-10-16 14:57:29.175798] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=267, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:29.175807] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10338) [2024-10-16 14:57:29.175816] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=76, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:29.175834] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:29.175854] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=19] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=985, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:29.175861] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=772, avg_log_submit_to_slide_time=955) [2024-10-16 14:57:29.175883] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=20] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:29.176198] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:29.176218] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:29.176674] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4174-0-0] [lt=22] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=292, avg_log_submit_to_slide_time=566) [2024-10-16 14:57:29.183339] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.202403] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3543) [2024-10-16 14:57:29.202461] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=24] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3543) [2024-10-16 14:57:29.202841] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FA1-0-0] [lt=31] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:29.202854] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FA1-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=655, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:29.204259] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.217464] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] no need to update min/max version(new_version={val:1729061848817459000, v:0}, min_version={val:1729061848817459000, v:0}, max_version={val:1729061848817459000, v:0}) [2024-10-16 14:57:29.228748] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:57:29.229486] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.229513] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=26] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.229537] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=19] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:29.229688] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846526670004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860721265000, latest_log_ts:{val:1729061844127564001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:29.229740] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=48] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:29.229757] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:29.229766] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=0, rec_log_ts={val:1729061844127564001, v:0}) [2024-10-16 14:57:29.229771] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.229781] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:29.229793] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:29.229801] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:29.230486] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.230501] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.230514] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:29.230581] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846526670004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:29.230625] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=41] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846526670004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:29.230649] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=23] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:29.230656] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:29.230664] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:29.230669] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:29.231243] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21659][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE87D-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317099637, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410526, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:29.231389] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.231408] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=18] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.231420] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:29.231506] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846526670004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:29.231567] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=59] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846526670004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:29.231625] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=54] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:29.231635] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:29.231647] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:29.231656] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:29.232569] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.232594] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=24] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:29.232617] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=21] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:29.232701] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846526670004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:29.232752] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=49] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061846526670004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:29.232795] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=41] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:29.232805] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:29.232816] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:29.232824] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:29.232831] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:29.234637] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1636, avg_group_log_size=163) [2024-10-16 14:57:29.234650] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1636) [2024-10-16 14:57:29.234681] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:29.234687] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:29.234933] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:29.240409] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E09-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061849239560000, start_id=1729061849239560000, end_id=1729061849239560001) [2024-10-16 14:57:29.259379] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061849258509000, start_id=1729061849258509000, end_id=1729061849258509001) [2024-10-16 14:57:29.264448] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1DFE-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061849262865000, start_id=1729061849262865000, end_id=1729061849262865001) [2024-10-16 14:57:29.265441] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061849262865000, start_id=1729061849262865000, end_id=1729061849262865001) [2024-10-16 14:57:29.266170] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=59] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:29.268184] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1030-0-0] [lt=9] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:29.268778] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107139][][T1003][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1003, ret=0, thread_count_=173) [2024-10-16 14:57:29.268794] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107139][][T1003][Y0-0000000000000000-0-0] [lt=15] Init thread local success [2024-10-16 14:57:29.268809] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] worker thread created(tenant_->id()=1003, group_id_=1, token=1) [2024-10-16 14:57:29.271244] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061849271151000, start_id=1729061849271151000, end_id=1729061849271151001) [2024-10-16 14:57:29.271782] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107139][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797B77-0-0] [lt=5] init point thread id with(&point=0x55e3435c4f40, idx_=643, point=[thread id=107139, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107139) [2024-10-16 14:57:29.275797] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:29.278885] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=5] dump tenant info(tenant={id:1, tenant_meta:{unit:{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, super_block:{tenant_id:1, replay_start_point:ObLogCursor{file_id=12, log_id=1497048, offset=40358755}, ls_meta_entry:[2738](ver=0,mode=0,seq=7639021), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"3.000000000000000000e+00", unit_max_cpu:"3.000000000000000000e+00", total_worker_cnt:54, min_worker_cnt:32, max_worker_cnt:150, stopped:0, worker_us:1096708596279, recv_hp_rpc_cnt:5717679, recv_np_rpc_cnt:154801160, recv_lp_rpc_cnt:0, recv_mysql_cnt:62153760, recv_task_cnt:6664561, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:19623835953, workers:32, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=7 cnt[3]=0 cnt[4]=0 cnt[5]=228764349 cnt[6]=10623 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52540921, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143148686}{group_id:10, queue_size:0, recv_req_cnt:19682035, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143048442}{group_id:20, queue_size:0, recv_req_cnt:401, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015260372185}{group_id:15, queue_size:0, recv_req_cnt:5821, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723554329}{group_id:5, queue_size:0, recv_req_cnt:35746055, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494262337428}{group_id:19, queue_size:0, recv_req_cnt:7245669, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061834968674}{group_id:9, queue_size:0, recv_req_cnt:15758054, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143222379}{group_id:1, queue_size:0, recv_req_cnt:153510715, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061836676474}, rpc_stat_info: pcode=0x51c:cnt=327 pcode=0x1502:cnt=216 pcode=0x717:cnt=187 pcode=0x150b:cnt=60 pcode=0x710:cnt=54, token_change_ts:1722494237022007, tenant_role:1}) [2024-10-16 14:57:29.279373] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=486] dump throttled time info(id_=1, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 10, group: OBCG_LOC_CACHE, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 9, group: OBCG_DETECT_RS, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1, tenant_throttled_time: 0;) [2024-10-16 14:57:29.279387] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] dump tenant info(tenant={id:508, tenant_meta:{unit:{tenant_id:508, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140910089, is_removed:false}, super_block:{tenant_id:508, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:15572876024, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:51691530, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6536470782, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:15, queue_size:0, recv_req_cnt:53884, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060722522359}, rpc_stat_info: pcode=0x5c0:cnt=72, token_change_ts:1722494140914497, tenant_role:0}) [2024-10-16 14:57:29.279817] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=429] dump throttled time info(id_=508, throttled_time_log=group_id: 15, group: OBCG_WR, throttled_time: 0;tenant_id: 508, tenant_throttled_time: 0;) [2024-10-16 14:57:29.279823] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=6] dump tenant info(tenant={id:509, tenant_meta:{unit:{tenant_id:509, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140901337, is_removed:false}, super_block:{tenant_id:509, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:12840222139, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6441809719, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1722494140916657, tenant_role:0}) [2024-10-16 14:57:29.280217] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=394] dump throttled time info(id_=509, throttled_time_log=tenant_id: 509, tenant_throttled_time: 0;) [2024-10-16 14:57:29.280222] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=5] dump tenant info(tenant={id:1001, tenant_meta:{unit:{tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1001, replay_start_point:ObLogCursor{file_id=11, log_id=1059563, offset=59267180}, ls_meta_entry:[7452](ver=0,mode=0,seq=7431948), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:468364335890, recv_hp_rpc_cnt:13589252, recv_np_rpc_cnt:126529241, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2784367477, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=9 cnt[3]=0 cnt[4]=0 cnt[5]=167011141 cnt[6]=3244 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52540920, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494145075365}{group_id:20, queue_size:0, recv_req_cnt:306, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015263681355}{group_id:5, queue_size:0, recv_req_cnt:58923431, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726724812431820}{group_id:19, queue_size:0, recv_req_cnt:7662178, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061844973518}{group_id:1, queue_size:0, recv_req_cnt:227492746, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061842999505}, rpc_stat_info: pcode=0x1502:cnt=350 pcode=0x51c:cnt=221 pcode=0x717:cnt=188 pcode=0x710:cnt=90 pcode=0x150b:cnt=60, token_change_ts:1722494144838773, tenant_role:1}) [2024-10-16 14:57:29.280674] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=451] dump throttled time info(id_=1001, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1001, tenant_throttled_time: 0;) [2024-10-16 14:57:29.280687] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] dump tenant info(tenant={id:1002, tenant_meta:{unit:{tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1002, replay_start_point:ObLogCursor{file_id=191, log_id=47529733, offset=57901112}, ls_meta_entry:[10485](ver=0,mode=0,seq=7657455), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:998832827647, recv_hp_rpc_cnt:198478828, recv_np_rpc_cnt:379058432, recv_lp_rpc_cnt:0, recv_mysql_cnt:280947390, recv_task_cnt:146658, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:4103762705, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=673 cnt[3]=0 cnt[4]=0 cnt[5]=64428781 cnt[6]=226112 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131354653, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1728209719937557}{group_id:20, queue_size:0, recv_req_cnt:1673, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264869418}{group_id:15, queue_size:0, recv_req_cnt:5733, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723564477}{group_id:5, queue_size:0, recv_req_cnt:401894008, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1727167492226332}{group_id:19, queue_size:0, recv_req_cnt:1399883, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061796075772}{group_id:1, queue_size:0, recv_req_cnt:977158929, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061846446256}, rpc_stat_info: pcode=0x710:cnt=1767 pcode=0x1502:cnt=1016 pcode=0x52b:cnt=970 pcode=0x1501:cnt=962 pcode=0x1516:cnt=886, token_change_ts:1729004298741539, tenant_role:1}) [2024-10-16 14:57:29.281131] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=444] dump throttled time info(id_=1002, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1002, tenant_throttled_time: 0;) [2024-10-16 14:57:29.281143] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] dump tenant info(tenant={id:1003, tenant_meta:{unit:{tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1003, replay_start_point:ObLogCursor{file_id=11, log_id=1067656, offset=59494798}, ls_meta_entry:[3704](ver=0,mode=0,seq=7380334), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:19, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:503355268761, recv_hp_rpc_cnt:13609642, recv_np_rpc_cnt:126131945, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2783964242, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=4 cnt[3]=0 cnt[4]=0 cnt[5]=196317457 cnt[6]=6245 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52540885, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722517512781421}{group_id:20, queue_size:0, recv_req_cnt:308, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015261556381}{group_id:5, queue_size:0, recv_req_cnt:63971647, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726083164255294}{group_id:19, queue_size:0, recv_req_cnt:7660878, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061802906653}{group_id:1, queue_size:0, recv_req_cnt:228853802, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:2, nesting_worker_cnt:0, token_change:1729061849267795}, rpc_stat_info: pcode=0x1502:cnt=356 pcode=0x51c:cnt=253 pcode=0x717:cnt=190 pcode=0x710:cnt=96 pcode=0x150b:cnt=63, token_change_ts:1722494150252707, tenant_role:1}) [2024-10-16 14:57:29.281575] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=431] dump throttled time info(id_=1003, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1003, tenant_throttled_time: 0;) [2024-10-16 14:57:29.281586] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=10] dump tenant info(tenant={id:1004, tenant_meta:{unit:{tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1004, replay_start_point:ObLogCursor{file_id=197, log_id=49173916, offset=15367427}, ls_meta_entry:[12150](ver=0,mode=0,seq=7655944), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:603322633277, recv_hp_rpc_cnt:12341744, recv_np_rpc_cnt:113755242, recv_lp_rpc_cnt:0, recv_mysql_cnt:11227, recv_task_cnt:1052, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2741348913, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=99 cnt[3]=0 cnt[4]=0 cnt[5]=72426255 cnt[6]=155203 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131354511, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494154950936}{group_id:20, queue_size:0, recv_req_cnt:798, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264132685}{group_id:15, queue_size:0, recv_req_cnt:5423, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723614999}{group_id:21, queue_size:0, recv_req_cnt:15, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269726417109}{group_id:5, queue_size:0, recv_req_cnt:31029537, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1724705543331737}{group_id:19, queue_size:0, recv_req_cnt:1325443, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061845553743}{group_id:16, queue_size:0, recv_req_cnt:18, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269721623294}{group_id:1, queue_size:0, recv_req_cnt:509698616, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061848848830}, rpc_stat_info: pcode=0x1502:cnt=406 pcode=0x1501:cnt=199 pcode=0x717:cnt=195 pcode=0x1516:cnt=185 pcode=0x150b:cnt=120, token_change_ts:1725266206494273, tenant_role:1}) [2024-10-16 14:57:29.282066] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=479] dump throttled time info(id_=1004, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 21, group: OBCG_UNLOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 16, group: OBCG_TRANSFER, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1004, tenant_throttled_time: 0;) [2024-10-16 14:57:29.283359] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.299748] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=4] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:29.300477] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985CC-0-0] [lt=11] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=315) [2024-10-16 14:57:29.300581] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061849299596000, start_id=1729061849299596000, end_id=1729061849299596001) [2024-10-16 14:57:29.304277] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.307029] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061849306105000, start_id=1729061849306105000, end_id=1729061849306105001) [2024-10-16 14:57:29.313627] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6083, mem_used=12478464) [2024-10-16 14:57:29.315241] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B20-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061849315165000, start_id=1729061849315165000, end_id=1729061849315165001) [2024-10-16 14:57:29.317845] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [26251][DDLScanTask][T1][YB4A0A000407-00061E9981EFDF1D-0-0] [lt=20] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12208, add_timestamp:1729061849317837}) [2024-10-16 14:57:29.318263] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061849317252000, start_id=1729061849317252000, end_id=1729061849317252001) [2024-10-16 14:57:29.321480] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [26251][DDLScanTask][T1][YB4A0A000407-00061E9981EFDF1E-0-0] [lt=17] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12255, add_timestamp:1729061849321477}) [2024-10-16 14:57:29.321589] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=18] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:29.324691] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E17-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12208, locations=[{table_id:12208, partition_id:12208, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061849324680, sql_renew_time:1729061849324680, is_mark_fail:false}]) [2024-10-16 14:57:29.324709] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E17-0-0] [lt=17] success to process renew task(task={tenant_id:1, table_id:12208, add_timestamp:1729061849317837}, locations=[{table_id:12208, partition_id:12208, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061849324680, sql_renew_time:1729061849324680, is_mark_fail:false}]) [2024-10-16 14:57:29.324727] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E18-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12255, locations=[{table_id:12255, partition_id:12255, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061849324724, sql_renew_time:1729061849324724, is_mark_fail:false}]) [2024-10-16 14:57:29.324737] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E18-0-0] [lt=9] success to process renew task(task={tenant_id:1, table_id:12255, add_timestamp:1729061849321477}, locations=[{table_id:12255, partition_id:12255, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061849324724, sql_renew_time:1729061849324724, is_mark_fail:false}]) [2024-10-16 14:57:29.335745] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=304, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:29.335759] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=752, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:29.335769] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:29.336657] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0A1-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:29.336667] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0A1-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1086, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:29.336671] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0A1-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=20, avg_log_submit_to_flush_time=326, avg_log_submit_to_slide_time=1066) [2024-10-16 14:57:29.337574] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0A2-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:29.337590] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0A2-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1786, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:29.351344] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA7-0-0] [lt=12] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:29.351363] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA7-0-0] [lt=15] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=24) [2024-10-16 14:57:29.353029] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE767F-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:29.353282] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:29.356542] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:29.358495] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789D9-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061849357864000, start_id=1729061849357864000, end_id=1729061849357864001) [2024-10-16 14:57:29.364961] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B5B-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:29.374505] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:29.383385] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.389890] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:29.398504] INFO [SERVER] refresh_avail_memory (ob_table_load_assigned_memory_manager.cpp:117) [22703][T1004_TntShared][T1004][Y0-0000000000000000-0-0] [lt=15] ObTableLoadAssignedMemoryManager::refresh_avail_memory(MTL_ID()=1004, avail_memory=53687091, chunk_count_=0, avail_sort_memory_=53687091, avail_memory_=53687091) [2024-10-16 14:57:29.404343] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.415806] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B7-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:29.415833] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B7-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061849133226002, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061849133226002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:29.416843] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9B7-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061849416200000, start_id=1729061849416200000, end_id=1729061849416200001) [2024-10-16 14:57:29.417573] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:29.417797] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9A279FA9B7-0-0] [lt=24] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.054650688171386719e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=302, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.073993929144144658e+03) [2024-10-16 14:57:29.419356] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9B7-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061849418346000, start_id=1729061849418346000, end_id=1729061849418346001) [2024-10-16 14:57:29.419977] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:29.423971] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=25] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:29.435031] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:29.437166] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CDC-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:29.437187] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CDC-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:29.437196] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CDC-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:29.454124] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061839452090, end_time=1729061849454122, dag_status={}) [2024-10-16 14:57:29.454148] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=21] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:29.454157] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:29.454165] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:29.454169] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:29.454172] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:29.454177] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:29.454182] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:29.454185] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:29.454191] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:29.454195] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454205] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454209] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454216] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454220] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454228] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454233] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454243] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454253] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454259] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454263] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454268] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454276] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454280] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454284] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454288] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454292] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454296] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454302] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454308] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454312] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454316] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454320] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454323] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454327] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454331] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454335] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454339] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454342] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454346] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454350] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454354] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454357] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454361] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454365] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454369] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454373] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454380] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454385] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454391] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454404] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454407] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454411] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454415] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454419] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454422] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454426] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454430] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454436] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454442] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454447] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454453] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454458] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454464] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454470] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454483] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:29.454487] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:29.454491] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:29.454495] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:29.454497] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:29.454501] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:29.454505] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:29.454508] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:29.454512] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:29.454514] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:29.454517] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:29.454520] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:29.457185] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A73-0-0] [lt=11] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:29.457204] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A73-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:29.459415] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061849458399000, start_id=1729061849458399000, end_id=1729061849458399001) [2024-10-16 14:57:29.465470] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061849464844000, start_id=1729061849464844000, end_id=1729061849464844001) [2024-10-16 14:57:29.465876] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21810][T1001_TntShared][T1001][Y0-0000000000000000-0-0] [lt=11] clear dtl interm result cost(us)(clear_cost=16, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:29.468645] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8BE-0-0] [lt=7] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:29.468665] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8BE-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:29.468681] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8BE-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:29.471487] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061849469092000, start_id=1729061849469092000, end_id=1729061849469092001) [2024-10-16 14:57:29.474640] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D2-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061849465837, keepalive_ts:-1}) [2024-10-16 14:57:29.474657] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D2-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061849465837, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:29.475930] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:29.476309] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:29.476325] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:29.476611] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=401, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:29.476620] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2194) [2024-10-16 14:57:29.476628] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=50, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:29.476633] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2194) [2024-10-16 14:57:29.476648] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:29.476788] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C417D-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:29.476794] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C417D-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=531, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:29.483442] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.484635] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:29.484647] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=12] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:29.484651] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECE-0-0] [lt=9] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:29.484661] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:29.484668] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:29.484664] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECE-0-0] [lt=11] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:29.484673] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:29.484680] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:29.484682] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:29.484685] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:29.484687] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:29.484690] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:29.484693] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:29.489385] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:29.489400] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:29.489412] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70555425, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:29.491907] INFO [STORAGE.TRANS] refresh_elr_tenant_config_ (ob_tx_elr_util.cpp:47) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C83-0-0] [lt=15] refresh tenant config success(tenant_id=1004, *this={last_refresh_ts:1729061849491180, can_tenant_elr:true}) [2024-10-16 14:57:29.492089] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C83-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061849491180000, start_id=1729061849491180000, end_id=1729061849491180001) [2024-10-16 14:57:29.496740] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C44-0-0] [lt=10] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:29.496765] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C44-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061849131137003, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061849131137003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:29.497783] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C44-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061849497677000, start_id=1729061849497677000, end_id=1729061849497677001) [2024-10-16 14:57:29.500144] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C44-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061849499802000, start_id=1729061849499802000, end_id=1729061849499802001) [2024-10-16 14:57:29.504634] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.505085] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:29.505752] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FA4-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=380, avg_log_submit_to_slide_time=733) [2024-10-16 14:57:29.507124] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061849506855000, start_id=1729061849506855000, end_id=1729061849506855001) [2024-10-16 14:57:29.518354] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061849517929000, start_id=1729061849517929000, end_id=1729061849517929001) [2024-10-16 14:57:29.518894] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=9] [APPLY STAT CB APPEND COST TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=49, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:29.518905] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=1081, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:29.518910] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:29.518913] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=-843, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:29.518916] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=287, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:29.522664] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=8] replay status is not follower(min_unreplayed_lsn={lsn:7385763025}, base_lsn={lsn:7314419712}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:29.522717] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=53] get_replay_process success(id={id:1}, replayed_log_size=71343313, unreplayed_log_size=0) [2024-10-16 14:57:29.522726] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:57923794664}, base_lsn={lsn:57911414784}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:29.522750] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=23] get_replay_process success(id={id:1001}, replayed_log_size=12379880, unreplayed_log_size=0) [2024-10-16 14:57:29.522755] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=152596825791, unreplayed_log_size=0) [2024-10-16 14:57:29.522759] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=2] get_replay_process success(id={id:1003}, replayed_log_size=149812327465, unreplayed_log_size=0) [2024-10-16 14:57:29.522774] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1002, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=288479, last_replayed_log_size(MB)=288478, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:29.523742] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061849523209000, start_id=1729061849523209000, end_id=1729061849523209001) [2024-10-16 14:57:29.523758] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] ts response task statistics(alloc_count=261363950, free_count=130681974) [2024-10-16 14:57:29.523767] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=4] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:29.523776] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5] gts statistics(tenant_id=1001, gts_rpc_cnt=11, get_gts_cache_cnt=502, get_gts_with_stc_cnt=78, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=76, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:29.523927] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=521, get_gts_with_stc_cnt=5, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:29.556792] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:29.565157] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B5F-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178334, log_task={header:{begin_lsn:{lsn:14538473271}, end_lsn:{lsn:14538473393}, log_id:115178334, min_scn:{val:1729061849131137003, v:0}, max_scn:{val:1729061849131137003, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538473149}, prev_proposal_id:23, committed_end_lsn:{lsn:14538473271}, data_checksum:1715074492, accum_checksum:4280235747, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061849465127, freeze_ts:1729061849465127, submit_ts:1729061849465132, flushed_ts:1729061849465800, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:668}, fs_cb_begin_ts=1729061849565144, log_life_time=100017) [2024-10-16 14:57:29.569179] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.569571] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.569617] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.569712] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.576967] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:29.576983] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1035, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:29.581326] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=17] BlockGCTimerTask success(ret=0, cost_time_us=7, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:29.582192] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1002}, {id:1003}]) [2024-10-16 14:57:29.583491] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.586612] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:29.586625] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:29.589728] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21679][T1_L0_G0][T1][YB4A0A000409-00061E9A11478B2A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061849589626000, start_id=1729061849589626000, end_id=1729061849589626001) [2024-10-16 14:57:29.589836] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D3-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061849557189, keepalive_ts:-1}) [2024-10-16 14:57:29.589846] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D3-0-0] [lt=9] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061849557189, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:29.592511] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1DFD-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061849591740000, start_id=1729061849591740000, end_id=1729061849591740001) [2024-10-16 14:57:29.598563] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=27] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4732, mem_used=10398720) [2024-10-16 14:57:29.604674] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.632430] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BDF-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061849631876000, start_id=1729061849631876000, end_id=1729061849631876001) [2024-10-16 14:57:29.632430] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BDD-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061849631876000, start_id=1729061849631876000, end_id=1729061849631876001) [2024-10-16 14:57:29.632685] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BE1-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061849631876000, start_id=1729061849631876000, end_id=1729061849631876001) [2024-10-16 14:57:29.632710] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BDE-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061849631876000, start_id=1729061849631876000, end_id=1729061849631876001) [2024-10-16 14:57:29.632773] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BE0-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061849631876000, start_id=1729061849631876000, end_id=1729061849631876001) [2024-10-16 14:57:29.655980] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=6] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:29.659629] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061849659617000, start_id=1729061849659617000, end_id=1729061849659617001) [2024-10-16 14:57:29.665575] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061849663815000, start_id=1729061849663815000, end_id=1729061849663815001) [2024-10-16 14:57:29.665650] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=31] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:29.665667] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:29.668047] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:29.670558] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EB3-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:29.672024] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061849671746000, start_id=1729061849671746000, end_id=1729061849671746001) [2024-10-16 14:57:29.675993] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:29.676430] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2072, avg_group_log_size=207) [2024-10-16 14:57:29.676443] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2072) [2024-10-16 14:57:29.682878] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D61-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:29.682907] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D61-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061849131137004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061849131137004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:29.683515] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.683620] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=16] LogLoopThread round_cost_time(us)(round_cost_time=6) [2024-10-16 14:57:29.686622] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D61-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061849686478000, start_id=1729061849686478000, end_id=1729061849686478001) [2024-10-16 14:57:29.687548] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=17] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:29.696473] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=17] gc stale ls task succ [2024-10-16 14:57:29.696568] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:29.697356] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061849697112000, start_id=1729061849697112000, end_id=1729061849697112001) [2024-10-16 14:57:29.697425] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=5060, avg_group_log_size=421) [2024-10-16 14:57:29.697655] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=249, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:29.697674] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=85, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:29.704757] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.707257] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061849705515000, start_id=1729061849705515000, end_id=1729061849705515001) [2024-10-16 14:57:29.709104] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B1D-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061849708569000, start_id=1729061849708569000, end_id=1729061849708569001) [2024-10-16 14:57:29.715651] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=11] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:29.715680] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=18] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=210) [2024-10-16 14:57:29.717754] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061849517929000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199818, in_cluster_service=true, cluster_version={val:1729061849317252000, v:0}, min_cluster_version={val:1729061849317252000, v:0}, max_cluster_version={val:1729061849317252000, v:0}, get_cluster_version_err=0, cluster_version_delta=400495, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65248780, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=346990462, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:29.718698] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061849718058000, start_id=1729061849718058000, end_id=1729061849718058001) [2024-10-16 14:57:29.723256] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:29.734972] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:29.735003] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:29.735225] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2617) [2024-10-16 14:57:29.735522] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0AD-0-0] [lt=29] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=909, avg_log_submit_to_slide_time=1613) [2024-10-16 14:57:29.737948] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:29.737967] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:29.755318] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:29.755347] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=17] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:29.755358] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:29.757043] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:29.765258] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B63-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:29.765271] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B63-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40488, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:29.765275] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B63-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=308, avg_log_submit_to_slide_time=40482) [2024-10-16 14:57:29.765315] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:29.765442] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:29.770572] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:29.770620] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EB5-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:29.770641] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EB5-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:29.770867] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=426, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:29.770886] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=111, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:29.770918] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:29.773497] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15D6-0-0] [lt=4] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:29.776902] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3FA3-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=282, avg_log_submit_to_slide_time=452) [2024-10-16 14:57:29.780965] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:29.783579] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.787880] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=14] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:57:29.788458] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.788477] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.788543] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.788545] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.788560] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.788575] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.788590] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.788608] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.788614] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.788622] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.789617] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:29.804655] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=17] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061845798915, last_first_register_time_us=1722494157071428, register_time_us=1729061845798915, curr_time_us=1729061849804654) [2024-10-16 14:57:29.804689] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061849804671, last_submit_register_req_time_us=1729061849804671, register_parent_reason=3) [2024-10-16 14:57:29.804697] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:29.804817] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.806909] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:29.824193] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1001}, {id:1003}]) [2024-10-16 14:57:29.826927] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=10] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:29.827514] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:29.827562] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061844396711000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:29.827676] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=17] ====== check clog disk timer task ====== [2024-10-16 14:57:29.827683] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:29.827693] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=52291635, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:29.842694] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:29.842710] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=10491, avg_group_log_size=524) [2024-10-16 14:57:29.842723] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10491) [2024-10-16 14:57:29.842729] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:29.851311] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3705, mem_used=8318976) [2024-10-16 14:57:29.853168] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7689-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:29.853189] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7689-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:29.853200] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7689-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20075, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:29.853205] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7689-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938149, log_task={header:{begin_lsn:{lsn:213058222003}, end_lsn:{lsn:213058222125}, log_id:220938149, min_scn:{val:1729061849631876001, v:0}, max_scn:{val:1729061849631876001, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058221881}, prev_proposal_id:88, committed_end_lsn:{lsn:213058222003}, data_checksum:1715074492, accum_checksum:2427607558, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061849753132, freeze_ts:1729061849753132, submit_ts:1729061849753136, flushed_ts:1729061849753375, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:239}, fs_cb_begin_ts=1729061849853183, log_life_time=100051) [2024-10-16 14:57:29.853238] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:29.859615] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061849858807000, start_id=1729061849858807000, end_id=1729061849858807001) [2024-10-16 14:57:29.859909] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=456, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:29.859924] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10491) [2024-10-16 14:57:29.859934] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=208, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:29.859951] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10491) [2024-10-16 14:57:29.859966] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=10, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:29.861067] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5763-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:29.861082] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5763-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=1088, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:29.861086] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5763-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=13, avg_log_gen_to_submit_time=41, avg_log_submit_to_flush_time=798, avg_log_submit_to_slide_time=1046) [2024-10-16 14:57:29.864828] INFO [ARCHIVE] produce_log_fetch_task_ (ob_archive_sequencer.cpp:171) [22242][T1002_ArcSeq][T1002][YB4A0A000407-00061E996F4960C1-0-0] [lt=16] archive round not in doing status, just skip(key={incarnation:-1, dest_id:-1, round:-1}, state={status:"INVALID"}) [2024-10-16 14:57:29.865808] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061849865163000, start_id=1729061849865163000, end_id=1729061849865163001) [2024-10-16 14:57:29.871413] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=15] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1004, min_version={val:1729061849133226004, v:0}, base_version={val:1729061849131137002, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061849523209000, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061849871402, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061849133226004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061849777874, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061849491180000, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061849800140, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:29.872034] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=42] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:29.872172] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061849870411000, start_id=1729061849870411000, end_id=1729061849870411001) [2024-10-16 14:57:29.872726] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=10] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:29.872787] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061841826930000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:29.873346] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=34489751, this={dio_aligned_buf:{buf_write_offset:1553, buf_padding_size:2543, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111299574, truncate_used_ts:40614800}, log_block_size:67108864, dir_fd:182, io_fd:127}, aligned_buf_len=4096, aligned_block_offset=34488320, buf_len=122, total_write_size_=533325, total_write_size_after_dio_=2039808, ob_pwrite_used_ts=137827, count=369, each_pwrite_cost=373) [2024-10-16 14:57:29.876034] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:29.883639] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.900572] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:29.904894] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.907478] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061849907393000, start_id=1729061849907393000, end_id=1729061849907393001) [2024-10-16 14:57:29.918776] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061849918031000, start_id=1729061849918031000, end_id=1729061849918031001) [2024-10-16 14:57:29.921089] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 142.75; ) [2024-10-16 14:57:29.921097] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:29.921110] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:29.921148] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 178.25; ) [2024-10-16 14:57:29.921158] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:29.921166] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:29.935362] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=313, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:29.935383] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=50, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:29.937828] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=318, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:29.937847] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:29.937858] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:29.937884] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:29.937933] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0B5-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:29.937947] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:29.937958] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0B5-0-0] [lt=24] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=825, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:29.937963] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0B5-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=355, avg_log_submit_to_slide_time=817) [2024-10-16 14:57:29.937962] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656834, miss_cnt:17, cache_read_size:8050150450, hit_ratio:"9.999997286809066965e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:29.939556] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=2414, mem_used=6239232) [2024-10-16 14:57:29.943105] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26142][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB25CC6B8-0-0] [lt=21] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:57:29.957315] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:29.965271] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B67-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:29.965292] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B67-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178338, log_task={header:{begin_lsn:{lsn:14538473759}, end_lsn:{lsn:14538473881}, log_id:115178338, min_scn:{val:1729061849631876002, v:0}, max_scn:{val:1729061849631876002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538473637}, prev_proposal_id:23, committed_end_lsn:{lsn:14538473759}, data_checksum:1715074492, accum_checksum:505095774, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061849865243, freeze_ts:1729061849865244, submit_ts:1729061849865248, flushed_ts:1729061849865804, gen_to_freeze cost time:1, gen_to_submit cost time:5, submit_to_flush cost time:556}, fs_cb_begin_ts=1729061849965287, log_life_time=100044) [2024-10-16 14:57:29.965349] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650097, miss_cnt:32, cache_read_size:8050402966, hit_ratio:"9.999994892269096214e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:29.965492] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=201, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:29.965512] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=127, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:29.965543] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:29.970082] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601340, offset=9611902}, end_cursor:ObLogCursor{file_id=192, log_id=47601341, offset=9612186}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bf4050, len:1410, log_data_len:284, seq:47601340, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:29.971835] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22067][T1002_TeRec][T1002][YB4A0A000407-00061E9B225FA498-0-0] [lt=18] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=0, MAX_GAP=900000000000, new_sync_scn={val:1729061848522922003, v:0}, new_replayable_scn={val:1729061848522922003, v:0}, new_readable_scn={val:1729061848522922003, v:0}, old_tenant_info={tenant_id:1002, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061847520135003, v:0}, replayable_scn:{val:1729061847520135003, v:0}, standby_scn:{val:1729061847520135003, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"NOARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:57:29.972016] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA498-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061849971875000, start_id=1729061849971875000, end_id=1729061849971875001) [2024-10-16 14:57:29.972158] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061849972157, bytes: 4446808670626, bw: 0.375110 MB/s, add_ts: 1001641, add_bytes: 393977 [2024-10-16 14:57:29.972231] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=52] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:29.972246] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598746, miss_cnt:9, cache_read_size:149777897642, hit_ratio:"9.999999345924314165e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:29.973008] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601341, offset=9612186}, end_cursor:ObLogCursor{file_id=192, log_id=47601342, offset=9612460}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bc0050, len:1126, log_data_len:274, seq:47601341, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:29.974958] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601342, offset=9612460}, end_cursor:ObLogCursor{file_id=192, log_id=47601343, offset=9612734}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b58050, len:852, log_data_len:274, seq:47601342, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:29.975086] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601343, offset=9612734}, end_cursor:ObLogCursor{file_id=192, log_id=47601344, offset=9613008}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b24050, len:578, log_data_len:274, seq:47601343, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:29.975130] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561161, miss_cnt:1, cache_read_size:152590662351, hit_ratio:"9.999999928346827716e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:29.976640] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:29.976667] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:29.976808] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=29] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=210, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:29.976822] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=29, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:29.976831] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:29.982160] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061849982159, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007615, add_bytes: 0 [2024-10-16 14:57:29.983697] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:29.985236] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC733-0-0] [lt=14] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:29.985251] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC733-0-0] [lt=12] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:29.987174] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257706, offset=39017225}, end_cursor:ObLogCursor{file_id=197, log_id=49257707, offset=39017508}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2acc050, len:1271, log_data_len:283, seq:49257706, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:29.987575] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA874-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:29.987594] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA874-0-0] [lt=9] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061849632934002, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061849632934002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:29.988564] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA874-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061849988185000, start_id=1729061849988185000, end_id=1729061849988185001) [2024-10-16 14:57:29.989108] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB363-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061849988185000, start_id=1729061849988185000, end_id=1729061849988185001) [2024-10-16 14:57:29.991334] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257707, offset=39017508}, end_cursor:ObLogCursor{file_id=197, log_id=49257708, offset=39017790}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2aec050, len:988, log_data_len:282, seq:49257707, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:29.991440] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA874-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061849990710000, start_id=1729061849990710000, end_id=1729061849990710001) [2024-10-16 14:57:29.992846] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=28] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:29.993619] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257708, offset=39017790}, end_cursor:ObLogCursor{file_id=197, log_id=49257709, offset=39018072}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d18050, len:706, log_data_len:282, seq:49257708, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:29.993813] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257709, offset=39018072}, end_cursor:ObLogCursor{file_id=197, log_id=49257710, offset=39018354}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b28050, len:424, log_data_len:282, seq:49257709, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:30.004958] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.015743] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:57:30.015844] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DA9-0-0] [lt=14] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061849990710000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061865590307000, latest_log_ts:{val:1729061848964072001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:30.015907] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DA9-0-0] [lt=58] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:30.017873] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=5][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:30.021711] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061850021269000, start_id=1729061850021269000, end_id=1729061850021269001) [2024-10-16 14:57:30.021794] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:30.021813] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=5058) [2024-10-16 14:57:30.021850] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=36] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:30.022616] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=5] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061849718058000, v:0}, new_max_version={val:1729061849718058000, v:0}, last_min_version={val:1729061849589626000, v:0}, last_max_version={val:1729061849589626000, v:0}, total_time_us=1974, delta_us=304556) [2024-10-16 14:57:30.022638] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061849718058000, v:0}, min_version={val:1729061849718058000, v:0}, max_version={val:1729061849718058000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:30.024078] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061850023397000, start_id=1729061850023397000, end_id=1729061850023397001) [2024-10-16 14:57:30.024111] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=534, get_gts_with_stc_cnt=81, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=77, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:30.024131] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=19] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061850024107], gts:1729061850023397000, latest_srr:[mts=1729061850024107]}) [2024-10-16 14:57:30.035157] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:30.035182] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:30.035528] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:30.035566] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2070) [2024-10-16 14:57:30.035846] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=2] [APPLY STAT CB APPEND COST TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=51, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:57:30.035864] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=17] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=917, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:57:30.035868] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:57:30.035871] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=-944, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:57:30.035874] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=24, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:57:30.040814] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB6-0-0] [lt=3] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10009, add_timestamp:1729061850040801}) [2024-10-16 14:57:30.040844] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E19-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10009, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850040828, sql_renew_time:1729061850040828, is_mark_fail:false}]) [2024-10-16 14:57:30.040870] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E19-0-0] [lt=26] success to process renew task(task={tenant_id:1, table_id:10009, add_timestamp:1729061850040801}, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850040828, sql_renew_time:1729061850040828, is_mark_fail:false}]) [2024-10-16 14:57:30.041269] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB6-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317909662, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410527, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.041660] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2634-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11039, add_timestamp:1729061850041651}) [2024-10-16 14:57:30.041675] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1A-0-0] [lt=9] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11039, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:30.042104] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1A-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11039, locations=[{table_id:11039, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850042088, sql_renew_time:0, is_mark_fail:false}, {table_id:11039, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850042088, sql_renew_time:0, is_mark_fail:false}, {table_id:11039, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850042088, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:30.042130] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1A-0-0] [lt=26] success to process renew task(task={tenant_id:1, table_id:11039, add_timestamp:1729061850041651}, locations=[{table_id:11039, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850042088, sql_renew_time:0, is_mark_fail:false}, {table_id:11039, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850042088, sql_renew_time:0, is_mark_fail:false}, {table_id:11039, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850042088, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:30.046302] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CB7-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317914694, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410528, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.048089] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CBA-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317916483, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410529, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.049970] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F59-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061850049883000, start_id=1729061850049883000, end_id=1729061850049883001) [2024-10-16 14:57:30.052117] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CBB-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317920510, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410530, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.054257] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE768D-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:30.055864] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF96-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317924257, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410531, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.058190] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CBD-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317926584, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410532, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.059879] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061850058615000, start_id=1729061850058615000, end_id=1729061850058615001) [2024-10-16 14:57:30.061453] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CBF-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317929847, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410533, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.061729] WDIAG [LIB.TIME] str_to_digit_with_date (ob_time_convert.cpp:1928) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=13][errcode=-4219] datetime format too short(ret=-4219, str=0) [2024-10-16 14:57:30.061744] WDIAG [LIB.TIME] str_to_ob_time_with_date (ob_time_convert.cpp:1990) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=14][errcode=-4219] failed to get digits(ret=-4219, str=0) [2024-10-16 14:57:30.061749] WDIAG [LIB.TIME] str_to_datetime (ob_time_convert.cpp:398) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=4][errcode=-4219] failed to convert string to datetime(ret=-4219) [2024-10-16 14:57:30.061765] WDIAG [LIB.TIME] str_to_digit_with_date (ob_time_convert.cpp:1928) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=4][errcode=-4219] datetime format too short(ret=-4219, str=0) [2024-10-16 14:57:30.061768] WDIAG [LIB.TIME] str_to_ob_time_with_date (ob_time_convert.cpp:1990) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=3][errcode=-4219] failed to get digits(ret=-4219, str=0) [2024-10-16 14:57:30.061778] WDIAG [LIB.TIME] str_to_datetime (ob_time_convert.cpp:398) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] failed to convert string to datetime(ret=-4219) [2024-10-16 14:57:30.061793] WDIAG [LIB.TIME] str_to_digit_with_date (ob_time_convert.cpp:1928) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] datetime format too short(ret=-4219, str=0) [2024-10-16 14:57:30.061796] WDIAG [LIB.TIME] str_to_ob_time_with_date (ob_time_convert.cpp:1990) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=3][errcode=-4219] failed to get digits(ret=-4219, str=0) [2024-10-16 14:57:30.061798] WDIAG [LIB.TIME] str_to_datetime (ob_time_convert.cpp:398) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] failed to convert string to datetime(ret=-4219) [2024-10-16 14:57:30.061803] WDIAG [LIB.TIME] str_to_digit_with_date (ob_time_convert.cpp:1928) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] datetime format too short(ret=-4219, str=0) [2024-10-16 14:57:30.061805] WDIAG [LIB.TIME] str_to_ob_time_with_date (ob_time_convert.cpp:1990) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=3][errcode=-4219] failed to get digits(ret=-4219, str=0) [2024-10-16 14:57:30.061808] WDIAG [LIB.TIME] str_to_datetime (ob_time_convert.cpp:398) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] failed to convert string to datetime(ret=-4219) [2024-10-16 14:57:30.061811] WDIAG [LIB.TIME] str_to_digit_with_date (ob_time_convert.cpp:1928) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] datetime format too short(ret=-4219, str=0) [2024-10-16 14:57:30.061814] WDIAG [LIB.TIME] str_to_ob_time_with_date (ob_time_convert.cpp:1990) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] failed to get digits(ret=-4219, str=0) [2024-10-16 14:57:30.061816] WDIAG [LIB.TIME] str_to_datetime (ob_time_convert.cpp:398) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] failed to convert string to datetime(ret=-4219) [2024-10-16 14:57:30.061820] WDIAG [LIB.TIME] str_to_digit_with_date (ob_time_convert.cpp:1928) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] datetime format too short(ret=-4219, str=0) [2024-10-16 14:57:30.061827] WDIAG [LIB.TIME] str_to_ob_time_with_date (ob_time_convert.cpp:1990) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=6][errcode=-4219] failed to get digits(ret=-4219, str=0) [2024-10-16 14:57:30.061830] WDIAG [LIB.TIME] str_to_datetime (ob_time_convert.cpp:398) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC0-0-0] [lt=2][errcode=-4219] failed to convert string to datetime(ret=-4219) [2024-10-16 14:57:30.063092] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC1-0-0] [lt=2][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317931487, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410534, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.064365] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC4-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317932760, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410535, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.064524] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC5-0-0] [lt=17] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11001, add_timestamp:1729061850064516}) [2024-10-16 14:57:30.065566] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC6-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317933963, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410536, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.067389] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CC9-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317935785, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410537, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.068410] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CCB-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317936806, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410538, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.068842] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1B-0-0] [lt=18] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11001, vtable_type={tenant_id:1, type:2}) [2024-10-16 14:57:30.069912] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061850069561000, start_id=1729061850069561000, end_id=1729061850069561001) [2024-10-16 14:57:30.069916] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=25] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:30.072831] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061850071866000, start_id=1729061850071866000, end_id=1729061850071866001) [2024-10-16 14:57:30.073059] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1B-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11001, locations=[{table_id:11001, partition_id:0, partition_cnt:1, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850073040, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:30.073076] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1B-0-0] [lt=16] success to process renew task(task={tenant_id:1, table_id:11001, add_timestamp:1729061850064516}, locations=[{table_id:11001, partition_id:0, partition_cnt:1, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850073040, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:30.073878] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EBB-0-0] [lt=13][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497299, log_task={header:{begin_lsn:{lsn:207589533201}, end_lsn:{lsn:207589533323}, log_id:218497299, min_scn:{val:1729061849632934003, v:0}, max_scn:{val:1729061849632934003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589533079}, prev_proposal_id:81, committed_end_lsn:{lsn:207589533201}, data_checksum:940009244, accum_checksum:347985974, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061849972170, freeze_ts:1729061849972170, submit_ts:1729061849972174, flushed_ts:1729061849972383, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:209}, fs_cb_begin_ts=1729061850073864, log_life_time=101694) [2024-10-16 14:57:30.076076] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:30.076550] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CCE-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317944944, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410539, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.077530] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:30.077566] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:30.081646] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD0-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317950040, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410540, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.083752] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.085959] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:30.086080] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC91E-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317954472, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410541, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.086447] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12358, add_timestamp:1729061850086437}) [2024-10-16 14:57:30.086478] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=22] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12396, add_timestamp:1729061850086477}) [2024-10-16 14:57:30.086682] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:30.086707] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=24] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:30.086733] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:30.086836] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061857008855000, latest_log_ts:{val:1729061840488170001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:30.086879] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=39] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:30.086888] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:57:30.086901] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061840488170001, v:0}) [2024-10-16 14:57:30.086905] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:30.086908] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:30.086916] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:30.086925] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:57:30.086932] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:30.086937] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:30.087525] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:30.087545] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=20] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.087557] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=11] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"TIMESTAMP":"2024-06-05 04:03:43.114853"}) [2024-10-16 14:57:30.087568] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.087575] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.087584] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.087588] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.087597] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.087615] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=18] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088593] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:30.088655] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=33] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:30.088667] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:30.088675] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:30.088685] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:30.088729] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:30.088736] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088743] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"TIMESTAMP":"2024-06-05 04:04:24.548679"}) [2024-10-16 14:57:30.088748] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088754] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088760] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088765] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088770] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088775] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088788] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:30.088793] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088798] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"TIMESTAMP":"2024-06-05 04:04:24.552444"}) [2024-10-16 14:57:30.088803] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088808] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088813] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088817] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088822] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088827] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088843] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:30.088850] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088856] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"TIMESTAMP":"2024-06-05 10:10:34.116037"}) [2024-10-16 14:57:30.088861] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088866] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088872] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088877] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088882] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088887] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088896] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:30.088903] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088908] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"TIMESTAMP":"2024-06-05 10:10:34.125347"}) [2024-10-16 14:57:30.088913] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088918] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088923] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088927] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:30.088932] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.088936] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD3-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:30.089246] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1C-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12358, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850089230, sql_renew_time:1729061850089230, is_mark_fail:false}]) [2024-10-16 14:57:30.089269] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1C-0-0] [lt=23] success to process renew task(task={tenant_id:1, table_id:12358, add_timestamp:1729061850086437}, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850089230, sql_renew_time:1729061850089230, is_mark_fail:false}]) [2024-10-16 14:57:30.089291] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1D-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12396, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850089286, sql_renew_time:1729061850089286, is_mark_fail:false}]) [2024-10-16 14:57:30.089304] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1D-0-0] [lt=12] success to process renew task(task={tenant_id:1, table_id:12396, add_timestamp:1729061850086477}, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850089286, sql_renew_time:1729061850089286, is_mark_fail:false}]) [2024-10-16 14:57:30.090289] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD4-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317958684, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410542, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.093831] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD7-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317962227, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410543, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.094030] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD8-0-0] [lt=14] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12450, add_timestamp:1729061850094021}) [2024-10-16 14:57:30.094050] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD8-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12095, add_timestamp:1729061850094047}) [2024-10-16 14:57:30.094059] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1E-0-0] [lt=10] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12450, locations=[{table_id:12450, partition_id:12450, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850094046, sql_renew_time:1729061850094046, is_mark_fail:false}]) [2024-10-16 14:57:30.094078] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1E-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:12450, add_timestamp:1729061850094021}, locations=[{table_id:12450, partition_id:12450, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850094046, sql_renew_time:1729061850094046, is_mark_fail:false}]) [2024-10-16 14:57:30.094093] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1F-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12095, locations=[{table_id:12095, partition_id:12095, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850094090, sql_renew_time:1729061850094090, is_mark_fail:false}]) [2024-10-16 14:57:30.094099] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E1F-0-0] [lt=6] success to process renew task(task={tenant_id:1, table_id:12095, add_timestamp:1729061850094047}, locations=[{table_id:12095, partition_id:12095, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061850094090, sql_renew_time:1729061850094090, is_mark_fail:false}]) [2024-10-16 14:57:30.098569] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CD9-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317966966, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410544, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.099949] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CDC-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317968345, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410545, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.104520] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CDE-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317972916, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410546, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.105016] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.105829] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CE0-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510317974226, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410547, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.107481] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061850106963000, start_id=1729061850106963000, end_id=1729061850106963001) [2024-10-16 14:57:30.108220] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get_max_applied_scn(scn={val:1729061849990710000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618402195}, palf_committed_end_scn_:{val:1729061850106963000, v:0}, last_check_scn_:{val:1729061850106963000, v:0}, max_applied_cb_scn_:{val:1729061849990710000, v:0}}) [2024-10-16 14:57:30.108244] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:30.110480] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:30.110492] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:30.125477] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=13] current monitor number(seq_=-1) [2024-10-16 14:57:30.127117] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:30.128572] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1029, mem_used=2079744) [2024-10-16 14:57:30.132750] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BE2-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061850132688000, start_id=1729061850132688000, end_id=1729061850132688001) [2024-10-16 14:57:30.132763] INFO [STORAGE.TRANS] handle_request (ob_timestamp_service.cpp:175) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BE2-0-0] [lt=11] handle gts request statistics(total_rt=4522, total_cnt=1530, avg_rt=2.953625081645983030e+00, avg_cnt=1.530000000000000000e+02) [2024-10-16 14:57:30.132843] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BE3-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061850132688000, start_id=1729061850132688000, end_id=1729061850132688001) [2024-10-16 14:57:30.132898] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BE6-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061850132688000, start_id=1729061850132688000, end_id=1729061850132688001) [2024-10-16 14:57:30.132914] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BE5-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061850132688000, start_id=1729061850132688000, end_id=1729061850132688001) [2024-10-16 14:57:30.133035] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BE4-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061850132688000, start_id=1729061850132688000, end_id=1729061850132688001) [2024-10-16 14:57:30.137653] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CEA-0-0] [lt=29][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173033, log_task={header:{begin_lsn:{lsn:14352360229}, end_lsn:{lsn:14352360351}, log_id:115173033, min_scn:{val:1729061849635314004, v:0}, max_scn:{val:1729061849635314004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352360107}, prev_proposal_id:17, committed_end_lsn:{lsn:14352360229}, data_checksum:940009244, accum_checksum:3048058395, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061850037583, freeze_ts:1729061850037583, submit_ts:1729061850037586, flushed_ts:1729061850038200, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:614}, fs_cb_begin_ts=1729061850137647, log_life_time=100064) [2024-10-16 14:57:30.143591] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:57:30.143609] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=18] try gc retain ctx [2024-10-16 14:57:30.143613] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:30.144228] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061865590307000, latest_log_ts:{val:1729061848964072001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:30.153514] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF97-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318021907, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410548, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.154995] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CE4-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318023390, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410549, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.156183] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE768F-0-0] [lt=24][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938152, log_task={header:{begin_lsn:{lsn:213058222369}, end_lsn:{lsn:213058222491}, log_id:220938152, min_scn:{val:1729061849631876004, v:0}, max_scn:{val:1729061849631876004, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058222247}, prev_proposal_id:88, committed_end_lsn:{lsn:213058222369}, data_checksum:1715074492, accum_checksum:2381378820, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061850054251, freeze_ts:1729061850054251, submit_ts:1729061850054272, flushed_ts:1729061850056141, gen_to_freeze cost time:0, gen_to_submit cost time:21, submit_to_flush cost time:1869}, fs_cb_begin_ts=1729061850156176, log_life_time=101925) [2024-10-16 14:57:30.157352] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=30] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=47, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:30.157390] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=38] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:30.157621] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:30.159765] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CE6-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318028160, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410550, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.159811] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=452, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:30.159824] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=284, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:30.160805] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF852-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318029191, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410551, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.160884] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:30.162417] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CEA-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318030813, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410552, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.167242] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CEC-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318035637, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410553, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.167468] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B6B-0-0] [lt=27] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:30.167499] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B6B-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178340, log_task={header:{begin_lsn:{lsn:14538474003}, end_lsn:{lsn:14538474125}, log_id:115178340, min_scn:{val:1729061849631876004, v:0}, max_scn:{val:1729061849631876004, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538473881}, prev_proposal_id:23, committed_end_lsn:{lsn:14538474003}, data_checksum:1715074492, accum_checksum:2559364904, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061850065969, freeze_ts:1729061850065969, submit_ts:1729061850065974, flushed_ts:1729061850073116, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:7142}, fs_cb_begin_ts=1729061850167487, log_life_time=101518) [2024-10-16 14:57:30.168903] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CEF-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318037299, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410554, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.170851] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EBD-0-0] [lt=24] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:30.170863] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EBD-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30314, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:30.171255] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061849870411000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=300838, in_cluster_service=true, cluster_version={val:1729061849865163000, v:0}, min_cluster_version={val:1729061849865163000, v:0}, max_cluster_version={val:1729061849865163000, v:0}, get_cluster_version_err=0, cluster_version_delta=306086, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244095, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=459241052, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:30.172670] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD002-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061850163792, keepalive_ts:-1}) [2024-10-16 14:57:30.172681] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD002-0-0] [lt=10] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061850163792, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:30.173948] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CF1-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318042342, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410555, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.175420] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10794, avg_group_log_size=568) [2024-10-16 14:57:30.175434] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10794) [2024-10-16 14:57:30.176415] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5020-0-0] [lt=14] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:30.176429] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5020-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:30.176465] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=338, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:30.176474] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10794) [2024-10-16 14:57:30.176483] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=176, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:30.176491] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10916) [2024-10-16 14:57:30.176508] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:30.176514] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=933, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:30.176519] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=4, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=640, avg_log_submit_to_slide_time=902) [2024-10-16 14:57:30.176531] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:30.176900] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:30.176908] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:30.177503] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CF5-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318045898, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410556, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.179239] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=749, avg_log_submit_to_slide_time=1275) [2024-10-16 14:57:30.181411] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CF7-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318049806, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410557, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.183822] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.184046] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CFA-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318052441, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410558, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.186077] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21674][T1_L0_G0][T1][YB4A0A000407-00061E9979BD0320-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318054471, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410559, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.187506] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3CFE-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318055902, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410560, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.192492] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D00-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318060887, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410561, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.194696] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D03-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318063091, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410562, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.196642] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D05-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318065038, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410563, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.199420] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D09-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318067815, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410564, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.203481] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D0B-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318071876, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410565, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.204790] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D0E-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318073186, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410566, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.205081] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.205193] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4937) [2024-10-16 14:57:30.205228] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4937) [2024-10-16 14:57:30.206114] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D10-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318074510, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410567, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.206193] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FAC-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:30.206202] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FAC-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=787, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:30.207626] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D14-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318076022, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410568, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.211776] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D16-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318080172, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410569, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.212989] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D19-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318081385, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410570, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.214046] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21666][T1_L0_G0][T1][YB4A0A000407-00061E9978CCFCB9-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318082441, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410571, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.216215] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D1E-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318084612, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410572, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.219430] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D20-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318087826, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410573, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.220641] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D23-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318089037, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410574, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.221664] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061850220237000, start_id=1729061850220237000, end_id=1729061850220237001) [2024-10-16 14:57:30.223111] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D25-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318091507, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410575, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.224639] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D29-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318093036, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410576, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.227647] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D2B-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318096043, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410577, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.229189] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D2E-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318097585, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410578, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.230305] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D30-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318098702, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410579, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.235264] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2070, avg_group_log_size=207) [2024-10-16 14:57:30.235278] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2070) [2024-10-16 14:57:30.235307] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:30.235312] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:30.235595] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:30.243913] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:468) [22402][T1003_SSTableDe][T1003][Y0-0000000000000000-0-0] [lt=8] skip defragment task(ret=0, need_defragment=false) [2024-10-16 14:57:30.243929] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [22402][T1003_SSTableDe][T1003][Y0-0000000000000000-0-0] [lt=14] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=9) [2024-10-16 14:57:30.243934] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [22402][T1003_SSTableDe][T1003][Y0-0000000000000000-0-0] [lt=5] successfully defragment data blocks(rewrite_cnt=0) [2024-10-16 14:57:30.243939] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [22402][T1003_SSTableDe][T1003][Y0-0000000000000000-0-0] [lt=3] tablet skip filter destructed(total_cnt=0, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105366175, total_skip_cnt=310289, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:57:30.259938] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061850259900000, start_id=1729061850259900000, end_id=1729061850259900001) [2024-10-16 14:57:30.272485] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061850271878000, start_id=1729061850271878000, end_id=1729061850271878001) [2024-10-16 14:57:30.272744] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061850271878000, start_id=1729061850271878001, end_id=1729061850271878002) [2024-10-16 14:57:30.272889] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD003-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061850264062, keepalive_ts:-1}) [2024-10-16 14:57:30.272905] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD003-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061850264062, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:30.274855] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=9] worker thread began to shrink(tenant_->id()=1003, group_id_=1, token=0) [2024-10-16 14:57:30.276154] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:30.283862] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.284109] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107139][T1003_L0_G1][T1003][Y0-0000000000000000-0-0] [lt=16] worker thread exit(tenant_->id()=1003, workers_.get_size()=2) [2024-10-16 14:57:30.284170] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107139][T1003_L0_G1][T0][Y0-0000000000000000-0-0] [lt=51] tenant thread end_run(id_=1003, ret=0, thread_count_=172) [2024-10-16 14:57:30.285267] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D39-0-0] [lt=20] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061850276473, keepalive_ts:-1}) [2024-10-16 14:57:30.285280] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D39-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061850276473, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:30.285445] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D38-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061850276417, keepalive_ts:-1}) [2024-10-16 14:57:30.285455] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107054][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D38-0-0] [lt=9] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061850276417, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:30.305146] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.306825] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=64] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061850106963000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199857, in_cluster_service=true, cluster_version={val:1729061849858807000, v:0}, min_cluster_version={val:1729061849858807000, v:0}, max_cluster_version={val:1729061849858807000, v:0}, get_cluster_version_err=0, cluster_version_delta=448013, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244345, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=224702991, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:30.307650] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061850306570000, start_id=1729061850306570000, end_id=1729061850306570001) [2024-10-16 14:57:30.311257] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:30.311274] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:30.311292] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373552947, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:30.313729] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=6153, mem_used=12478464) [2024-10-16 14:57:30.321665] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:30.337714] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:30.337732] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:30.337750] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:30.338588] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107144][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=192) [2024-10-16 14:57:30.338617] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107144][][T1004][Y0-0000000000000000-0-0] [lt=28] Init thread local success [2024-10-16 14:57:30.338651] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107144][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0BF-0-0] [lt=11] init point thread id with(&point=0x55e3435c5080, idx_=648, point=[thread id=107144, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107144) [2024-10-16 14:57:30.338674] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=650, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:30.338697] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=49, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:30.338723] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:30.338728] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1028, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:30.338729] INFO [LIB] stat (utility.h:1165) [107144][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD1E-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:30.338732] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:30.338735] INFO [LIB] stat (utility.h:1165) [107144][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD1E-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=886, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:30.338739] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107144][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD1E-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=22, avg_log_submit_to_flush_time=327, avg_log_submit_to_slide_time=864) [2024-10-16 14:57:30.339109] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107145][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=193) [2024-10-16 14:57:30.339123] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107145][][T1004][Y0-0000000000000000-0-0] [lt=13] Init thread local success [2024-10-16 14:57:30.339286] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107146][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=194) [2024-10-16 14:57:30.339302] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107146][][T1004][Y0-0000000000000000-0-0] [lt=15] Init thread local success [2024-10-16 14:57:30.339451] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107147][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=195) [2024-10-16 14:57:30.339458] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107147][][T1004][Y0-0000000000000000-0-0] [lt=6] Init thread local success [2024-10-16 14:57:30.339673] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107148][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=196) [2024-10-16 14:57:30.339674] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] worker thread created(tenant_->id()=1004, group_id_=1, token=1) [2024-10-16 14:57:30.339682] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107148][][T1004][Y0-0000000000000000-0-0] [lt=9] Init thread local success [2024-10-16 14:57:30.348209] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:30.348228] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:30.348241] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:30.353700] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:30.353961] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52C5-0-0] [lt=30] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1333) [2024-10-16 14:57:30.357909] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:30.365492] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B6F-0-0] [lt=4] init point thread id with(&point=0x55e3435c5100, idx_=650, point=[thread id=107146, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107146) [2024-10-16 14:57:30.365543] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B6F-0-0] [lt=27] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:30.375980] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22442][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=15][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:30.376061] WDIAG read_multi_row (ob_table_access_helper.h:238) [22442][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=19][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:30.376113] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] [APPLY STAT CB APPEND COST TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=43, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:57:30.376131] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=17] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=1326, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:57:30.376135] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:57:30.376138] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=-657, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:57:30.376142] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=43, stat_interval=5000000, avg (count/cost)=712, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:57:30.376827] INFO delete_row (ob_table_access_helper.h:312) [22442][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=8] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1003, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061850376069) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061850376069) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:30.380671] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=12] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:30.380690] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=212) [2024-10-16 14:57:30.383926] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.389908] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:30.396073] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=12] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:30.405212] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.421765] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3ACF84-0-0] [lt=12] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.738636016845703125e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=869, Throttle Trigger(MB)=521, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.115848223054147326e+01) [2024-10-16 14:57:30.421993] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=28] submit log success(service_type_=0, last_id=1729061867008855000, limited_id=1729061867008855000) [2024-10-16 14:57:30.422004] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] submit log with lock success(service_type_=0, last_id=1729061867008855000, limited_id=1729061867008855000) [2024-10-16 14:57:30.422008] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061850421032000, start_id=1729061850421032000, end_id=1729061850421032001) [2024-10-16 14:57:30.422078] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B8-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:30.422097] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B8-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061850133744001, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061850133744001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:30.422610] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061867008855000, latest_scn={val:1729061850421032001, v:0}, *this={tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061867008855000, latest_log_ts:{val:1729061850421032001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:30.422651] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=40] handle submit callback(service_type_=0, success=true, used_ts=658, log_ts={val:1729061850421032001, v:0}, limited_id=1729061867008855000) [2024-10-16 14:57:30.422658] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:30.422662] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=5] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:30.422754] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3CA9-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061850422089000, start_id=1729061850422089000, end_id=1729061850422089001) [2024-10-16 14:57:30.422860] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=5] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:30.422886] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get_max_applied_scn(scn={val:1729061850220237000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390078907}, palf_committed_end_scn_:{val:1729061850421032002, v:0}, last_check_scn_:{val:1729061850421032002, v:0}, max_applied_cb_scn_:{val:1729061850220237000, v:0}}) [2024-10-16 14:57:30.422909] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:30.423520] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9B8-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061850423113000, start_id=1729061850423113000, end_id=1729061850423113001) [2024-10-16 14:57:30.426524] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9B8-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061850426310000, start_id=1729061850426310000, end_id=1729061850426310001) [2024-10-16 14:57:30.427131] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=2] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:30.433203] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488A-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061850400965, keepalive_ts:-1}) [2024-10-16 14:57:30.433221] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488A-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061850400965, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:30.435578] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:30.437773] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CF0-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:30.437792] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CF0-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:30.437801] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CF0-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:30.456426] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=40] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:30.457110] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=19] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:30.457176] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061848719447000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:30.457937] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A74-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:30.457951] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A74-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:30.460028] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061850459049000, start_id=1729061850459049000, end_id=1729061850459049001) [2024-10-16 14:57:30.471167] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:30.471696] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=23] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:30.472856] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061850471860000, start_id=1729061850471860000, end_id=1729061850471860001) [2024-10-16 14:57:30.472956] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061850023397002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=449555, in_cluster_service=true, cluster_version={val:1729061849523209003, v:0}, min_cluster_version={val:1729061849523209003, v:0}, max_cluster_version={val:1729061849523209003, v:0}, get_cluster_version_err=0, cluster_version_delta=949743, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228126, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=395864473, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:30.473875] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=41] get number(ret=0, service_type_=0, range=1, base_id=1729061850472902000, start_id=1729061850472902000, end_id=1729061850472902001) [2024-10-16 14:57:30.476229] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:30.477043] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:30.477058] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:30.477286] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=255, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:30.477322] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=35] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2339) [2024-10-16 14:57:30.477334] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=159, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:30.477364] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2339) [2024-10-16 14:57:30.477370] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:30.477610] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C419B-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:30.477616] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C419B-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=775, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:30.484756] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.505282] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.509495] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061850508324000, start_id=1729061850508324000, end_id=1729061850508324001) [2024-10-16 14:57:30.512021] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C45-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:30.512051] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C45-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061850132688002, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061850132688002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:30.513139] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C45-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061850512976000, start_id=1729061850512976000, end_id=1729061850512976001) [2024-10-16 14:57:30.515559] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C45-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061850515095000, start_id=1729061850515095000, end_id=1729061850515095001) [2024-10-16 14:57:30.524474] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061850523518000, start_id=1729061850523518000, end_id=1729061850523518001) [2024-10-16 14:57:30.524498] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:30.524509] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061850524505], gts:1729061850523518000, latest_srr:[mts=1729061850524505]}) [2024-10-16 14:57:30.532221] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:57:30.532241] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:30.532260] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444466636, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:30.539439] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=904810496, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=168931328, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:30.539493] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=25] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:30.539504] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:30.539536] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=967602176, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=106139648, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:30.539580] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:30.539586] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:30.539624] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=940462080, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=133279744, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:30.539667] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:30.539673] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011857, drift_size_=0, pre_drift_size=1043719, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:30.539705] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=965230592, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=108511232, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:30.539742] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:30.539749] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:30.558202] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:30.559293] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061850132688002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=426600, in_cluster_service=true, cluster_version={val:1729061850049883001, v:0}, min_cluster_version={val:1729061850049883001, v:0}, max_cluster_version={val:1729061850049883001, v:0}, get_cluster_version_err=0, cluster_version_delta=509405, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244045, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=521889127, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:30.568711] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.569153] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.569543] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.569575] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.577656] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C419E-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:30.577671] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C419E-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=916, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:30.582301] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1}, {id:1003}]) [2024-10-16 14:57:30.584822] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.586835] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:30.586851] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:30.598652] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4753, mem_used=10398720) [2024-10-16 14:57:30.602523] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=16] BlockGCTimerTask success(ret=0, cost_time_us=7, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:30.605076] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=9] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:57:30.605088] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:57:30.605092] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:30.605681] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061867008855000, latest_log_ts:{val:1729061850421032001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:30.605879] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=23] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:30.605895] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:30.605901] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.606088] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=12, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:30.606634] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45B7A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=13, avg_log_gen_to_freeze_time=30, avg_log_gen_to_submit_time=62, avg_log_submit_to_flush_time=430, avg_log_submit_to_slide_time=698) [2024-10-16 14:57:30.621841] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=46] get number(ret=0, service_type_=0, range=1, base_id=1729061850621624000, start_id=1729061850621624000, end_id=1729061850621624001) [2024-10-16 14:57:30.633107] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BE7-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061850632114000, start_id=1729061850632114000, end_id=1729061850632114001) [2024-10-16 14:57:30.633729] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BEB-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061850633180000, start_id=1729061850633180000, end_id=1729061850633180001) [2024-10-16 14:57:30.633772] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BE9-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061850633180000, start_id=1729061850633180000, end_id=1729061850633180001) [2024-10-16 14:57:30.633798] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BEA-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061850633180000, start_id=1729061850633180000, end_id=1729061850633180001) [2024-10-16 14:57:30.633842] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BE8-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061850633180000, start_id=1729061850633180000, end_id=1729061850633180001) [2024-10-16 14:57:30.656204] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:30.660806] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061850658959000, start_id=1729061850658959000, end_id=1729061850658959001) [2024-10-16 14:57:30.666955] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D31-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510318535348, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410580, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:30.669273] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D35-0-0] [lt=16][errcode=-4201] hashtable not init [2024-10-16 14:57:30.669290] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D35-0-0] [lt=10][errcode=-4016] get cursor info failed(cursor_id=1111, get_sessid()=3221565766) [2024-10-16 14:57:30.669368] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D36-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061850668747000, start_id=1729061850668747000, end_id=1729061850668747001) [2024-10-16 14:57:30.671002] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EC7-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:30.672696] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061850671063000, start_id=1729061850671063000, end_id=1729061850671063001) [2024-10-16 14:57:30.672749] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:30.672762] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:30.673071] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=15] no need to update min/max version(new_version={val:1729061850023397002, v:0}, min_version={val:1729061850023397002, v:0}, max_version={val:1729061850023397002, v:0}) [2024-10-16 14:57:30.673864] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get_max_applied_scn(scn={val:1729061850548787001, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264615761}, palf_committed_end_scn_:{val:1729061850671063000, v:0}, last_check_scn_:{val:1729061850671063000, v:0}, max_applied_cb_scn_:{val:1729061850548787001, v:0}}) [2024-10-16 14:57:30.673886] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:30.676308] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:30.677149] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2339, avg_group_log_size=233) [2024-10-16 14:57:30.677162] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2339) [2024-10-16 14:57:30.684881] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.689362] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D62-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:30.689393] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D62-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061850132688004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061850132688004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:30.693234] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D62-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061850692257000, start_id=1729061850692257000, end_id=1729061850692257001) [2024-10-16 14:57:30.696533] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=24] gc stale ls task succ [2024-10-16 14:57:30.705928] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.706669] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22141][T1002_TntShared][T1002][Y0-0000000000000000-0-0] [lt=13] clear dtl interm result cost(us)(clear_cost=894, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:30.709568] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061850709220000, start_id=1729061850709220000, end_id=1729061850709220001) [2024-10-16 14:57:30.723353] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:30.735531] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:30.735547] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:30.735788] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:30.735806] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=212) [2024-10-16 14:57:30.736169] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107144][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD28-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=39, avg_log_submit_to_flush_time=672, avg_log_submit_to_slide_time=914) [2024-10-16 14:57:30.738445] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=44] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=54, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:30.738456] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:30.755395] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=2] gc stale ls task succ [2024-10-16 14:57:30.755425] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:30.755443] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=16] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:30.758526] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:30.765712] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B77-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:30.765728] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B77-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=29789, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:30.765733] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B77-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=976, avg_log_submit_to_slide_time=29783) [2024-10-16 14:57:30.765773] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:30.765898] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:30.770628] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:30.771343] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EC9-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:30.771362] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EC9-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:30.771567] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=29] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=307, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:30.771580] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=148, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:30.771636] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:30.774052] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061850772997000, start_id=1729061850772997000, end_id=1729061850772997001) [2024-10-16 14:57:30.777683] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F3FC0-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=507, avg_log_submit_to_slide_time=749) [2024-10-16 14:57:30.781084] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=14] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:30.784925] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.787884] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.787886] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.787900] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.787909] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.788477] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.788535] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.789188] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.789190] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.789208] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.789627] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.789629] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:30.805246] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=53] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=13, avg_log_batch_cnt=1, total_group_log_size=4804, avg_group_log_size=369) [2024-10-16 14:57:30.805553] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=329, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:30.805572] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=61, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:30.805982] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.808668] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=10] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:30.813420] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=10][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:30.813481] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=12][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1001 and flag like "%DUPLICATE%", tenant_id=1001, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:30.814972] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:30.815021] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1 and flag like "%DUPLICATE%", tenant_id=1, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:30.821822] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061850821071000, start_id=1729061850821071000, end_id=1729061850821071001) [2024-10-16 14:57:30.824305] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=23] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1001}, {id:1003}]) [2024-10-16 14:57:30.835808] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:30.844086] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=43] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:30.844103] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10325, avg_group_log_size=543) [2024-10-16 14:57:30.844113] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10325) [2024-10-16 14:57:30.844127] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:30.851409] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3766, mem_used=8318976) [2024-10-16 14:57:30.853802] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE769D-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:30.854265] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52CF-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:30.854275] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52CF-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27250, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:30.854277] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1333, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:30.860829] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061850860190000, start_id=1729061850860190000, end_id=1729061850860190001) [2024-10-16 14:57:30.861132] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=693, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:30.861144] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10324) [2024-10-16 14:57:30.861155] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=78, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:30.861176] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10325) [2024-10-16 14:57:30.861185] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=10, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:30.861362] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5776-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:30.861371] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5776-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=1097, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:30.861381] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5776-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=869, avg_log_submit_to_slide_time=1063) [2024-10-16 14:57:30.872873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1729061850871866000, start_id=1729061850871866000, end_id=1729061850871866001) [2024-10-16 14:57:30.874011] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:30.877892] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=1981791, replace_num=31457) [2024-10-16 14:57:30.877909] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:30.878742] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=14] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:57:30.884991] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.887106] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:30.887286] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D45-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061850523518002, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061847520135003, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860098662000, latest_log_ts:{val:1729061843439703001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:30.887424] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D45-0-0] [lt=94] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061850523518002, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061847520135003, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:30.887495] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D45-0-0] [lt=36] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061850633180002, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061847520135003, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:30.887562] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D45-0-0] [lt=35] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061850634239002, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061847520135003, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:30.887595] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D45-0-0] [lt=31] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:30.900253] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:30.900328] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1004 and flag like "%DUPLICATE%", tenant_id=1004, exec_tenant_id=1003, need_member_list=false) [2024-10-16 14:57:30.900639] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=15] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:30.901519] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=12] ====== tenant freeze timer task ====== [2024-10-16 14:57:30.906043] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.909780] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061850908407000, start_id=1729061850908407000, end_id=1729061850908407001) [2024-10-16 14:57:30.919957] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD6-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061850878360, keepalive_ts:-1}) [2024-10-16 14:57:30.919990] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD6-0-0] [lt=33] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061850878360, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:30.921193] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:30.922899] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 156.75; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 17408.00, iops: 3.99, rt: 32105.00; ) [2024-10-16 14:57:30.922917] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:30.922957] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 354.75; ) [2024-10-16 14:57:30.922963] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:30.922971] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:30.929767] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15D0-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:30.936210] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=263, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:30.936228] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=32, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:30.940096] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2451, mem_used=6239232) [2024-10-16 14:57:30.949228] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1218, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:30.949241] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:30.949254] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=32, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:30.949266] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:17493451876}], scn_array=[cnt:1, {val:1729061850545633002, v:0}], curr_size=122, accum_size=122, time_cost=10145) [2024-10-16 14:57:30.949318] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:30.949328] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=5] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:30.949335] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=5] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=147) [2024-10-16 14:57:30.949328] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:30.960354] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1173282816, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:30.963217] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Cache wash clean map node details(ret=0, clean_node_count=36, clean_time=0, clean_start_pos=692054, clean_num=62914) [2024-10-16 14:57:30.964646] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] no need to update min/max version(new_version={val:1729061850132688004, v:0}, min_version={val:1729061850132688004, v:0}, max_version={val:1729061850132688004, v:0}) [2024-10-16 14:57:30.966906] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B7B-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:30.968025] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=521, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:30.968042] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=492, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:30.968120] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:30.976016] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=26] PNIO [ratelimit] time: 1729061850976014, bytes: 4446809105523, bw: 0.413157 MB/s, add_ts: 1003857, add_bytes: 434897 [2024-10-16 14:57:30.976200] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:30.976215] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598757, miss_cnt:9, cache_read_size:149777898984, hit_ratio:"9.999999345924366345e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:30.977350] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:30.977362] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:30.978012] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=351, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:30.978025] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=258, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:30.978048] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:30.978102] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601344, offset=9613008}, end_cursor:ObLogCursor{file_id=192, log_id=47601345, offset=9613292}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c5c050, len:4400, log_data_len:284, seq:47601344, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:30.980620] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061850979299000, start_id=1729061850979299000, end_id=1729061850979299001) [2024-10-16 14:57:30.980846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA499-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061850979299000, start_id=1729061850979299000, end_id=1729061850979299001) [2024-10-16 14:57:30.982211] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061850982209, bytes: 0, bw: 0.000000 MB/s, add_ts: 1000050, add_bytes: 0 [2024-10-16 14:57:30.987727] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=3] ====== tenant freeze timer task ====== [2024-10-16 14:57:30.987742] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=26789460, tenant_id=1004) [2024-10-16 14:57:30.987756] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=12] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:30.989646] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:30.993847] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=40] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601345, offset=9613292}, end_cursor:ObLogCursor{file_id=192, log_id=47601346, offset=9613566}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b88050, len:4116, log_data_len:274, seq:47601345, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:30.996385] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA875-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:30.996405] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA875-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061850634239003, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061850634239003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:30.996456] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257710, offset=39018354}, end_cursor:ObLogCursor{file_id=197, log_id=49257711, offset=39018637}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b98050, len:4238, log_data_len:283, seq:49257710, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:30.998888] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB364-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061850997202000, start_id=1729061850997202000, end_id=1729061850997202001) [2024-10-16 14:57:31.000706] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA875-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061851000413000, start_id=1729061851000413000, end_id=1729061851000413001) [2024-10-16 14:57:31.000815] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601346, offset=9613566}, end_cursor:ObLogCursor{file_id=192, log_id=47601347, offset=9613840}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c80050, len:3842, log_data_len:274, seq:47601346, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:31.001540] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601347, offset=9613840}, end_cursor:ObLogCursor{file_id=192, log_id=47601348, offset=9614114}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b74050, len:3568, log_data_len:274, seq:47601347, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:31.001722] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561169, miss_cnt:1, cache_read_size:152590663440, hit_ratio:"9.999999928346832156e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:31.004984] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=23] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:31.006282] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA875-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061851006126000, start_id=1729061851006126000, end_id=1729061851006126001) [2024-10-16 14:57:31.008557] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.008712] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:31.008746] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=25] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257711, offset=39018637}, end_cursor:ObLogCursor{file_id=197, log_id=49257712, offset=39018919}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c44050, len:3955, log_data_len:282, seq:49257711, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:31.011815] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257712, offset=39018919}, end_cursor:ObLogCursor{file_id=197, log_id=49257713, offset=39019201}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bbc050, len:3673, log_data_len:282, seq:49257712, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:31.011949] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257713, offset=39019201}, end_cursor:ObLogCursor{file_id=197, log_id=49257714, offset=39019483}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b78050, len:3391, log_data_len:282, seq:49257713, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:31.011999] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650107, miss_cnt:32, cache_read_size:8050404177, hit_ratio:"9.999994892269912228e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:31.014633] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=43] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656843, miss_cnt:17, cache_read_size:8050151548, hit_ratio:"9.999997286809456654e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:31.024200] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061851020474000, start_id=1729061851020474000, end_id=1729061851020474001) [2024-10-16 14:57:31.024267] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=26, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:31.024281] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:31.025816] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061850821071000, v:0}, new_max_version={val:1729061850821071000, v:0}, last_min_version={val:1729061850621624000, v:0}, last_max_version={val:1729061850621624000, v:0}, total_time_us=2407, delta_us=204743) [2024-10-16 14:57:31.025845] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=28] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061850821071000, v:0}, min_version={val:1729061850821071000, v:0}, max_version={val:1729061850821071000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:31.028668] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061851028255000, start_id=1729061851028255000, end_id=1729061851028255001) [2024-10-16 14:57:31.028696] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061851028693], gts:1729061851028255000, latest_srr:[mts=1729061851028693]}) [2024-10-16 14:57:31.029064] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061851028255000, start_id=1729061851028255001, end_id=1729061851028255002) [2024-10-16 14:57:31.039036] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:31.039059] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:31.039147] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:31.039156] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:31.039167] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=291885101, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:31.040634] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:31.040973] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CFC-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:31.040982] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CFC-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30401, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:31.040986] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CFC-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=1280, avg_log_submit_to_slide_time=30396) [2024-10-16 14:57:31.040994] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53CFC-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173042, log_task={header:{begin_lsn:{lsn:14352361327}, end_lsn:{lsn:14352361449}, log_id:115173042, min_scn:{val:1729061850634239003, v:0}, max_scn:{val:1729061850634239003, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352361205}, prev_proposal_id:17, committed_end_lsn:{lsn:14352361327}, data_checksum:940009244, accum_checksum:3344698414, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061850939282, freeze_ts:1729061850939282, submit_ts:1729061850939286, flushed_ts:1729061850949302, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:10016}, fs_cb_begin_ts=1729061851040967, log_life_time=101685) [2024-10-16 14:57:31.041047] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:31.041377] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD0D3-0-0] [lt=17] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:31.042550] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:31.042562] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151784220, tenant_id=1) [2024-10-16 14:57:31.042576] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:31.042830] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E20-0-0] [lt=5] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12274, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:31.043338] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E20-0-0] [lt=10] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12274, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851043324, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851043324, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851043324, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.043361] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E20-0-0] [lt=22] success to process renew task(task={tenant_id:1, table_id:12274, add_timestamp:1729061851041988}, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851043324, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851043324, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851043324, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.043779] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB26C-0-0] [lt=66][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:31.055003] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB26C-0-0] [lt=17][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:31.063848] INFO [SQL.PC] need_check_schema_version (ob_plan_cache_value.cpp:2237) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9986D1C20E-0-0] [lt=13] need check schema(new_schema_version=1722495466719360, cached_tenant_schema_version=1722495466719360, is_contain_synonym()=false, contain_sys_name_table_=true, is_contain_tmp_tbl()=false, is_contain_sys_pl_object()=false, pc_ctx.sql_ctx_.session_info_->get_has_temp_table_flag()=false, need_check=true, constructed_sql_=select * from __all_ls_log_archive_progress where tenant_id=? and dest_id=? and round_id=? and ls_id=? order by piece_id desc limit ?) [2024-10-16 14:57:31.064299] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F68-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061851064148000, start_id=1729061851064148000, end_id=1729061851064148001) [2024-10-16 14:57:31.065558] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061851065208000, start_id=1729061851065208000, end_id=1729061851065208001) [2024-10-16 14:57:31.074150] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061851068677000, start_id=1729061851068677000, end_id=1729061851068677001) [2024-10-16 14:57:31.077622] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:31.077657] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:31.077980] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:31.083632] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:31.083810] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D43-0-0] [lt=27] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061851028255000, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061848671146002, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860721265000, latest_log_ts:{val:1729061844127564001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:31.083980] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D43-0-0] [lt=88] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061851028255000, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061848671146002, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:31.084123] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D43-0-0] [lt=64] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061850633180004, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061848671146002, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:31.084258] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D43-0-0] [lt=61] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061850634239003, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061848671146002, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:31.084321] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D43-0-0] [lt=61] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:31.089690] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.108630] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.109946] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061851108964000, start_id=1729061851108964000, end_id=1729061851108964001) [2024-10-16 14:57:31.110614] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:31.110627] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:31.127179] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:31.128672] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1284, mem_used=4159488) [2024-10-16 14:57:31.133687] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BEF-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061851133014000, start_id=1729061851133014000, end_id=1729061851133014001) [2024-10-16 14:57:31.133782] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BEC-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061851133014000, start_id=1729061851133014000, end_id=1729061851133014001) [2024-10-16 14:57:31.133813] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BF0-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061851133014000, start_id=1729061851133014000, end_id=1729061851133014001) [2024-10-16 14:57:31.134020] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BEE-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061851133014000, start_id=1729061851133014000, end_id=1729061851133014001) [2024-10-16 14:57:31.134019] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BED-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061851133014000, start_id=1729061851133014000, end_id=1729061851133014001) [2024-10-16 14:57:31.140626] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107148][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD33-0-0] [lt=5] init point thread id with(&point=0x55e3435c5180, idx_=652, point=[thread id=107148, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107148) [2024-10-16 14:57:31.154004] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76A3-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:31.163541] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:31.163730] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D37-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510319032124, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410581, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:31.166714] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=15] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:31.169866] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=11] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:31.169885] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:31.169892] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:31.169898] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=5] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:31.169901] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:31.169905] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16723) [2024-10-16 14:57:31.176299] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52D6-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:31.176311] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52D6-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9900, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:31.176334] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=23, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:31.176346] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=23, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:31.177227] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5021-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:31.177245] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5021-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:31.177475] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:31.177487] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:31.180570] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061851180073000, start_id=1729061851180073000, end_id=1729061851180073001) [2024-10-16 14:57:31.180643] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=10094, avg_group_log_size=630) [2024-10-16 14:57:31.180653] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9624) [2024-10-16 14:57:31.180903] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=324, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:31.180913] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10094) [2024-10-16 14:57:31.180922] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=189, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:31.180956] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9502) [2024-10-16 14:57:31.180969] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:31.181205] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD084-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:31.181220] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD084-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=1463, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:31.181224] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD084-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=28, avg_log_gen_to_submit_time=57, avg_log_submit_to_flush_time=1138, avg_log_submit_to_slide_time=1405) [2024-10-16 14:57:31.188035] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480B-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061851155737, keepalive_ts:-1}) [2024-10-16 14:57:31.188062] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480B-0-0] [lt=25] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061851155737, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:31.189749] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.195681] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=18] set tenant work area memory(tenant_id=1001, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:31.195741] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16] set tenant work area memory(tenant_id=1002, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:31.195790] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6] set tenant work area memory(tenant_id=1003, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:31.195832] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6] set tenant work area memory(tenant_id=1004, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:31.195954] EDIAG [SHARE.CONFIG] check_500_tenant_hold (ob_server_config.cpp:438) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] the hold memory of tenant_500 is over the reserved memory(hold=1084416000, reserved=1073741824) BACKTRACE:0x1a5796b1 0x6847325 0x6a4dfba 0x6a4d9a4 0x6a4d8e0 0x6a4d707 0x1479497f 0x11dea49b 0x6809ea1 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d [2024-10-16 14:57:31.195974] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:32) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=18] === Run print tenant memory usage task === [2024-10-16 14:57:31.196005] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:102) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=3] ====== tenants memory info ====== === TENANTS MEMORY INFO === divisive_memory_used= 220,549,120 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,084,416,000 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 19,554,304 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 17,457,152 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 1 now= 1,729,061,851,194,929 active_memstore_used= 247,463,936 total_memstore_used= 245,366,784 total_memstore_hold= 245,366,784 memstore_freeze_trigger_limit= 151,993,940 memstore_limit= 858,993,440 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 1,980,207,104 max_mem_memstore_can_get_now= 759,969,784 memstore_alloc_pos= 20,002,635,776 memstore_frozen_pos= 19,755,171,840 memstore_reclaimed_pos= 19,757,268,992 [TENANT_MEMORY] tenant_id= 1,001 now= 1,729,061,851,194,929 active_memstore_used= 48,234,496 total_memstore_used= 44,040,192 total_memstore_hold= 44,040,192 memstore_freeze_trigger_limit= 54,222,420 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 904,810,496 max_mem_memstore_can_get_now= 271,112,192 memstore_alloc_pos= 32,717,668,352 memstore_frozen_pos= 32,669,433,856 memstore_reclaimed_pos= 32,673,628,160 [TENANT_MEMORY] tenant_id= 1,002 now= 1,729,061,851,194,929 active_memstore_used= 44,040,192 total_memstore_used= 44,040,192 total_memstore_hold= 44,040,192 memstore_freeze_trigger_limit= 28,230,020 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 967,602,176 max_mem_memstore_can_get_now= 141,150,192 memstore_alloc_pos= 465,951,522,816 memstore_frozen_pos= 465,907,482,624 memstore_reclaimed_pos= 465,907,482,624 [TENANT_MEMORY] tenant_id= 1,003 now= 1,729,061,851,194,929 active_memstore_used= 60,817,408 total_memstore_used= 60,817,408 total_memstore_hold= 60,817,408 memstore_freeze_trigger_limit= 53,383,560 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 940,462,080 max_mem_memstore_can_get_now= 266,917,858 memstore_alloc_pos= 34,273,755,136 memstore_frozen_pos= 34,212,937,728 memstore_reclaimed_pos= 34,212,937,728 [TENANT_MEMORY] tenant_id= 1,004 now= 1,729,061,851,194,929 active_memstore_used= 39,845,888 total_memstore_used= 39,845,888 total_memstore_hold= 39,845,888 memstore_freeze_trigger_limit= 26,789,460 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 965,230,592 max_mem_memstore_can_get_now= 133,947,388 memstore_alloc_pos= 1,602,224,128 memstore_frozen_pos= 1,562,378,240 memstore_reclaimed_pos= 1,562,378,240 [2024-10-16 14:57:31.196069] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:114) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16] [CHUNK_MGR] limit= 12,884,901,888 hold= 7,739,707,392 total_hold= 8,642,363,392 used= 6,879,739,904 freelists_hold= 859,967,488 total_maps= 388,959 total_unmaps= 385,792 large_maps= 301 large_unmaps= 0 huge_maps= 200 huge_unmaps= 187 memalign=0 resident_size= 8,383,107,072 virtual_memory_used= 9,723,944,960 [CHUNK_MGR] 2 MB_CACHE: hold= 132,120,576 free= 63 pushes= 6,196,520,868 pops= 6,196,520,805 maps= 388,458 unmaps= 385,605 [CHUNK_MGR] 4 MB_CACHE: hold= 154,873,856 free= 73 pushes= 6,821,111 pops= 6,821,038 maps= 172 unmaps= 0 [CHUNK_MGR] 6 MB_CACHE: hold= 193,880,064 free= 46 pushes= 6,920 pops= 6,874 maps= 59 unmaps= 0 [CHUNK_MGR] 8 MB_CACHE: hold= 25,247,744 free= 4 pushes= 16,248,808 pops= 16,248,804 maps= 11 unmaps= 0 [CHUNK_MGR] 10 MB_CACHE: hold= 109,318,144 free= 13 pushes= 123 pops= 110 maps= 30 unmaps= 0 [CHUNK_MGR] 12 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [CHUNK_MGR] 14 MB_CACHE: hold= 26,435,584 free= 2 pushes= 4 pops= 2 maps= 9 unmaps= 0 [CHUNK_MGR] 16 MB_CACHE: hold= 33,316,864 free= 2 pushes= 4 pops= 2 maps= 2 unmaps= 0 [CHUNK_MGR] 18 MB_CACHE: hold= 184,774,656 free= 11 pushes= 993 pops= 982 maps= 18 unmaps= 0 [CHUNK_MGR] 20 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [2024-10-16 14:57:31.196109] INFO print (ob_malloc_time_monitor.cpp:39) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7] [MALLOC_TIME_MONITOR] show the distribution of ob_malloc's cost_time [MALLOC_TIME_MONITOR] [ 0, 10): delta_total_cost_time= 1467276014, delta_count= 7243433057, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 10, 100): delta_total_cost_time= 62894333, delta_count= 4016698, avg_cost_time= 15 [MALLOC_TIME_MONITOR] [ 100, 1000): delta_total_cost_time= 714851, delta_count= 3754, avg_cost_time= 190 [MALLOC_TIME_MONITOR] [ 1000, 10000): delta_total_cost_time= 644928, delta_count= 236, avg_cost_time= 2732 [MALLOC_TIME_MONITOR] [ 10000, 100000): delta_total_cost_time= 668844, delta_count= 36, avg_cost_time= 18579 [MALLOC_TIME_MONITOR] [ 100000, 1000000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 1000000, 9223372036854775807): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [2024-10-16 14:57:31.205507] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4806) [2024-10-16 14:57:31.205779] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4219) [2024-10-16 14:57:31.205814] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4219) [2024-10-16 14:57:31.206836] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FB8-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:31.206847] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FB8-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1034, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:31.208658] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.210811] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5876-0-0] [lt=15] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:31.210909] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5876-0-0] [lt=15][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:31.224897] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061851224497000, start_id=1729061851224497000, end_id=1729061851224497001) [2024-10-16 14:57:31.226194] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:57:31.226210] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=82, cache_node_num=82) [2024-10-16 14:57:31.226216] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=82, cache_node_num=82) [2024-10-16 14:57:31.226221] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:31.228377] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=6] Dumping All Cache Objs(alloc_obj_list.count()=82, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}]) [2024-10-16 14:57:31.228489] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=110] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:31.238752] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D00-0-0] [lt=5][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173044, log_task={header:{begin_lsn:{lsn:14352361571}, end_lsn:{lsn:14352361693}, log_id:115173044, min_scn:{val:1729061851134047000, v:0}, max_scn:{val:1729061851134047000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352361449}, prev_proposal_id:17, committed_end_lsn:{lsn:14352361571}, data_checksum:940009244, accum_checksum:1029418175, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061851138645, freeze_ts:1729061851138646, submit_ts:1729061851138651, flushed_ts:1729061851138952, gen_to_freeze cost time:1, gen_to_submit cost time:6, submit_to_flush cost time:301}, fs_cb_begin_ts=1729061851238745, log_life_time=100100) [2024-10-16 14:57:31.239123] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:57:31.239993] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061851239220000, start_id=1729061851239220000, end_id=1729061851239220001) [2024-10-16 14:57:31.240150] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:31.240161] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:31.240183] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:31.240188] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:31.240398] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:31.246689] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21660][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3DE6-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510319115083, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410582, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:31.250549] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD07-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:31.255126] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1103, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:31.255158] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=571, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:31.255210] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:31.265614] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061851264281000, start_id=1729061851264281000, end_id=1729061851264281001) [2024-10-16 14:57:31.266148] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B81-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:31.266450] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_max_applied_scn(scn={val:1729061851064148000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959681390}, palf_committed_end_scn_:{val:1729061851064148001, v:0}, last_check_scn_:{val:1729061851064148001, v:0}, max_applied_cb_scn_:{val:1729061851064148000, v:0}}) [2024-10-16 14:57:31.266471] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:31.266544] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=59] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:57:31.266552] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] get_min_unreplayed_log_info(lsn={lsn:213058223824}, scn={val:1729061851133014001, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061851154484, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058223824}, next_to_submit_scn_:{val:1729061851133014001, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058223702}, end_lsn:{lsn:213058223824}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061851133014000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061851133014000, v:0}, data_checksum:3348837161, flag:1}}, init_mode_version:0, accumulate_checksum:4263439648, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061851133014000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561172, miss_cnt:1, cache_read_size:152590663806, hit_ratio:"9.999999928346833267e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:31.270911] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1031-0-0] [lt=9] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:31.274295] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061851271286000, start_id=1729061851271286000, end_id=1729061851271286001) [2024-10-16 14:57:31.277249] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB95-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12239, add_timestamp:1729061851277239}) [2024-10-16 14:57:31.277280] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E21-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12239, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851277267, sql_renew_time:1729061851277267, is_mark_fail:false}]) [2024-10-16 14:57:31.277299] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E21-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:12239, add_timestamp:1729061851277239}, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851277267, sql_renew_time:1729061851277267, is_mark_fail:false}]) [2024-10-16 14:57:31.281010] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:31.281730] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=674, avg_log_submit_to_slide_time=1189) [2024-10-16 14:57:31.282324] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061851281172000, start_id=1729061851281172000, end_id=1729061851281172001) [2024-10-16 14:57:31.282542] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:31.284308] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB95-0-0] [lt=14][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:31.285380] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=17] start to print blacklist info [2024-10-16 14:57:31.285525] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=9] ls blacklist refresh finish(cost_time=8873) [2024-10-16 14:57:31.288389] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:45338084276}, base_lsn={lsn:45161508864}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:31.288451] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=61] get_replay_process success(id={id:1}, replayed_log_size=176575412, unreplayed_log_size=0) [2024-10-16 14:57:31.288473] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=6] dump tenant replay process(tenant_id=1003, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=168, last_replayed_log_size(MB)=168, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:31.290781] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.308715] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.309995] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061851308060000, start_id=1729061851308060000, end_id=1729061851308060001) [2024-10-16 14:57:31.310753] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] [WRS] update tenant weak read server version(tenant_id=1001, server_version={version:{val:1729061851108964000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061851310732}, version_delta=201788) [2024-10-16 14:57:31.311513] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=7] ====== traversal_flush timer task ====== [2024-10-16 14:57:31.311595] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA8-0-0] [lt=10] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061859029444000, latest_log_ts:{val:1729061842479541001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:31.311663] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA8-0-0] [lt=64] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:31.313832] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6252, mem_used=12478464) [2024-10-16 14:57:31.321744] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:31.340169] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=14] LogLoopThread round_cost_time(us)(round_cost_time=8) [2024-10-16 14:57:31.340971] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1671, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:31.340994] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=482, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:31.341029] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:31.341315] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD39-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:31.341325] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD39-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=2500, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:31.342627] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD3A-0-0] [lt=26] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:31.342639] INFO [LIB] stat (utility.h:1165) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD3A-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1498, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:31.342644] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD3A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=456, avg_log_submit_to_slide_time=1482) [2024-10-16 14:57:31.348119] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=9] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:31.348262] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E03-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061851344705000, start_id=1729061851344705000, end_id=1729061851344705001) [2024-10-16 14:57:31.348340] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107147][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=15] worker thread exit(tenant_->id()=1004, workers_.get_size()=6) [2024-10-16 14:57:31.348352] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107147][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=8] tenant thread end_run(id_=1004, ret=0, thread_count_=195) [2024-10-16 14:57:31.352157] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA8-0-0] [lt=9] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:31.352173] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA8-0-0] [lt=11] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=17) [2024-10-16 14:57:31.354630] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52D9-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:31.363849] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:31.364881] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=21] get_max_applied_scn(scn={val:1729061851064148001, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023088535}, palf_committed_end_scn_:{val:1729061851064148002, v:0}, last_check_scn_:{val:1729061851064148002, v:0}, max_applied_cb_scn_:{val:1729061851064148001, v:0}}) [2024-10-16 14:57:31.364904] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=20] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:31.366063] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B83-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:31.382484] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=7] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:57:31.382502] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=17] try gc retain ctx [2024-10-16 14:57:31.382506] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:31.383122] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061859029444000, latest_log_ts:{val:1729061842479541001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:31.389969] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:31.391776] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.408764] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.425029] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3ACF93-0-0] [lt=19] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.738636016845703125e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=869, Throttle Trigger(MB)=521, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.115848223054147326e+01) [2024-10-16 14:57:31.425202] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061851424884000, start_id=1729061851424884000, end_id=1729061851424884001) [2024-10-16 14:57:31.432670] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B9-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:31.432701] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9B9-0-0] [lt=19] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061851134047002, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061851134047002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:31.433754] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9B9-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061851433292000, start_id=1729061851433292000, end_id=1729061851433292001) [2024-10-16 14:57:31.437042] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9B9-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061851436484000, start_id=1729061851436484000, end_id=1729061851436484001) [2024-10-16 14:57:31.438015] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:31.438751] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D04-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:31.438770] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D04-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:31.438787] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D04-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:31.440870] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:31.454235] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:31.454662] INFO [COMMON] loop_running_dag_net_list (ob_tenant_dag_scheduler.cpp:3738) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] loop running dag net list(ret=0, running_dag_net_list_size=0, slow_dag_net_cnt=0, finished_dag_net_list_size=0) [2024-10-16 14:57:31.454675] INFO [COMMON] loop_finished_dag_net_list (ob_tenant_dag_scheduler.cpp:3761) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=13] loop finsihed dag net list(ret=0, finished_dag_net_list_size=0) [2024-10-16 14:57:31.458705] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A75-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:31.458725] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A75-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:31.464252] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:57:31.464268] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:31.464275] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:31.464280] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:31.466107] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B85-0-0] [lt=8][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178353, log_task={header:{begin_lsn:{lsn:14538475580}, end_lsn:{lsn:14538475702}, log_id:115178353, min_scn:{val:1729061851133014002, v:0}, max_scn:{val:1729061851133014002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538475458}, prev_proposal_id:23, committed_end_lsn:{lsn:14538475580}, data_checksum:1715074492, accum_checksum:24548807, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061851366050, freeze_ts:1729061851366050, submit_ts:1729061851366057, flushed_ts:1729061851366334, gen_to_freeze cost time:0, gen_to_submit cost time:7, submit_to_flush cost time:277}, fs_cb_begin_ts=1729061851466101, log_life_time=100051) [2024-10-16 14:57:31.466640] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:57:31.466779] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=137] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:31.467688] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=45] get number(ret=0, service_type_=0, range=1, base_id=1729061851467225000, start_id=1729061851467225000, end_id=1729061851467225001) [2024-10-16 14:57:31.474459] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061851473551000, start_id=1729061851473551000, end_id=1729061851473551001) [2024-10-16 14:57:31.474625] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E99801B4F12-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061851474618000, start_id=1729061851474618000, end_id=1729061851474618001) [2024-10-16 14:57:31.477002] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:31.477653] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:31.477669] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:31.477883] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=258, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:31.477901] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=416, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:31.477921] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:31.478179] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41B8-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:31.478186] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41B8-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1013, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:31.478291] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41B9-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:31.481087] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:31.482459] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061851482287000, start_id=1729061851482287000, end_id=1729061851482287001) [2024-10-16 14:57:31.485615] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:31.485629] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=13] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:31.485630] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECF-0-0] [lt=11] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:31.485641] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:31.485643] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ECF-0-0] [lt=11] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:31.485647] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:31.485650] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:31.485653] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:31.485656] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:31.485658] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:31.485660] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:31.485663] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:31.485666] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:31.487149] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E99801B4F12-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061851486597000, start_id=1729061851486597000, end_id=1729061851486597001) [2024-10-16 14:57:31.489486] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:31.489500] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:31.489513] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70576418, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:31.491936] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.497516] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E99801B4F12-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061851497301000, start_id=1729061851497301000, end_id=1729061851497301001) [2024-10-16 14:57:31.497542] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0B-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061851488758, keepalive_ts:-1}) [2024-10-16 14:57:31.497564] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0B-0-0] [lt=21] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061851488758, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:31.508896] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.510023] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061851507793000, start_id=1729061851507793000, end_id=1729061851507793001) [2024-10-16 14:57:31.512644] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] [COMPACTION DAG STATUS] (start_time=1729061841505973, end_time=1729061851512642, dag_status={}) [2024-10-16 14:57:31.512657] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:31.512664] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:31.512668] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:31.512672] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:31.512675] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:31.512679] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:31.512682] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:31.512685] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:31.512689] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:31.512692] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512698] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512702] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512706] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512710] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512714] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512717] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512721] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512725] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512729] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512742] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512746] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512751] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512755] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512758] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512762] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512766] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512770] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512773] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512777] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512781] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512785] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512789] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512792] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512796] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512800] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512804] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512807] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512811] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512820] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512823] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512827] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512831] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512835] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512839] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512843] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512846] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512850] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512854] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512858] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512862] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512865] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512869] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512873] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512877] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512880] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512884] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512888] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512891] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512895] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512899] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512902] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512906] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512910] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512914] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512917] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512921] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512925] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:31.512929] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:31.512933] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:31.512936] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:31.512939] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:31.512942] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:31.512944] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:31.512946] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:31.512949] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:31.512951] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:31.512953] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:31.512956] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:31.518640] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E99801B4F12-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061851517630000, start_id=1729061851517630000, end_id=1729061851517630001) [2024-10-16 14:57:31.518998] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C46-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:31.519017] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C46-0-0] [lt=9] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061851133014003, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061851133014003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:31.522765] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C46-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061851521951000, start_id=1729061851521951000, end_id=1729061851521951001) [2024-10-16 14:57:31.529010] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061851528192000, start_id=1729061851528192000, end_id=1729061851528192001) [2024-10-16 14:57:31.529031] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:31.529208] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061851529197], gts:1729061851528192000, latest_srr:[mts=1729061851529197]}) [2024-10-16 14:57:31.533708] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221732213, support_ssl=false) [2024-10-16 14:57:31.533737] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] sm conn init succ(conn.sessid_=3221732213, sess.client_addr_="10.0.4.7:53368") [2024-10-16 14:57:31.533745] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0f27414df0, session_id:3221732213, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:214}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:31.533864] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] username and tenantname(user_name=root, tenant_name=) [2024-10-16 14:57:31.533878] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=5] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:31.533882] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:295) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=3] mysql login queue(mysql_queue->queue_.size()=1) [2024-10-16 14:57:31.533952] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AA-0-0] [lt=18] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:31.534396] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AA-0-0] [lt=18] MySQL LOGIN(direct_client_ip="10.0.4.7", client_ip=10.0.4.7, tenant_name=sys, tenant_id=1, user_name=root, host_name=%, sessid=3221732213, proxy_sessid=0, client_sessid=3221732213, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=3908101, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:31.564173] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:31.568495] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.569051] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.569365] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.569533] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.577936] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:31.578121] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41BB-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:31.578140] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41BB-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1235, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:31.582403] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1001}, {id:1}]) [2024-10-16 14:57:31.586818] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=2][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:31.586836] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:31.586915] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=13] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:57:31.591983] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.598762] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=4772, mem_used=10398720) [2024-10-16 14:57:31.608937] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.624313] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] no need to update min/max version(new_version={val:1729061851224497000, v:0}, min_version={val:1729061851224497000, v:0}, max_version={val:1729061851224497000, v:0}) [2024-10-16 14:57:31.624331] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:31.634108] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BF2-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061851633174000, start_id=1729061851633174000, end_id=1729061851633174001) [2024-10-16 14:57:31.634113] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BF1-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061851633174000, start_id=1729061851633174000, end_id=1729061851633174001) [2024-10-16 14:57:31.634311] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BF5-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061851634232000, start_id=1729061851634232000, end_id=1729061851634232001) [2024-10-16 14:57:31.634475] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BF3-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061851634232000, start_id=1729061851634232000, end_id=1729061851634232001) [2024-10-16 14:57:31.634502] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BF4-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061851634232000, start_id=1729061851634232000, end_id=1729061851634232001) [2024-10-16 14:57:31.635399] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B966-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061851635326000, start_id=1729061851635326000, end_id=1729061851635326001) [2024-10-16 14:57:31.656313] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=20] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:31.663415] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26147][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB18CC6EB-0-0] [lt=24] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:31.664579] INFO [SQL.ENG] execute (ob_database_executor.cpp:126) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D3A-0-0] [lt=18] use default database(db=oceanbase) [2024-10-16 14:57:31.668110] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061851665622000, start_id=1729061851665622000, end_id=1729061851665622001) [2024-10-16 14:57:31.676042] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EDB-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:31.677734] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:57:31.677744] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:57:31.677748] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:31.678387] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061848522922003, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860098662000, latest_log_ts:{val:1729061843439703001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:31.679106] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=48] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061848522922003, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:31.679869] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=46] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061848522922003, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:31.680495] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=43] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061848522922003, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:31.681159] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:31.682675] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=46] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:31.682691] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:31.683475] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061851682922000, start_id=1729061851682922000, end_id=1729061851682922001) [2024-10-16 14:57:31.684624] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061851682922000, start_id=1729061851682922001, end_id=1729061851682922002) [2024-10-16 14:57:31.684714] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=21] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:31.694719] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.696229] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D63-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:31.696257] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D63-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061851133014004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061851133014004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:31.697641] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:31.710644] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.710694] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1001, min_version={val:1729061851507793000, v:0}, base_version={val:1729061851264281000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061851507793000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061851710683, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061851521951000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061851610452, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061851521951000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061851683624, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:31.711498] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061851710034000, start_id=1729061851710034000, end_id=1729061851710034001) [2024-10-16 14:57:31.717744] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=1387, avg_log_submit_to_slide_time=1728) [2024-10-16 14:57:31.717760] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=11, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:31.723452] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:31.725438] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061851723170000, start_id=1729061851723170000, end_id=1729061851723170001) [2024-10-16 14:57:31.730764] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D63-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061851729930000, start_id=1729061851729930000, end_id=1729061851729930001) [2024-10-16 14:57:31.740797] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:31.740819] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:31.743059] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15D6-0-0] [lt=8] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:31.743100] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=2214, avg_log_submit_to_slide_time=2409) [2024-10-16 14:57:31.755431] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:31.755458] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:31.755466] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:31.755625] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=19] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:31.764471] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:31.766291] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B8B-0-0] [lt=27] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:31.766303] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B8B-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30126, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:31.766307] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B8B-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=341, avg_log_submit_to_slide_time=30120) [2024-10-16 14:57:31.767680] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:31.767736] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=77, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:31.767743] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:31.767856] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:31.770690] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:31.776157] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EDD-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:31.776180] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EDD-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:31.776435] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=196, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:31.776447] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=103, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:31.776490] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:31.777786] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=33] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1333, avg_group_log_size=121) [2024-10-16 14:57:31.777797] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1333) [2024-10-16 14:57:31.781113] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=18] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:31.788470] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.788485] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.788632] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.788653] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.788677] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.788753] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.788762] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.788770] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.789156] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.789616] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.791220] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=24] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}) [2024-10-16 14:57:31.794791] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.799651] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:31.810325] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:31.810703] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.824409] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1002}, {id:1001}]) [2024-10-16 14:57:31.827755] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:31.827768] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:31.827783] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52311538, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:31.838822] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D0C-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173050, log_task={header:{begin_lsn:{lsn:14352362303}, end_lsn:{lsn:14352362425}, log_id:115173050, min_scn:{val:1729061851635326001, v:0}, max_scn:{val:1729061851635326001, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352362181}, prev_proposal_id:17, committed_end_lsn:{lsn:14352362303}, data_checksum:940009244, accum_checksum:2881609168, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061851738798, freeze_ts:1729061851738798, submit_ts:1729061851738803, flushed_ts:1729061851746909, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:8106}, fs_cb_begin_ts=1729061851838815, log_life_time=100017) [2024-10-16 14:57:31.845241] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:31.845257] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10668, avg_group_log_size=592) [2024-10-16 14:57:31.845267] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10668) [2024-10-16 14:57:31.845273] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:31.852020] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3844, mem_used=8318976) [2024-10-16 14:57:31.854302] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76B1-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:31.854728] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52E3-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:31.854748] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52E3-0-0] [lt=20] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20507, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:31.854771] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:31.857875] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:31.861854] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] new mysql sessid created(conn.sessid_=3221737333, support_ssl=false) [2024-10-16 14:57:31.861883] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=14] sm conn init succ(conn.sessid_=3221737333, sess.client_addr_="10.0.4.7:53370") [2024-10-16 14:57:31.861891] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0f2742c030, session_id:3221737333, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:215}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:31.862144] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13] username and tenantname(user_name=root, tenant_name=) [2024-10-16 14:57:31.862165] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=10] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:31.862244] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB5-0-0] [lt=12] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:31.862701] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB5-0-0] [lt=7] MySQL LOGIN(direct_client_ip="10.0.4.7", client_ip=10.0.4.7, tenant_name=sys, tenant_id=1, user_name=root, host_name=%, sessid=3221737333, proxy_sessid=0, client_sessid=3221737333, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=29336205, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:31.864231] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D41-0-0] [lt=12] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11092, add_timestamp:1729061851864221}) [2024-10-16 14:57:31.864252] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E22-0-0] [lt=8] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11092, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:31.864373] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E22-0-0] [lt=15] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11092, locations=[{table_id:11092, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851864360, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851864360, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851864360, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.864392] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E22-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:11092, add_timestamp:1729061851864221}, locations=[{table_id:11092, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851864360, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851864360, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851864360, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.866572] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D41-0-0] [lt=19] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:31.867413] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D41-0-0] [lt=13][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:31.868253] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061851866879000, start_id=1729061851866879000, end_id=1729061851866879001) [2024-10-16 14:57:31.868438] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B8D-0-0] [lt=24][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178357, log_task={header:{begin_lsn:{lsn:14538476068}, end_lsn:{lsn:14538476190}, log_id:115178357, min_scn:{val:1729061851634232001, v:0}, max_scn:{val:1729061851634232001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538475946}, prev_proposal_id:23, committed_end_lsn:{lsn:14538476068}, data_checksum:1715074492, accum_checksum:644983842, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061851766271, freeze_ts:1729061851766271, submit_ts:1729061851766277, flushed_ts:1729061851767876, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:1599}, fs_cb_begin_ts=1729061851868417, log_life_time=102146) [2024-10-16 14:57:31.868712] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=335, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:31.868724] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10669) [2024-10-16 14:57:31.868735] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=732, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:31.868743] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10668) [2024-10-16 14:57:31.868760] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=10, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:31.868840] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F942C-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:31.868846] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F942C-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=2080, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:31.868849] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F942C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=1901, avg_log_submit_to_slide_time=2048) [2024-10-16 14:57:31.873659] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26143][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB26CC6B8-0-0] [lt=28] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:57:31.875229] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EA9-0-0] [lt=8] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:31.876078] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EA9-0-0] [lt=13][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:31.877703] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=40] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:31.878271] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=12] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:31.878315] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061847238487000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:31.879616] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41C4-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=1192, avg_log_submit_to_slide_time=1559) [2024-10-16 14:57:31.881238] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=9] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:31.881842] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D42-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11096, add_timestamp:1729061851881833}) [2024-10-16 14:57:31.881865] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E23-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11096, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:31.882009] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E23-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11096, locations=[{table_id:11096, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851881998, sql_renew_time:0, is_mark_fail:false}, {table_id:11096, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851881998, sql_renew_time:0, is_mark_fail:false}, {table_id:11096, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851881998, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.882029] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E23-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:11096, add_timestamp:1729061851881833}, locations=[{table_id:11096, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851881998, sql_renew_time:0, is_mark_fail:false}, {table_id:11096, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851881998, sql_renew_time:0, is_mark_fail:false}, {table_id:11096, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851881998, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.884668] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] no need to update min/max version(new_version={val:1729061851473551000, v:0}, min_version={val:1729061851473551000, v:0}, max_version={val:1729061851473551000, v:0}) [2024-10-16 14:57:31.887528] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061851885928000, start_id=1729061851885928000, end_id=1729061851885928001) [2024-10-16 14:57:31.895718] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D42-0-0] [lt=8][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:31.896589] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061847891766, last_first_register_time_us=1722494150998907, register_time_us=1729061847891766, curr_time_us=1729061851896588) [2024-10-16 14:57:31.896627] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=20] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061851896611, last_submit_register_req_time_us=1729061851896611, register_parent_reason=3) [2024-10-16 14:57:31.896636] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:31.897237] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.900698] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=14] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:31.907719] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=21] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3666, avg_group_log_size=333) [2024-10-16 14:57:31.907983] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1249, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:31.907998] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=147, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:31.909444] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D44-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12362, add_timestamp:1729061851909433}) [2024-10-16 14:57:31.909475] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E24-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12362, locations=[{table_id:12362, partition_id:12362, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851909461, sql_renew_time:1729061851909461, is_mark_fail:false}]) [2024-10-16 14:57:31.909495] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E24-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:12362, add_timestamp:1729061851909433}, locations=[{table_id:12362, partition_id:12362, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851909461, sql_renew_time:1729061851909461, is_mark_fail:false}]) [2024-10-16 14:57:31.911579] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061851910068000, start_id=1729061851910068000, end_id=1729061851910068001) [2024-10-16 14:57:31.920180] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:31.920197] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=211) [2024-10-16 14:57:31.921041] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DD7-0-0] [lt=11] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:57:31.921285] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=9] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:31.922621] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:31.926747] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061851926423000, start_id=1729061851926423000, end_id=1729061851926423001) [2024-10-16 14:57:31.927134] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 6144.00, iops: 3.98, rt: 1185.25; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 4096.00, iops: 1.00, rt: 128.00; ) [2024-10-16 14:57:31.927157] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=4, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=49999, free_result_count=49999, callback_queues=[0]) [2024-10-16 14:57:31.927220] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.98, rt: 211.75; ) [2024-10-16 14:57:31.927239] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:31.927248] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:31.940206] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2490, mem_used=6239232) [2024-10-16 14:57:31.941852] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=362, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:31.941868] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=225, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:31.941898] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2035) [2024-10-16 14:57:31.954572] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D45-0-0] [lt=29] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12254, add_timestamp:1729061851954564}) [2024-10-16 14:57:31.954936] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E25-0-0] [lt=8] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12254, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:31.955640] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E25-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12254, locations=[{table_id:12254, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851955626, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851955626, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851955626, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.955664] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E25-0-0] [lt=23] success to process renew task(task={tenant_id:1, table_id:12254, add_timestamp:1729061851954564}, locations=[{table_id:12254, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851955626, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851955626, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851955626, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.961349] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D47-0-0] [lt=14] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12261, add_timestamp:1729061851961339}) [2024-10-16 14:57:31.961377] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E26-0-0] [lt=17] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12261, locations=[{table_id:12261, partition_id:12261, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851961365, sql_renew_time:1729061851961365, is_mark_fail:false}]) [2024-10-16 14:57:31.961396] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E26-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:12261, add_timestamp:1729061851961339}, locations=[{table_id:12261, partition_id:12261, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851961365, sql_renew_time:1729061851961365, is_mark_fail:false}]) [2024-10-16 14:57:31.964773] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:31.965235] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D48-0-0] [lt=12] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12291, add_timestamp:1729061851965228}) [2024-10-16 14:57:31.965260] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E27-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12291, locations=[{table_id:12291, partition_id:12291, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851965248, sql_renew_time:1729061851965248, is_mark_fail:false}]) [2024-10-16 14:57:31.965280] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E27-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:12291, add_timestamp:1729061851965228}, locations=[{table_id:12291, partition_id:12291, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851965248, sql_renew_time:1729061851965248, is_mark_fail:false}]) [2024-10-16 14:57:31.967655] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B8F-0-0] [lt=42] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:31.968823] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D49-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12334, add_timestamp:1729061851968818}) [2024-10-16 14:57:31.968974] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=204, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:31.968990] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=277, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:31.969021] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:31.969158] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E28-0-0] [lt=11] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12334, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:31.971636] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E28-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12334, locations=[{table_id:12334, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851971624, sql_renew_time:0, is_mark_fail:false}, {table_id:12334, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851971624, sql_renew_time:0, is_mark_fail:false}, {table_id:12334, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851971624, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.971657] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E28-0-0] [lt=21] success to process renew task(task={tenant_id:1, table_id:12334, add_timestamp:1729061851968818}, locations=[{table_id:12334, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851971624, sql_renew_time:0, is_mark_fail:false}, {table_id:12334, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851971624, sql_renew_time:0, is_mark_fail:false}, {table_id:12334, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061851971624, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:31.977906] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:31.977920] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:31.978096] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061851978095, bytes: 4446809700740, bw: 0.566464 MB/s, add_ts: 1002081, add_bytes: 595217 [2024-10-16 14:57:31.979343] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=448, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:31.979358] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=832, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:31.979375] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:31.982366] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EE1-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497318, log_task={header:{begin_lsn:{lsn:207589535519}, end_lsn:{lsn:207589535641}, log_id:218497318, min_scn:{val:1729061851634232003, v:0}, max_scn:{val:1729061851634232003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589535397}, prev_proposal_id:81, committed_end_lsn:{lsn:207589535519}, data_checksum:940009244, accum_checksum:1607243335, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061851877930, freeze_ts:1729061851877931, submit_ts:1729061851877933, flushed_ts:1729061851878077, gen_to_freeze cost time:1, gen_to_submit cost time:3, submit_to_flush cost time:144}, fs_cb_begin_ts=1729061851982361, log_life_time=104431) [2024-10-16 14:57:31.982454] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:31.982472] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598766, miss_cnt:9, cache_read_size:149777900082, hit_ratio:"9.999999345924409644e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:31.984727] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] no need to update min/max version(new_version={val:1729061851473551000, v:0}, min_version={val:1729061851473551000, v:0}, max_version={val:1729061851473551000, v:0}) [2024-10-16 14:57:31.989233] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061851989232, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007023, add_bytes: 0 [2024-10-16 14:57:31.991390] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D36-0-0] [lt=13] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:31.991406] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D36-0-0] [lt=12] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:31.994641] INFO [CLOG] do_thread_task_ (ob_remote_fetch_log_worker.cpp:249) [21822][T1001_RFLWorker][T1001][YB4A0A000407-00061E996D1960C1-0-0] [lt=15] ObRemoteFetchWorker is running(thread_index=0) [2024-10-16 14:57:31.997269] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:31.997288] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:31.997295] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.003395] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601348, offset=9614114}, end_cursor:ObLogCursor{file_id=192, log_id=47601349, offset=9614398}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b70050, len:3294, log_data_len:284, seq:47601348, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.008658] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA49A-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061852007063000, start_id=1729061852007063000, end_id=1729061852007063001) [2024-10-16 14:57:32.010958] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA876-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:32.010978] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA876-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061851634232003, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061851634232003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:32.012318] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA876-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061852012281000, start_id=1729061852012281000, end_id=1729061852012281001) [2024-10-16 14:57:32.015151] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=7] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257714, offset=39019483}, end_cursor:ObLogCursor{file_id=197, log_id=49257715, offset=39019766}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b04050, len:3109, log_data_len:283, seq:49257714, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.018065] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB365-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061852015429000, start_id=1729061852015429000, end_id=1729061852015429001) [2024-10-16 14:57:32.018195] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=12][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:32.020876] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601349, offset=9614398}, end_cursor:ObLogCursor{file_id=192, log_id=47601350, offset=9614672}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b48050, len:3010, log_data_len:274, seq:47601349, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.020926] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257715, offset=39019766}, end_cursor:ObLogCursor{file_id=197, log_id=49257716, offset=39020048}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d64050, len:2826, log_data_len:282, seq:49257715, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.022696] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.031687] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061852031255000, start_id=1729061852031255000, end_id=1729061852031255001) [2024-10-16 14:57:32.031735] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061852031682], gts:1729061852031255000, latest_srr:[mts=1729061852031682]}) [2024-10-16 14:57:32.032252] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=8] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257716, offset=39020048}, end_cursor:ObLogCursor{file_id=197, log_id=49257717, offset=39020330}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2be8050, len:2544, log_data_len:282, seq:49257716, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.032372] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA876-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061852032315000, start_id=1729061852032315000, end_id=1729061852032315001) [2024-10-16 14:57:32.032479] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601350, offset=9614672}, end_cursor:ObLogCursor{file_id=192, log_id=47601351, offset=9614946}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b90050, len:2736, log_data_len:274, seq:47601350, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.032843] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257717, offset=39020330}, end_cursor:ObLogCursor{file_id=197, log_id=49257718, offset=39020612}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d1c050, len:2262, log_data_len:282, seq:49257717, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.032884] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601351, offset=9614946}, end_cursor:ObLogCursor{file_id=192, log_id=47601352, offset=9615220}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d68050, len:2462, log_data_len:274, seq:47601351, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.032983] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:32.033150] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561180, miss_cnt:1, cache_read_size:152590664782, hit_ratio:"9.999999928346837708e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:32.033229] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650116, miss_cnt:32, cache_read_size:8050405397, hit_ratio:"9.999994892270646085e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:32.033245] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656853, miss_cnt:17, cache_read_size:8050152768, hit_ratio:"9.999997286809889641e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:32.046263] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:32.046283] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:32.047432] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1913) [2024-10-16 14:57:32.047458] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:32.047619] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D10-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:32.047628] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D10-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=50826, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:32.047632] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D10-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=8, avg_log_submit_to_flush_time=1422, avg_log_submit_to_slide_time=50817) [2024-10-16 14:57:32.047639] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D10-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173052, log_task={header:{begin_lsn:{lsn:14352362547}, end_lsn:{lsn:14352362669}, log_id:115173052, min_scn:{val:1729061851635326003, v:0}, max_scn:{val:1729061851635326003, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352362425}, prev_proposal_id:17, committed_end_lsn:{lsn:14352362547}, data_checksum:940009244, accum_checksum:3600119715, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061851939362, freeze_ts:1729061851939362, submit_ts:1729061851939367, flushed_ts:1729061851939594, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:227}, fs_cb_begin_ts=1729061852047612, log_life_time=108250) [2024-10-16 14:57:32.047675] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:32.047933] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=412, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:32.047941] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:32.047960] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=891, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:32.047979] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:32.065780] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=43] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:32.066043] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_tablet_compaction_info.cpp:86) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D49-0-0] [lt=8][errcode=-4008] fail to execute(ret=-4008) [2024-10-16 14:57:32.066380] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:32.066427] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061850192549000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:32.067737] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B91-0-0] [lt=16][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178359, log_task={header:{begin_lsn:{lsn:14538476312}, end_lsn:{lsn:14538476434}, log_id:115178359, min_scn:{val:1729061851634232003, v:0}, max_scn:{val:1729061851634232003, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538476190}, prev_proposal_id:23, committed_end_lsn:{lsn:14538476312}, data_checksum:1715074492, accum_checksum:242526766, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061851967644, freeze_ts:1729061851967644, submit_ts:1729061851967649, flushed_ts:1729061851969001, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:1352}, fs_cb_begin_ts=1729061852067730, log_life_time=100086) [2024-10-16 14:57:32.068566] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061852068424000, start_id=1729061852068424000, end_id=1729061852068424001) [2024-10-16 14:57:32.069238] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=42, this=0x2b0e07718280, extra_info="") [2024-10-16 14:57:32.069249] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=1328, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:57:32.069253] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:57:32.069256] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=-645, this=0x2b0e07718910, extra_info="") [2024-10-16 14:57:32.069260] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB TOTAL TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=726, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:57:32.070919] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:32.070931] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=12] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.070943] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:32.070947] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.070952] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:32.070955] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.070959] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:32.070962] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.070966] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:32.070968] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4A-0-0] [lt=2] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"NORMAL", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.071872] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=23] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061851528192002, v:0}, new_max_version={val:1729061851528192002, v:0}, last_min_version={val:1729061851133014004, v:0}, last_max_version={val:1729061851133014004, v:0}, total_time_us=4278, delta_us=543672) [2024-10-16 14:57:32.071893] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061851528192002, v:0}, min_version={val:1729061851528192002, v:0}, max_version={val:1729061851528192002, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:57:32.073755] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F78-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061852071065000, start_id=1729061852071065000, end_id=1729061852071065001) [2024-10-16 14:57:32.079028] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:32.079067] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:32.079746] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D3-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061852070152, keepalive_ts:-1}) [2024-10-16 14:57:32.079776] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D3-0-0] [lt=28] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061852070152, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:32.080501] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=16] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:32.080515] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=14] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:32.080526] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:32.080530] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:32.080542] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=11] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:32.080545] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:32.080552] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:32.080555] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:32.080562] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:32.080565] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21659][T1_L0_G0][T1][YB4A0A000407-00061E99789E2635-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":0}) [2024-10-16 14:57:32.081243] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4C-0-0] [lt=3] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12306, add_timestamp:1729061852081235}) [2024-10-16 14:57:32.081260] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4C-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12308, add_timestamp:1729061852081259}) [2024-10-16 14:57:32.081722] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:32.084435] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E29-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12306, locations=[{table_id:12306, partition_id:12306, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852084423, sql_renew_time:1729061852084423, is_mark_fail:false}]) [2024-10-16 14:57:32.084453] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E29-0-0] [lt=17] success to process renew task(task={tenant_id:1, table_id:12306, add_timestamp:1729061852081235}, locations=[{table_id:12306, partition_id:12306, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852084423, sql_renew_time:1729061852084423, is_mark_fail:false}]) [2024-10-16 14:57:32.084467] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2A-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12308, locations=[{table_id:12308, partition_id:12308, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852084464, sql_renew_time:1729061852084464, is_mark_fail:false}]) [2024-10-16 14:57:32.084474] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2A-0-0] [lt=6] success to process renew task(task={tenant_id:1, table_id:12308, add_timestamp:1729061852081259}, locations=[{table_id:12308, partition_id:12308, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852084464, sql_renew_time:1729061852084464, is_mark_fail:false}]) [2024-10-16 14:57:32.085544] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061852084570000, start_id=1729061852084570000, end_id=1729061852084570001) [2024-10-16 14:57:32.087816] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4D-0-0] [lt=17] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.088655] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4D-0-0] [lt=15][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.089949] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061852088821000, start_id=1729061852088821000, end_id=1729061852088821001) [2024-10-16 14:57:32.097332] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.106937] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=47] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:32.106983] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D4F-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12297, add_timestamp:1729061852106976}) [2024-10-16 14:57:32.108881] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:32.108899] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:32.110388] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2B-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12297, locations=[{table_id:12297, partition_id:12297, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852110375, sql_renew_time:1729061852110375, is_mark_fail:false}]) [2024-10-16 14:57:32.110420] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2B-0-0] [lt=31] success to process renew task(task={tenant_id:1, table_id:12297, add_timestamp:1729061852106976}, locations=[{table_id:12297, partition_id:12297, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852110375, sql_renew_time:1729061852110375, is_mark_fail:false}]) [2024-10-16 14:57:32.110552] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:32.110564] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:32.112430] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061852110646000, start_id=1729061852110646000, end_id=1729061852110646001) [2024-10-16 14:57:32.114466] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D50-0-0] [lt=8] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.115258] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D50-0-0] [lt=12][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.119527] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D51-0-0] [lt=6] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.120302] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D51-0-0] [lt=15][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.122718] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.125617] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=15] current monitor number(seq_=-1) [2024-10-16 14:57:32.127239] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=15] gc stale ls task succ [2024-10-16 14:57:32.127791] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D52-0-0] [lt=11] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.128580] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D52-0-0] [lt=16][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.128808] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061852127103000, start_id=1729061852127103000, end_id=1729061852127103001) [2024-10-16 14:57:32.129166] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1405, mem_used=4159488) [2024-10-16 14:57:32.132589] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D54-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11093, add_timestamp:1729061852132579}) [2024-10-16 14:57:32.132613] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2C-0-0] [lt=9] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11093, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:32.132661] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D54-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061852132552000, start_id=1729061852132552000, end_id=1729061852132552001) [2024-10-16 14:57:32.133901] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2C-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11093, locations=[{table_id:11093, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852133888, sql_renew_time:0, is_mark_fail:false}, {table_id:11093, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852133888, sql_renew_time:0, is_mark_fail:false}, {table_id:11093, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852133888, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:32.133926] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2C-0-0] [lt=24] success to process renew task(task={tenant_id:1, table_id:11093, add_timestamp:1729061852132579}, locations=[{table_id:11093, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852133888, sql_renew_time:0, is_mark_fail:false}, {table_id:11093, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852133888, sql_renew_time:0, is_mark_fail:false}, {table_id:11093, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852133888, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:32.135081] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BF6-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061852134673000, start_id=1729061852134673000, end_id=1729061852134673001) [2024-10-16 14:57:32.135116] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BF7-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061852134673000, start_id=1729061852134673000, end_id=1729061852134673001) [2024-10-16 14:57:32.135204] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0BF8-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061852134673000, start_id=1729061852134673000, end_id=1729061852134673001) [2024-10-16 14:57:32.135303] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BF9-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061852134673000, start_id=1729061852134673000, end_id=1729061852134673001) [2024-10-16 14:57:32.135350] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BFA-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061852134673000, start_id=1729061852134673000, end_id=1729061852134673001) [2024-10-16 14:57:32.137519] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:32.137536] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=16] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.138204] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:32.138234] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=16] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:32.138244] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:32.138256] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:32.138261] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:32.138287] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:32.138293] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.138301] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:32.138304] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.138309] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:32.138314] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.138320] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:32.138325] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D56-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:32.140879] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D57-0-0] [lt=9] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.141687] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D57-0-0] [lt=14][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.150687] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D58-0-0] [lt=3] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12435, add_timestamp:1729061852150679}) [2024-10-16 14:57:32.152435] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15CE-0-0] [lt=16] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:32.153833] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2D-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12435, locations=[{table_id:12435, partition_id:12435, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852153821, sql_renew_time:1729061852153821, is_mark_fail:false}]) [2024-10-16 14:57:32.153853] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2D-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:12435, add_timestamp:1729061852150679}, locations=[{table_id:12435, partition_id:12435, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852153821, sql_renew_time:1729061852153821, is_mark_fail:false}]) [2024-10-16 14:57:32.154526] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76B7-0-0] [lt=23] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:32.156221] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15D2-0-0] [lt=13] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:32.156504] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D59-0-0] [lt=11] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.157294] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D59-0-0] [lt=13][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.158190] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15D4-0-0] [lt=10] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:32.160470] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D5A-0-0] [lt=19] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12402, add_timestamp:1729061852160462}) [2024-10-16 14:57:32.160530] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2E-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12402, locations=[{table_id:12402, partition_id:12402, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852160518, sql_renew_time:1729061852160518, is_mark_fail:false}]) [2024-10-16 14:57:32.160548] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2E-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:12402, add_timestamp:1729061852160462}, locations=[{table_id:12402, partition_id:12402, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852160518, sql_renew_time:1729061852160518, is_mark_fail:false}]) [2024-10-16 14:57:32.165079] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.167742] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED790-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510320036135, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410583, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.175641] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:32.178099] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5022-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:32.178115] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5022-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:32.178545] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:32.178557] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:32.181282] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52EA-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:32.181296] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D52EA-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=31386, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:32.181639] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:32.181647] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:32.196885] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=6] ====== traversal_flush timer task ====== [2024-10-16 14:57:32.197004] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D83-0-0] [lt=12] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061867008855000, latest_log_ts:{val:1729061850421032001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:32.197059] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D83-0-0] [lt=51] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:32.197408] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.197570] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D5C-0-0] [lt=12] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12072, add_timestamp:1729061852197562}) [2024-10-16 14:57:32.206865] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2F-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12072, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852206853, sql_renew_time:1729061852206853, is_mark_fail:false}]) [2024-10-16 14:57:32.206889] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E2F-0-0] [lt=23] success to process renew task(task={tenant_id:1, table_id:12072, add_timestamp:1729061852197562}, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061852206853, sql_renew_time:1729061852206853, is_mark_fail:false}]) [2024-10-16 14:57:32.212070] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=11] Garbage Collector is running(seq_=656698, gc_interval=10000000) [2024-10-16 14:57:32.212098] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=13] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:32.212109] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=27) [2024-10-16 14:57:32.212115] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:32.212850] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:32.212863] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=10] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:32.212868] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=6) [2024-10-16 14:57:32.212872] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:32.214778] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D5D-0-0] [lt=12] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.219058] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D5E-0-0] [lt=14] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.219913] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D5E-0-0] [lt=13][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.222777] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.228356] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21658][T1_L0_G0][T1][YB4A0A000407-00061E99794D04FA-0-0] [lt=10] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.229167] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21658][T1_L0_G0][T1][YB4A0A000407-00061E99794D04FA-0-0] [lt=18][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.237092] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D60-0-0] [lt=11] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.237947] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D60-0-0] [lt=17][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.246399] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1913, avg_group_log_size=191) [2024-10-16 14:57:32.246415] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1913) [2024-10-16 14:57:32.246441] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:32.246446] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:32.246802] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:32.248099] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D61-0-0] [lt=12] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.248950] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D61-0-0] [lt=13][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:32.255671] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2742c030, session_id:3221737333, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:3, sql_initiative_shutdown:false, reader:{fd:215}, err:5, last_decode_time:1729061852255560, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:3765, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:32.255705] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=28] kill and revert session(conn.sessid_=3221737333, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:32.255756] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:866) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D62-0-0] [lt=13][errcode=0] server close connection(sessid=3221737333, proxy_sessid=0, stack="0x1a5796b1 0xc7f3c2c 0xc7baa9b 0xc95cc71 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.255769] INFO [SERVER] process (obmp_quit.h:56) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D62-0-0] [lt=10] quit [2024-10-16 14:57:32.255775] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:163) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D62-0-0] [lt=5][errcode=-4009] sock has error(ret=-4009) [2024-10-16 14:57:32.255781] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:1069) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D62-0-0] [lt=6][errcode=-4009] write response fail(ret=-4009) [2024-10-16 14:57:32.255785] WDIAG [SERVER] response (obmp_base.cpp:87) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D62-0-0] [lt=4][errcode=-4009] failed to flush_buffer(ret=-4009) [2024-10-16 14:57:32.255791] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:50) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D62-0-0] [lt=4][errcode=-4009] response rpc result fail(ret=-4009, tmp_ret=-4009) [2024-10-16 14:57:32.255797] WDIAG [SERVER.OMT] process_one (ob_worker_processor.cpp:89) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D62-0-0] [lt=5][errcode=-4009] process request fail(ret=-4009) [2024-10-16 14:57:32.255803] WDIAG [SERVER.OMT] process (ob_worker_processor.cpp:157) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D62-0-0] [lt=5][errcode=-4009] process request fail(ret=-4009) [2024-10-16 14:57:32.255807] WDIAG [SERVER.OMT] process_request (ob_th_worker.cpp:249) [21653][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3][errcode=-4009] process request fail(ret=-4009) [2024-10-16 14:57:32.264857] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=231, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:32.264884] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1051, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:32.264938] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:32.267789] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B95-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:32.269583] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061852268702000, start_id=1729061852268702000, end_id=1729061852268702001) [2024-10-16 14:57:32.281799] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:32.283814] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9615, avg_group_log_size=565) [2024-10-16 14:57:32.283828] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10085) [2024-10-16 14:57:32.283989] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=413, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:32.284001] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9615) [2024-10-16 14:57:32.284010] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1078, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:32.284024] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10085) [2024-10-16 14:57:32.284032] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:32.284385] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD095-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:32.284396] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD095-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=2801, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:32.284410] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD095-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=69, avg_log_gen_to_submit_time=95, avg_log_submit_to_flush_time=1778, avg_log_submit_to_slide_time=2705) [2024-10-16 14:57:32.285848] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061852285722000, start_id=1729061852285722000, end_id=1729061852285722001) [2024-10-16 14:57:32.289974] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061852289952000, start_id=1729061852289952000, end_id=1729061852289952001) [2024-10-16 14:57:32.290120] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106820][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D39-0-0] [lt=18] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:32.290139] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106820][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D39-0-0] [lt=18] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:32.297463] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.311375] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:32.311396] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:32.311418] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] cannot_recycle_log_size statistics(cannot_recycle_log_size=373563034, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:32.311444] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3666) [2024-10-16 14:57:32.311695] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3666) [2024-10-16 14:57:32.311803] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3666) [2024-10-16 14:57:32.312073] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FC3-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:32.312083] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FC3-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1937, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:32.312827] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061852312237000, start_id=1729061852312237000, end_id=1729061852312237001) [2024-10-16 14:57:32.313942] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6329, mem_used=12478464) [2024-10-16 14:57:32.321822] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=16] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:32.324316] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.335950] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061852326670000, start_id=1729061852326670000, end_id=1729061852326670001) [2024-10-16 14:57:32.347824] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183785867, max_lsn:114618424494, max_scn:{val:1729061852312237000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45685691230}, reuse_lsn:{lsn:114618424494}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114576481454}}, last_submit_lsn:{lsn:114618423902}, last_submit_end_lsn:{lsn:114618424494}, last_submit_log_id:183785867, last_submit_log_pid:33705, max_flushed_lsn:{lsn:114618423902}, max_flushed_end_lsn:{lsn:114618424494}, max_flushed_log_pid:33705, committed_end_lsn:{lsn:114618424494}, last_slide_log_id:183785867, last_slide_scn:{val:1729061852312237000, v:0}, last_slide_lsn:{lsn:114618423902}, last_slide_end_lsn:{lsn:114618424494}, last_slide_log_pid:33705, last_slide_log_accum_checksum:3934491898, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0df99e2530}, [StateMgr]={this:0x2b0df99e6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, leader:"10.0.4.7:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183785868, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261592457, palf_role_change_cb:0x2b0df99ed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, reconfig_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, persistent_config_version:{proposal_id:33705, config_seq:14}, ms_ack_list:0, resend_config_version:{proposal_id:33705, config_seq:14}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258289572, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258289572, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0df99e3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261792660, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729029056668344001, v:0}, min_block_id:1702, min_block_min_scn:{val:1729022447183332001, v:0}, base_lsn_for_block_gc:{lsn:114547838976}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, log_config_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, log_snapshot_meta:{version:1, base_lsn:{lsn:114547838976}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:233218048}, readable_log_tail:{lsn:233218048}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114682048512}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/meta", dir_fd:146, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:35201024, block_header_serialize_buf_:0x2b0df99e89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114618424494}, readable_log_tail:{lsn:114618424494}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114614943744}, min_scn:{val:1729061506448191001, v:0}, curr_block_id:1708, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/log", dir_fd:148, min_block_id:1702, max_block_id:1709, curr_writable_block_id:1708}, logical_block_size_:67104768, curr_block_writable_size_:63624018, block_header_serialize_buf_:0x2b0df99ea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261592575, this:0x2b0df99e6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33705, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33704, majority_max_lsn:{lsn:45686897415}, saved_end_lsn:{lsn:45686897415}, last_submit_prepare_req_time_us:1722494261592686, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0df99e69b0}) [2024-10-16 14:57:32.348000] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=171] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061852313186, last_flushed_end_lsn:{lsn:114618424494}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061852313334, last_flushed_end_lsn:{lsn:114618424494}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061852313543, last_flushed_end_lsn:{lsn:114618424494}}]) [2024-10-16 14:57:32.348012] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183785868, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061852313186, last_flushed_end_lsn:{lsn:114618424494}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061852313334, last_flushed_end_lsn:{lsn:114618424494}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061852313543, last_flushed_end_lsn:{lsn:114618424494}}]) [2024-10-16 14:57:32.349240] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:32.349258] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:32.349276] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:32.351109] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:45, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:45, col_desc_array:[cnt:45, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=42 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=45 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=46 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=47 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=48 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=49 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=50 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=51 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=52 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=53 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=54 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=55 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=56 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=57 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=58 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:45, 2197175160, 2197175160, 2197175160, 0, 0, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2388842353, 2197175160, 2388842353, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2388842353, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 1244474940, 1244474940], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:0, index_type:0, row_store_type:1, schema_version:1722494633924376, column_cnt:43, store_column_cnt:43, tablet_size:134217728, pctfree:10, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:1, encryption:"", encrypt_key:"", rowkey_cnt:3, rowkey_array:[cnt:3, {column_idx:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0}, {column_idx:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:43, column_array:[cnt:43, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TINYINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TINYINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":-1}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":-1}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:1, row_store_type:1, block_size:16384, schema_column_cnt:43, rowkey_column_cnt:5, schema_rowkey_column_cnt:3, column_cnt:45, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:45, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=42 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=45 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=46 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=47 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=48 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=49 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=50 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=51 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=52 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=53 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=54 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=55 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=56 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=57 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=58 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.351433] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=7] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:32.351447] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:32.351454] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:32.351457] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:32.351460] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:32.351464] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17113) [2024-10-16 14:57:32.351501] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=191] Async write macro block(macro_id_=[14770](ver=0,mode=0,seq=7667497)) [2024-10-16 14:57:32.351595] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:2072576, macro_id:[14770](ver=0,mode=0,seq=7667497)}, offset_=2076672, old_block=[]) [2024-10-16 14:57:32.351634] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:5, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f009fc408, hex: 0000000000000000, int: 0}, idx=1:{len: 25, flag: 0, null: 0, ptr: 0x2b0f009fc520, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f009fc478, hex: 0800000000000000, int: 8}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f009fc540, hex: 90502A24D42C01E8, int: -1729051492282118000}, idx=4:{len: 8, flag: 0, null: 0, ptr: 0x2b0f009fc4e8, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:45, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:1376, meta_block_size:309, last_macro_size:1685, macro_metas:0x2b0f009fc100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.351696] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=20] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764845606764544-0](offset=0,size=309,type=3,seq=0), buf:0x2b0f334c5070, height:1, is_meta_root:true}) [2024-10-16 14:57:32.351707] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764797288382464-0](offset=0,size=264,type=3,seq=0), buf:0x2b0f334c4560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764845606764544-0](offset=0,size=309,type=3,seq=0), buf:0x2b0f334c5070, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:45, data_column_checksums:[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], row_count:3, max_merged_trans_version:1729051493406896000, contain_uncommitted_row:false, occupy_size:1112, original_size:495, data_checksum:3450587089, use_old_macro_block_count:0, compressor_type:1, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.351739] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=16] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0eda24a060, key:{tablet_id:{id:377}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}, ref_cnt:0, upper_trans_version:1729051493406896000, timestamp:0}, this:0x2b0eda24a060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2072576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:1112, data_checksum:3450587089, max_merged_trans_version:1729051493406896000, upper_trans_version:1729051493406896000, filled_tx_scn:{val:1729051758122012000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:1112, original_size_:495, data_checksum_:3450587089, index_type_:0, rowkey_column_count_:5, column_cnt_:45, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729051493406896000, max_merged_trans_version_:1729051493406896000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729051758122012000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764797288382464-0](offset=0,size=264,type=3,seq=0), orig_block_buf:0x2b0eda24a390, block_data:{buf:0x2b0eda24a600, size:64, extra_buf:0x2b0eda24a640, extra_size:1144, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764845606764544-0](offset=0,size=309,type=3,seq=0), orig_block_buf:0x2b0eda24a4a0, block_data:{buf:0x2b0eda24a4a0, size:309, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0eda24a5e0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2072576, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.351789] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=49] finish set upper trans version(ret=0, key_={tablet_id:{id:377}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}, meta=0x2b0eda24a130, old_val=1729051493406896000, upper_trans_version=1729051493406896000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2072576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:1112, data_checksum:3450587089, max_merged_trans_version:1729051493406896000, upper_trans_version:1729051493406896000, filled_tx_scn:{val:1729051758122012000, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.351800] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:2072576, macro_id:[14770](ver=0,mode=0,seq=7667497)}, new_sstable.get_key()={tablet_id:{id:377}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}) [2024-10-16 14:57:32.352368] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E7-0-0] [lt=38] [LS_LOCATION]dump tenant ls location caches(tenant_id=1, tenant_ls_locations=[{cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061849982477, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:32.352390] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E7-0-0] [lt=21] [LS_LOCATION]dump tenant ls location caches(tenant_id=1001, tenant_ls_locations=[{cache_key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061849983186, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:32.352400] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E7-0-0] [lt=9] [LS_LOCATION]dump tenant ls location caches(tenant_id=1002, tenant_ls_locations=[{cache_key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061849983907, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061849983906, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061849983908, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}]}, {cache_key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061849983903, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:32.352430] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E7-0-0] [lt=30] [LS_LOCATION]dump tenant ls location caches(tenant_id=1003, tenant_ls_locations=[{cache_key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061849984554, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:32.352439] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E7-0-0] [lt=8] [LS_LOCATION]dump tenant ls location caches(tenant_id=1004, tenant_ls_locations=[{cache_key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061849985249, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061849985253, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061849985253, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}]}, {cache_key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061849985251, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:32.353814] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=1) [2024-10-16 14:57:32.358815] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107144][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=14] worker thread exit(tenant_->id()=1004, workers_.get_size()=5) [2024-10-16 14:57:32.358838] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107144][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=17] tenant thread end_run(id_=1004, ret=0, thread_count_=194) [2024-10-16 14:57:32.359124] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:66390085369}], scn_array=[cnt:1, {val:1729061852326670000, v:0}], curr_size=587, accum_size=587, time_cost=21133) [2024-10-16 14:57:32.360260] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:14352363035}], scn_array=[cnt:1, {val:1729061852135733002, v:0}], curr_size=122, accum_size=366, time_cost=10668) [2024-10-16 14:57:32.360316] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:13645407796}], scn_array=[cnt:1, {val:1729061852031255003, v:0}], curr_size=122, accum_size=1098, time_cost=12076) [2024-10-16 14:57:32.360389] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:32.360397] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2476, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:32.360401] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=2067, avg_log_submit_to_slide_time=2461) [2024-10-16 14:57:32.360477] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1779, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:32.360483] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:17493454277}], scn_array=[cnt:1, {val:1729061852031255003, v:0}], curr_size=122, accum_size=488, time_cost=12177) [2024-10-16 14:57:32.360491] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=349, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:32.360611] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:32.360615] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2394, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:32.360619] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:32.363620] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f334e05c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:1, sstable_array:0x2b0f334e1370, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f334e05c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:1, sstable_array:0x2b0f334e1370, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f334e1db0, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729000842663168000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINI 1729051493406896000 1729051493406896000 1729001176236986000 1729051758122012000 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560287003454000 1717560287003454000 1 1728978212337222000 0 false }}) [2024-10-16 14:57:32.363838] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=32] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:377}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f334e0060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8487dda0, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e8487ddc0, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.363863] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=22] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6a8a60a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663348-6841-0-4611686767901044736-0](offset=1863680,size=698,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:377}, data_tablet_id:{id:377}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729001176236986000, v:0}, clog_checkpoint_scn:{val:1729051758122012000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729051758122012000, multi_version_start:1729015257002301000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:372883027, row_count:17, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:16384, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:16384}}, table_store_addr:{ptr:0x2b0e6a8a66d4, addr:[7663656-3934-0-2305843504209862656-0](offset=1187840,size=461,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663656-3934-0-2305844138791281101-0](offset=1188301,size=1052,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a8a892c, addr:[7663348-6841-0-4611686241767551466-0](offset=1864170,size=208,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f334e0060, is_inited:true, wash_score:1729061852360373000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:377}, data_tablet_id:{id:377}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729001176236986000, v:0}, clog_checkpoint_scn:{val:1729051758122012000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729051758122012000, multi_version_start:1729015257002301000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:372883027, row_count:17, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:6291456, data_size:0, shared_meta_size:2521, meta_size:0, occupy_bytes:16384}}, table_store_addr:{ptr:0x2b0f334e0450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0e8487c980, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0e8487dd10, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.363983] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=83] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334e05d0, key:{tablet_id:{id:377}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f334e05d0, addr:[7663656-3934-0-2305844829207269376-0](offset=1183744,size=1695,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:192512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:17, occupy_size:3078, data_checksum:372883027, max_merged_trans_version:1729000842663168000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:92, row_count_:17, occupy_size_:3078, original_size_:9838, data_checksum_:372883027, index_type_:0, rowkey_column_count_:5, column_cnt_:45, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1722494633924376, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729000842663168000, max_merged_trans_version_:1729000842663168000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334e08e8, column_checksum_count:45, data_root_info:{addr:[0-0-0-3458764944391012352-0](offset=0,size=401,type=3,seq=0), orig_block_buf:0x2b0f334e0ac0, block_data:{buf:0x2b0f334e0a50, size:112, extra_buf:0x2b0f334e0c51, extra_size:1108, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765249333690368-0](offset=0,size=685,type=3,seq=0), orig_block_buf:0x2b0f334e10a5, block_data:{buf:0x2b0f334e10a5, size:685, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334e1352, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:192512, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1607, old_pos=0, pos=1607) [2024-10-16 14:57:32.364037] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=49] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334e1380, key:{tablet_id:{id:377}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}, ref_cnt:0, upper_trans_version:1729051493406896000, timestamp:0}, this:0x2b0f334e1380, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2072576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:1112, data_checksum:3450587089, max_merged_trans_version:1729051493406896000, upper_trans_version:1729051493406896000, filled_tx_scn:{val:1729051758122012000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:1112, original_size_:495, data_checksum_:3450587089, index_type_:0, rowkey_column_count_:5, column_cnt_:45, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729051493406896000, max_merged_trans_version_:1729051493406896000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729051758122012000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334e1698, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764797288382464-0](offset=0,size=264,type=3,seq=0), orig_block_buf:0x2b0f334e16d8, block_data:{buf:0x2b0f334e1698, size:64, extra_buf:0x2b0f334e17e0, extra_size:1144, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764845606764544-0](offset=0,size=309,type=3,seq=0), orig_block_buf:0x2b0f334e1c58, block_data:{buf:0x2b0f334e1c58, size:309, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334e1d8d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2072576, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=837, old_pos=0, pos=837) [2024-10-16 14:57:32.364066] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0e8487c050, key:{tablet_id:{id:377}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0e8487c050, addr:[7663656-3934-0-2305843896125626940-0](offset=1186364,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:876544, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:625, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:625, original_size_:328, data_checksum_:3146711327, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0e8487c368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0e8487c3a8, block_data:{buf:0x2b0e8487c368, size:64, extra_buf:0x2b0e8487c495, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0e8487c864, block_data:{buf:0x2b0e8487c864, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e8487c959, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:876544, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=738, old_pos=0, pos=738) [2024-10-16 14:57:32.364126] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.367018] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1181712384, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.370015] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:32.370030] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:32.370042] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:32.370115] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76BB-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:32.370140] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76BB-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938174, log_task={header:{begin_lsn:{lsn:213058225044}, end_lsn:{lsn:213058225166}, log_id:220938174, min_scn:{val:1729061852134673001, v:0}, max_scn:{val:1729061852134673001, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058224922}, prev_proposal_id:88, committed_end_lsn:{lsn:213058225044}, data_checksum:1715074492, accum_checksum:647097179, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061852254619, freeze_ts:1729061852254619, submit_ts:1729061852254625, flushed_ts:1729061852264902, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:10277}, fs_cb_begin_ts=1729061852370124, log_life_time=115505) [2024-10-16 14:57:32.370409] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B97-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:32.370426] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B97-0-0] [lt=11][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178362, log_task={header:{begin_lsn:{lsn:14538476678}, end_lsn:{lsn:14538476800}, log_id:115178362, min_scn:{val:1729061852134673001, v:0}, max_scn:{val:1729061852134673001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538476556}, prev_proposal_id:23, committed_end_lsn:{lsn:14538476678}, data_checksum:1715074492, accum_checksum:3018592312, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061852267780, freeze_ts:1729061852267781, submit_ts:1729061852267803, flushed_ts:1729061852271142, gen_to_freeze cost time:1, gen_to_submit cost time:23, submit_to_flush cost time:3339}, fs_cb_begin_ts=1729061852370422, log_life_time=102642) [2024-10-16 14:57:32.371356] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=8][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27414df0, session_id:3221732213, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:214}, err:5, last_decode_time:1729061851664572, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:215, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:32.371382] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=19] kill and revert session(conn.sessid_=3221732213, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:32.371389] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:978) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] can close safely, do destroy(*s={this:0x2b0f2742c030, session_id:3221737333, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:215}, err:5, last_decode_time:1729061852255560, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:3770, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2024-10-16 14:57:32.371404] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] unlock session of tenant(conn.sessid_=3221737333, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:32.371409] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] connection close(sessid=3221737333, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B304F3D61-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:221, pkt_rec[189]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[190]:{send:obp_mysql_header_{pkt_num_:100, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[191]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[192]:{receive:obp_mysql_header_{len_:79, rec_:130, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[193]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[194]:{send:obp_mysql_header_{pkt_num_:80, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[195]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[196]:{send:obp_mysql_header_{pkt_num_:290, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[197]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[198]:{receive:obp_mysql_header_{len_:79, rec_:198, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[199]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[200]:{send:obp_mysql_header_{pkt_num_:75, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[201]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[202]:{send:obp_mysql_header_{pkt_num_:76, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[203]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[204]:{receive:obp_mysql_header_{len_:74, rec_:232, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[205]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[206]:{send:obp_mysql_header_{pkt_num_:149, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[207]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[208]:{receive:obp_mysql_header_{len_:78, rec_:194, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[209]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[210]:{send:obp_mysql_header_{pkt_num_:82, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[211]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[212]:{send:obp_mysql_header_{pkt_num_:116, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[213]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[214]:{receive:obp_mysql_header_{len_:78, rec_:262, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[215]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[216]:{send:obp_mysql_header_{pkt_num_:82, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[217]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[218]:{send:obp_mysql_header_{pkt_num_:123, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[219]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[220]:{receive:obp_mysql_header_{len_:1, rec_:200, seq_:0}, pkt_name:"COM_QUIT", obp_mysql_header_.type_:1, obp_mysql_header_.is_send_:0}], disconnect_state=3) [2024-10-16 14:57:32.371461] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=49] sock ref clean, do destroy(*s={this:0x2b0f27414df0, session_id:3221732213, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:214}, err:5, last_decode_time:1729061851664572, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:215, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:32.371470] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221732213, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:32.371473] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221732213, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B304F3D3B-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:10, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:9, rec_:9, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:1, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{receive:obp_mysql_header_{len_:14, rec_:14, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[8]:{send:obp_mysql_header_{len_:11, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_mysql_header_{len_:11, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:32.371690] INFO [SERVER] run (obmp_disconnect.cpp:82) [21653][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] free session successfully(sessid=3221737333) [2024-10-16 14:57:32.371695] INFO [SERVER] run (obmp_disconnect.cpp:89) [21653][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221737333) [2024-10-16 14:57:32.371723] INFO [SERVER] run (obmp_disconnect.cpp:82) [21653][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221732213) [2024-10-16 14:57:32.371726] INFO [SERVER] run (obmp_disconnect.cpp:89) [21653][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221732213) [2024-10-16 14:57:32.379618] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27406030, session_id:3221737991, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061846894254, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:172, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:32.379662] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=37] kill and revert session(conn.sessid_=3221737991, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:32.379669] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] sock ref clean, do destroy(*s={this:0x2b0f27406030, session_id:3221737991, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061846894254, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:172, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:32.379683] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=7] unlock session of tenant(conn.sessid_=3221737991, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:32.379689] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] connection close(sessid=3221737991, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B3A0A55C5-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:13, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:9, rec_:9, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:1, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{receive:obp_mysql_header_{len_:15, rec_:15, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[8]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_mysql_header_{pkt_num_:1, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[10]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[11]:{send:obp_mysql_header_{pkt_num_:12, seq_:16}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[12]:{send:obp_mysql_header_{len_:9, seq_:17}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:32.380051] INFO [SERVER] run (obmp_disconnect.cpp:82) [21659][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=9] free session successfully(sessid=3221737991) [2024-10-16 14:57:32.380061] INFO [SERVER] run (obmp_disconnect.cpp:89) [21659][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=9] mark session id unused(sessid=3221737991) [2024-10-16 14:57:32.380826] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f1adbe6a0, key:{tablet_id:{id:377}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f1adbe6a0, addr:[7667511-10399-0-2305844829206122496-0](offset=36864,size=1695,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:192512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:17, occupy_size:3078, data_checksum:372883027, max_merged_trans_version:1729000842663168000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:92, row_count_:17, occupy_size_:3078, original_size_:9838, data_checksum_:372883027, index_type_:0, rowkey_column_count_:5, column_cnt_:45, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1722494633924376, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729000842663168000, max_merged_trans_version_:1729000842663168000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f1adbe9b8, column_checksum_count:45, data_root_info:{addr:[0-0-0-3458764944391012352-0](offset=0,size=401,type=3,seq=0), orig_block_buf:0x2b0f1adbeb90, block_data:{buf:0x2b0f1adbeb20, size:112, extra_buf:0x2b0f1adbed21, extra_size:1108, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765249333690368-0](offset=0,size=685,type=3,seq=0), orig_block_buf:0x2b0f1adbf175, block_data:{buf:0x2b0f1adbf175, size:685, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f1adbf422, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:192512, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.380894] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=67] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f1adbf450, key:{tablet_id:{id:377}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}, ref_cnt:0, upper_trans_version:1729051493406896000, timestamp:0}, this:0x2b0f1adbf450, addr:[7667511-10399-0-2305844002424919711-0](offset=38559,size=925,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2072576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:1112, data_checksum:3450587089, max_merged_trans_version:1729051493406896000, upper_trans_version:1729051493406896000, filled_tx_scn:{val:1729051758122012000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:1112, original_size_:495, data_checksum_:3450587089, index_type_:0, rowkey_column_count_:5, column_cnt_:45, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729051493406896000, max_merged_trans_version_:1729051493406896000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729051758122012000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f1adbf768, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764797288382464-0](offset=0,size=264,type=3,seq=0), orig_block_buf:0x2b0f1adbf7a8, block_data:{buf:0x2b0f1adbf768, size:64, extra_buf:0x2b0f1adbf8b0, extra_size:1144, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764845606764544-0](offset=0,size=309,type=3,seq=0), orig_block_buf:0x2b0f1adbfd28, block_data:{buf:0x2b0f1adbfd28, size:309, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f1adbfe5d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2072576, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.380931] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0e931ea050, key:{tablet_id:{id:377}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0e931ea050, addr:[7667511-10399-0-2305843896124480060-0](offset=39484,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:876544, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:625, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:625, original_size_:328, data_checksum_:3146711327, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0e931ea368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0e931ea3a8, block_data:{buf:0x2b0e931ea368, size:64, extra_buf:0x2b0e931ea495, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0e931ea864, block_data:{buf:0x2b0e931ea864, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e931ea959, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:876544, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.381007] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.381188] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:213058225166}], scn_array=[cnt:1, {val:1729061852134673002, v:0}], curr_size=122, accum_size=1098, time_cost=10874) [2024-10-16 14:57:32.382371] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=1140, avg_log_submit_to_slide_time=2786) [2024-10-16 14:57:32.393863] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] submit log success(service_type_=0, last_id=1729061869029444000, limited_id=1729061869029444000) [2024-10-16 14:57:32.393882] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] submit log with lock success(service_type_=0, last_id=1729061869029444000, limited_id=1729061869029444000) [2024-10-16 14:57:32.393887] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061852392712000, start_id=1729061852392712000, end_id=1729061852392712001) [2024-10-16 14:57:32.393931] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:32.394001] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=16] no need to update min/max version(new_version={val:1729061851634232003, v:0}, min_version={val:1729061851634232003, v:0}, max_version={val:1729061851634232003, v:0}) [2024-10-16 14:57:32.395676] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6a1d80a0, start_pos=10380, remain=55068) [2024-10-16 14:57:32.395807] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.397713] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.406959] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:114264632301}], scn_array=[cnt:1, {val:1729061852392712002, v:0}], curr_size=593, accum_size=1907, time_cost=11252) [2024-10-16 14:57:32.408478] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] update id meta(ret=0, service_type=0, limited_id=1729061869029444000, latest_scn={val:1729061852392712001, v:0}, *this={tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061869029444000, latest_log_ts:{val:1729061852392712001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:32.408517] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=38] handle submit callback(service_type_=0, success=true, used_ts=14656, log_ts={val:1729061852392712001, v:0}, limited_id=1729061869029444000) [2024-10-16 14:57:32.408525] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:32.408530] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:32.408546] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061852406635000, start_id=1729061852406635000, end_id=1729061852406635001) [2024-10-16 14:57:32.408992] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:377}, table_store_addr_.addr_=[7667511-10399-0-2305843504208715776-0](offset=40960,size=461,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305844138790134221-0](offset=41421,size=1052,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686767900196864-0](offset=1015808,size=698,type=4,seq=0), this=0x2b0e6a1d80a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e931eb760, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e931eb780, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.409025] WDIAG [STORAGE] print_time_stats (ob_tablet_persister.cpp:259) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=28][errcode=0] [TABLET PERSISTER TIME STATS] cost too much time (multi_stats={stats[0]: owner:'persist_and_transform_tablet' total=45071, time_dist: persist_and_fill_tablet=31821, check_tablet_meta_ids=2, persist_aggregated_meta=13248 stats[1]: owner:'persist_and_fill_tablet' total=31736, time_dist: convert_tablet_to_disk_arg=20065, init_tabelt_macro_info=37, transform=11634 stats[2]: owner:'convert_tablet_to_disk_arg' total=20045, time_dist: fetch_table_store_and_write_info=16998, load_storage_schema=16, write_and_fill_args=3031 stats[3]: owner:'persist_and_fill_tablet' total=16996, time_dist: fetch_table_store=0, get_all_sstable=0, fetch_and_persist_sstable=16847, fill_write_info=149 stats[4]: owner:'fetch_and_persist_sstable' total=16843, time_dist: fill_all_sstable_write_info=119, batch_write_sstable_info=16711, init_new_table_store=13 extra_info:large_co_sst_cnt:0,small_co_sst_cnt:0,normal_sst_cnt:3,cg_sst_cnt:0 stats[5]: owner:'transform' total=11626, time_dist: before_load_table_store=4, load_table_store=347, cache_table_store=11261, cache_macro_info=14, init_table_store_cache=0 stats[6]: owner:'load_table_store' total=344, time_dist: read_io=335, deserialize_table_store=9}) [2024-10-16 14:57:32.417253] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601352, offset=9615220}, end_cursor:ObLogCursor{file_id=192, log_id=47601353, offset=9615336}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dd8050, len:2188, log_data_len:116, seq:47601352, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.417294] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=17] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686767900196864-0](offset=1015808,size=698,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6a1d80a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6a1d80a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686767900196864-0](offset=1015808,size=698,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:377}, data_tablet_id:{id:377}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729001176236986000, v:0}, clog_checkpoint_scn:{val:1729051758122012000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729051758122012000, multi_version_start:1729015257002301000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:372883027, row_count:17, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:16384, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:16384}}, table_store_addr:{ptr:0x2b0e6a1d86d4, addr:[7667511-10399-0-2305843504208715776-0](offset=40960,size=461,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305844138790134221-0](offset=41421,size=1052,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a1da92c, addr:[7663823-15172-0-4611686241766703594-0](offset=1016298,size=208,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.417369] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=53] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:377}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f334e0060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8487dda0, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e8487ddc0, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.417392] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=17] reset tablet(this=0x2b0f334e0060, ls_id={id:1}, tablet_id={id:377}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.417838] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061852416337000, start_id=1729061852416337000, end_id=1729061852416337001) [2024-10-16 14:57:32.420633] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:6, col_desc_array:[cnt:6, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:6, 2197175160, 2197175160, 2197175160, 0, 0, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:5, index_type:8, row_store_type:1, schema_version:1717560328798944, column_cnt:4, store_column_cnt:4, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:3, rowkey_array:[cnt:3, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:4, column_array:[cnt:4, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:4, rowkey_column_cnt:5, schema_rowkey_column_cnt:3, column_cnt:6, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:6, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.420952] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=61] Async write macro block(macro_id_=[14770](ver=0,mode=0,seq=7667497)) [2024-10-16 14:57:32.431937] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.432084] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=26] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:377}, table_store_addr_.addr_=[7663656-3934-0-2305843504209862656-0](offset=1187840,size=461,type=2,seq=0), storage_schema_addr_.addr_=[7663656-3934-0-2305844138791281101-0](offset=1188301,size=1052,type=2,seq=0), tablet_addr_=[7663348-6841-0-4611686767901044736-0](offset=1863680,size=698,type=4,seq=0), this=0x2b0e6a8a60a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6a8a89ac, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e6a8a89c4, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.432125] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=34] reset tablet(this=0x2b0e6a8a60a0, ls_id={id:1}, tablet_id={id:377}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.432141] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to gc tablet(ret=0, tablet=0x2b0e6a8a60a0) [2024-10-16 14:57:32.432146] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:827) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] gc tablets in queue(gc_tablets_cnt=1, err_tablets_cnt=0, tablet_gc_queue_.count()=0) [2024-10-16 14:57:32.432679] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:2076672, macro_id:[14770](ver=0,mode=0,seq=7667497)}, offset_=2080768, old_block=[]) [2024-10-16 14:57:32.432718] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:5, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea17fe408, hex: 8A9C2E0000000000, int: 3054730}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea17fe440, hex: 0000000000000000, int: 0}, idx=2:null, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea17fe520, hex: E80D715F7D2301E8, int: -1729061760554431000}, idx=4:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea17fe4e8, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:6, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:745, meta_block_size:222, last_macro_size:967, macro_metas:0x2b0ea17fe100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.432781] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=17] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), buf:0x2b0ebd3ff060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.432792] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), buf:0x2b0ebd3fe560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), buf:0x2b0ebd3ff060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:6, data_column_checksums:[0, 0, 0, 0, 0, 0], row_count:12, max_merged_trans_version:1729061761547003000, contain_uncommitted_row:false, occupy_size:561, original_size:601, data_checksum:1956896740, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.432822] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0ef9088060, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0ef9088060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2076672, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:561, data_checksum:1956896740, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:12, occupy_size_:561, original_size_:601, data_checksum_:1956896740, index_type_:8, rowkey_column_count_:5, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0ef9088390, block_data:{buf:0x2b0ef90885b0, size:64, extra_buf:0x2b0ef90885f0, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0ef9088490, block_data:{buf:0x2b0ef9088490, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9088590, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2076672, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.432873] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=48] finish set upper trans version(ret=0, key_={tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0ef9088130, old_val=1729061761547003000, upper_trans_version=1729061761547003000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2076672, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:561, data_checksum:1956896740, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.432883] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:2076672, macro_id:[14770](ver=0,mode=0,seq=7667497)}, new_sstable.get_key()={tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.434102] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f3d7245c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f3d725150, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f3d7245c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f3d725150, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f56f04a20, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015201157706000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061401350833000 1729061401350833000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061761547003000 1729061761547003000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328401377000 1717560328401377000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.434336] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=39] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846982}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f3d724060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f56f05b60, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f56f05b80, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.434357] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e0c0f20a0, is_inited:true, wash_score:1729061852001752000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686804406853632-0](offset=450560,size=732,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846982}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3506043244, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332160, space_usage:{shared_data_size:724992, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:724992}}, table_store_addr:{ptr:0x2b0e0c0f26d4, addr:[7663857-6566-0-2305843631984676864-0](offset=724992,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843276576133700-0](offset=725572,size=249,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e0c0f5078, addr:[7663823-15172-0-4611686276125876716-0](offset=451052,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f3d724060, is_inited:true, wash_score:1729061852424730000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846982}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3506043244, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332160, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2959, meta_size:0, occupy_bytes:724992}}, table_store_addr:{ptr:0x2b0f3d724450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0f56f05360, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0f56f05ad0, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.434452] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=64] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f3d7245d0, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f3d7245d0, addr:[7663857-6566-0-2305844358907879424-0](offset=712704,size=1257,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:704512, nested_offset:421888, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90523, occupy_size:702354, data_checksum:3506043244, max_merged_trans_version:1729015201157706000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90523, occupy_size_:702354, original_size_:4345104, data_checksum_:3506043244, index_type_:8, rowkey_column_count_:5, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:45, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328332160, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201157706000, max_merged_trans_version_:1729015201157706000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f3d7248e8, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764930432368640-0](offset=0,size=388,type=3,seq=0), orig_block_buf:0x2b0f3d724988, block_data:{buf:0x2b0f3d724918, size:112, extra_buf:0x2b0f3d724b0c, extra_size:1095, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765019552940032-0](offset=0,size=471,type=3,seq=0), orig_block_buf:0x2b0f3d724f53, block_data:{buf:0x2b0f3d724f53, size:471, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f3d72512a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:421888, nested_size:704512}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1169, old_pos=0, pos=1169) [2024-10-16 14:57:32.434522] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=67] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f3d725170, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401350833000, timestamp:0}, this:0x2b0f3d725170, addr:[7663857-6566-0-2305843942296052969-0](offset=713961,size=869,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:12288, nested_offset:229376, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:430, occupy_size:9585, data_checksum:0, max_merged_trans_version:1729061401350833000, upper_trans_version:1729061401350833000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:430, occupy_size_:9585, original_size_:21493, data_checksum_:4142536556, index_type_:8, rowkey_column_count_:5, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401350833000, max_merged_trans_version_:1729061401350833000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f3d725488, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0f3d7254c8, block_data:{buf:0x2b0f3d725488, size:64, extra_buf:0x2b0f3d7255ba, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), orig_block_buf:0x2b0f3d725a19, block_data:{buf:0x2b0f3d725a19, size:265, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f3d725b22, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:229376, nested_size:12288}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=781, old_pos=0, pos=781) [2024-10-16 14:57:32.434553] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=27] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f04050, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0f56f04050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2076672, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:561, data_checksum:1956896740, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:12, occupy_size_:561, original_size_:601, data_checksum_:1956896740, index_type_:8, rowkey_column_count_:5, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f04368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0f56f043a8, block_data:{buf:0x2b0f56f04368, size:64, extra_buf:0x2b0f56f0449a, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0f56f048f9, block_data:{buf:0x2b0f56f048f9, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f049f8, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2076672, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=761, old_pos=0, pos=761) [2024-10-16 14:57:32.434580] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f04a30, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328401377000, timestamp:0}, this:0x2b0f56f04a30, addr:[7663857-6566-0-2305843903641349023-0](offset=715679,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1175552, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328401377000, upper_trans_version:1717560328401377000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:965475280, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328401377000, max_merged_trans_version_:1717560328401377000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f56f04d48, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f56f04d88, block_data:{buf:0x2b0f56f04d48, size:64, extra_buf:0x2b0f56f04e75, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0f56f05244, block_data:{buf:0x2b0f56f05244, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f05340, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1175552, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=745, old_pos=0, pos=745) [2024-10-16 14:57:32.434643] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=37] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.434816] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f1773c650, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f1773c650, addr:[7667511-10399-0-2305844358907211776-0](offset=45056,size=1257,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:704512, nested_offset:421888, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90523, occupy_size:702354, data_checksum:3506043244, max_merged_trans_version:1729015201157706000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90523, occupy_size_:702354, original_size_:4345104, data_checksum_:3506043244, index_type_:8, rowkey_column_count_:5, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:45, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328332160, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201157706000, max_merged_trans_version_:1729015201157706000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f1773c968, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764930432368640-0](offset=0,size=388,type=3,seq=0), orig_block_buf:0x2b0f1773ca08, block_data:{buf:0x2b0f1773c998, size:112, extra_buf:0x2b0f1773cb8c, extra_size:1095, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765019552940032-0](offset=0,size=471,type=3,seq=0), orig_block_buf:0x2b0f1773cfd3, block_data:{buf:0x2b0f1773cfd3, size:471, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f1773d1aa, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:421888, nested_size:704512}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.434851] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=33] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f1773d1f0, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401350833000, timestamp:0}, this:0x2b0f1773d1f0, addr:[7667511-10399-0-2305843942295385321-0](offset=46313,size=869,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:12288, nested_offset:229376, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:430, occupy_size:9585, data_checksum:0, max_merged_trans_version:1729061401350833000, upper_trans_version:1729061401350833000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:430, occupy_size_:9585, original_size_:21493, data_checksum_:4142536556, index_type_:8, rowkey_column_count_:5, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401350833000, max_merged_trans_version_:1729061401350833000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f1773d508, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0f1773d548, block_data:{buf:0x2b0f1773d508, size:64, extra_buf:0x2b0f1773d63a, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), orig_block_buf:0x2b0f1773da99, block_data:{buf:0x2b0f1773da99, size:265, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f1773dba2, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:229376, nested_size:12288}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.434877] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f009fc050, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0f009fc050, addr:[7667511-10399-0-2305843920820549710-0](offset=47182,size=849,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2076672, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:561, data_checksum:1956896740, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:12, occupy_size_:561, original_size_:601, data_checksum_:1956896740, index_type_:8, rowkey_column_count_:5, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f009fc368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0f009fc3a8, block_data:{buf:0x2b0f009fc368, size:64, extra_buf:0x2b0f009fc49a, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0f009fc8f9, block_data:{buf:0x2b0f009fc8f9, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f009fc9f8, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2076672, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.434902] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f009fca30, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328401377000, timestamp:0}, this:0x2b0f009fca30, addr:[7667511-10399-0-2305843903640681375-0](offset=48031,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1175552, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328401377000, upper_trans_version:1717560328401377000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:965475280, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328401377000, max_merged_trans_version_:1717560328401377000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f009fcd48, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f009fcd88, block_data:{buf:0x2b0f009fcd48, size:64, extra_buf:0x2b0f009fce75, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0f009fd244, block_data:{buf:0x2b0f009fd244, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f009fd340, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1175552, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.434946] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.452778] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D18-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:32.452802] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D18-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:32.452812] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D18-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:32.452826] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D18-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173056, log_task={header:{begin_lsn:{lsn:14352363035}, end_lsn:{lsn:14352363157}, log_id:115173056, min_scn:{val:1729061852135733002, v:0}, max_scn:{val:1729061852135733002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352362913}, prev_proposal_id:17, committed_end_lsn:{lsn:14352363035}, data_checksum:940009244, accum_checksum:2056013894, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061852348149, freeze_ts:1729061852348149, submit_ts:1729061852348151, flushed_ts:1729061852360291, gen_to_freeze cost time:0, gen_to_submit cost time:2, submit_to_flush cost time:12140}, fs_cb_begin_ts=1729061852452819, log_life_time=104670) [2024-10-16 14:57:32.456748] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [55867][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BA-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:32.456772] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [55867][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BA-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061852135733001, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061852135733001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:32.457135] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0df1c5e0a0, start_pos=12248, remain=53200) [2024-10-16 14:57:32.457253] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.457517] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:32.457563] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:32.458073] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846982}, table_store_addr_.addr_=[7667511-10399-0-2305843631984001024-0](offset=49152,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843276575457860-0](offset=49732,size=249,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686804407422976-0](offset=1019904,size=732,type=4,seq=0), this=0x2b0df1c5e0a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f009fd880, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f009fd8a0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.460433] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A76-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:32.460449] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A76-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:32.470810] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9BA-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061852470610000, start_id=1729061852470610000, end_id=1729061852470610001) [2024-10-16 14:57:32.470848] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B99-0-0] [lt=5] init point thread id with(&point=0x55e3435c50c0, idx_=649, point=[thread id=107145, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107145) [2024-10-16 14:57:32.470889] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B99-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178363, log_task={header:{begin_lsn:{lsn:14538476800}, end_lsn:{lsn:14538476922}, log_id:115178363, min_scn:{val:1729061852134673002, v:0}, max_scn:{val:1729061852134673002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538476678}, prev_proposal_id:23, committed_end_lsn:{lsn:14538476800}, data_checksum:1715074492, accum_checksum:3946707059, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061852370401, freeze_ts:1729061852370401, submit_ts:1729061852370404, flushed_ts:1729061852381453, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:11049}, fs_cb_begin_ts=1729061852470884, log_life_time=100483) [2024-10-16 14:57:32.471388] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601353, offset=9615336}, end_cursor:ObLogCursor{file_id=192, log_id=47601354, offset=9615452}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a5c050, len:2072, log_data_len:116, seq:47601353, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.471548] WDIAG [STORAGE] ~ObOccamTimeGuard (ob_occam_time_guard.h:269) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26][errcode=0] cost too much time:ob_tablet_slog_helper.cpp:static int oceanbase::storage::ObTabletSlogHelper::write_update_tablet_slog(const share::ObLSID &, const common::ObTabletID &, const oceanbase::storage::ObMetaDiskAddr &), (*this=|threshold=10.00ms|start at 14:57:32.458|45=0us|53=10.55ms|57=2.88ms|total=13.45ms) [2024-10-16 14:57:32.471575] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=20] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686804407422976-0](offset=1019904,size=732,type=4,seq=0), new_tablet_hdl={obj:0x2b0df1c5e0a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0df1c5e0a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686804407422976-0](offset=1019904,size=732,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846982}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3506043244, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332160, space_usage:{shared_data_size:724992, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:724992}}, table_store_addr:{ptr:0x2b0df1c5e6d4, addr:[7667511-10399-0-2305843631984001024-0](offset=49152,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843276575457860-0](offset=49732,size=249,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0df1c61078, addr:[7663823-15172-0-4611686276126446060-0](offset=1020396,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.471666] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=57] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846982}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f3d724060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f56f05b60, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f56f05b80, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.471688] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=17] reset tablet(this=0x2b0f3d724060, ls_id={id:1001}, tablet_id={id:1152921504606846982}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.472257] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061852470610000, start_id=1729061852470610000, end_id=1729061852470610001) [2024-10-16 14:57:32.473347] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:4, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:4, col_desc_array:[cnt:4, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:4, 2197175160, 2197175160, 0, 0], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:5, index_type:7, row_store_type:1, schema_version:1717560328798944, column_cnt:2, store_column_cnt:2, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:2, column_array:[cnt:2, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:2, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:4, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:4, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.477757] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846982}, table_store_addr_.addr_=[7663857-6566-0-2305843631984676864-0](offset=724992,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843276576133700-0](offset=725572,size=249,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686804406853632-0](offset=450560,size=732,type=4,seq=0), this=0x2b0e0c0f20a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e0c0f50f8, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e0c0f5110, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.477786] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=24] reset tablet(this=0x2b0e0c0f20a0, ls_id={id:1001}, tablet_id={id:1152921504606846982}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.477804] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to gc tablet(ret=0, tablet=0x2b0e0c0f20a0) [2024-10-16 14:57:32.477808] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:827) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] gc tablets in queue(gc_tablets_cnt=1, err_tablets_cnt=0, tablet_gc_queue_.count()=0) [2024-10-16 14:57:32.477861] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=46] Async write macro block(macro_id_=[14770](ver=0,mode=0,seq=7667497)) [2024-10-16 14:57:32.490614] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:2080768, macro_id:[14770](ver=0,mode=0,seq=7667497)}, offset_=2084864, old_block=[]) [2024-10-16 14:57:32.490653] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=16] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36f30408, hex: 8A9C2E0000000000, int: 3054730}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36f30440, hex: 8A9C2E0000000000, int: 3054730}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36f304e0, hex: E80D715F7D2301E8, int: -1729061760554431000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36f304b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:4, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:696, meta_block_size:225, last_macro_size:921, macro_metas:0x2b0f36f30100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.490713] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), buf:0x2b0ef7ed1060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.490725] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), buf:0x2b0ef7ed0560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), buf:0x2b0ef7ed1060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:4, data_column_checksums:[0, 0, 0, 0], row_count:12, max_merged_trans_version:1729061761547003000, contain_uncommitted_row:false, occupy_size:510, original_size:577, data_checksum:227810858, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.490756] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0eda24a060, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0eda24a060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2080768, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:510, data_checksum:227810858, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:12, occupy_size_:510, original_size_:577, data_checksum_:227810858, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), orig_block_buf:0x2b0eda24a390, block_data:{buf:0x2b0eda24a5b0, size:64, extra_buf:0x2b0eda24a5f0, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0eda24a490, block_data:{buf:0x2b0eda24a490, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0eda24a590, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2080768, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.490806] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=49] finish set upper trans version(ret=0, key_={tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0eda24a130, old_val=1729061761547003000, upper_trans_version=1729061761547003000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2080768, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:510, data_checksum:227810858, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.490817] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:2080768, macro_id:[14770](ver=0,mode=0,seq=7667497)}, new_sstable.get_key()={tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.491087] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:32.491101] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:32.491796] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:32.492644] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f1adbe5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f1adbf060, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f1adbe5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f1adbf060, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f36e76990, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015201157706000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061401350833000 1729061401350833000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061761547003000 1729061761547003000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328401377000 1717560328401377000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.492838] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=30] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846983}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f1adbe060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f36e77a00, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f36e77a20, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.492860] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0df1ca60a0, is_inited:true, wash_score:1729061852081028000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964406784-0](offset=454656,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846983}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:443971405, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332168, space_usage:{shared_data_size:724992, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:724992}}, table_store_addr:{ptr:0x2b0df1ca66b0, addr:[7663857-6566-0-2305843630910943232-0](offset=733184,size=579,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843227184017987-0](offset=733763,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0df1ca8e46, addr:[7663823-15172-0-4611686276125880806-0](offset=455142,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f1adbe060, is_inited:true, wash_score:1729061852492248000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846983}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:443971405, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332168, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2873, meta_size:0, occupy_bytes:724992}}, table_store_addr:{ptr:0x2b0f1adbe450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0f36e772d0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0f36e77970, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.492984] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [21236][DBMS_SCHEDULER][T1002][YB4A0A000407-00061E99757960C1-0-0] [lt=11] refresh tablet status cache(ret=0, ls_id={id:1}, tablet_id={id:377}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560287003454000, delete_commit_version:-1}, snapshot_version=1729061852071065000) [2024-10-16 14:57:32.492960] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=65] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f1adbe5d0, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f1adbe5d0, addr:[7663857-6566-0-2305844264418615296-0](offset=729088,size=1169,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:704512, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90523, occupy_size:702665, data_checksum:443971405, max_merged_trans_version:1729015201157706000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90523, occupy_size_:702665, original_size_:2896736, data_checksum_:443971405, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:46, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328332168, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201157706000, max_merged_trans_version_:1729015201157706000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f1adbe8e8, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f1adbe970, block_data:{buf:0x2b0f1adbe908, size:104, extra_buf:0x2b0f1adbead6, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764972308299776-0](offset=0,size=427,type=3,seq=0), orig_block_buf:0x2b0f1adbee8d, block_data:{buf:0x2b0f1adbee8d, size:427, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f1adbf038, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:704512}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1081, old_pos=0, pos=1081) [2024-10-16 14:57:32.493005] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=42] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f1adbf080, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401350833000, timestamp:0}, this:0x2b0f1adbf080, addr:[7663857-6566-0-2305843944443552913-0](offset=730257,size=871,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:12288, nested_offset:217088, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:430, occupy_size:9202, data_checksum:0, max_merged_trans_version:1729061401350833000, upper_trans_version:1729061401350833000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:430, occupy_size_:9202, original_size_:19893, data_checksum_:944109238, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401350833000, max_merged_trans_version_:1729061401350833000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f1adbf398, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), orig_block_buf:0x2b0f1adbf3d8, block_data:{buf:0x2b0f1adbf398, size:64, extra_buf:0x2b0f1adbf4cc, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), orig_block_buf:0x2b0f1adbf89b, block_data:{buf:0x2b0f1adbf89b, size:265, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f1adbf9a4, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:217088, nested_size:12288}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=783, old_pos=0, pos=783) [2024-10-16 14:57:32.493035] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=27] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f36e76050, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0f36e76050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2080768, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:510, data_checksum:227810858, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:12, occupy_size_:510, original_size_:577, data_checksum_:227810858, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e76368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), orig_block_buf:0x2b0f36e763a8, block_data:{buf:0x2b0f36e76368, size:64, extra_buf:0x2b0f36e7649c, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0f36e7686b, block_data:{buf:0x2b0f36e7686b, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e7696a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2080768, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=762, old_pos=0, pos=762) [2024-10-16 14:57:32.493062] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f36e769a0, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328401377000, timestamp:0}, this:0x2b0f36e769a0, addr:[7663857-6566-0-2305843903641365322-0](offset=731978,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:217088, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328401377000, upper_trans_version:1717560328401377000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:965475280, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328401377000, max_merged_trans_version_:1717560328401377000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f36e76cb8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f36e76cf8, block_data:{buf:0x2b0f36e76cb8, size:64, extra_buf:0x2b0f36e76de5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0f36e771b4, block_data:{buf:0x2b0f36e771b4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e772b0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:217088, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=745, old_pos=0, pos=745) [2024-10-16 14:57:32.493102] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EEB-0-0] [lt=18][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497323, log_task={header:{begin_lsn:{lsn:207589536129}, end_lsn:{lsn:207589536251}, log_id:218497323, min_scn:{val:1729061852134673003, v:0}, max_scn:{val:1729061852134673003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589536007}, prev_proposal_id:81, committed_end_lsn:{lsn:207589536129}, data_checksum:940009244, accum_checksum:3502206867, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061852381894, freeze_ts:1729061852381895, submit_ts:1729061852381898, flushed_ts:1729061852394642, gen_to_freeze cost time:1, gen_to_submit cost time:4, submit_to_flush cost time:12744}, fs_cb_begin_ts=1729061852493097, log_life_time=111203) [2024-10-16 14:57:32.493130] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=42] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.493154] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41D6-0-0] [lt=25] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:57:32.493257] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:32.493266] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1855, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:32.493283] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=694, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:32.493290] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=622, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:32.493300] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:32.495342] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061852492248000, start_id=1729061852492248001, end_id=1729061852492248002) [2024-10-16 14:57:32.496007] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f334e0650, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f334e0650, addr:[7667511-10399-0-2305844264417939456-0](offset=53248,size=1169,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:704512, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90523, occupy_size:702665, data_checksum:443971405, max_merged_trans_version:1729015201157706000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90523, occupy_size_:702665, original_size_:2896736, data_checksum_:443971405, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:46, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328332168, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201157706000, max_merged_trans_version_:1729015201157706000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334e0968, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f334e09f0, block_data:{buf:0x2b0f334e0988, size:104, extra_buf:0x2b0f334e0b56, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764972308299776-0](offset=0,size=427,type=3,seq=0), orig_block_buf:0x2b0f334e0f0d, block_data:{buf:0x2b0f334e0f0d, size:427, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334e10b8, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:704512}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.496068] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=59] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f334e1100, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401350833000, timestamp:0}, this:0x2b0f334e1100, addr:[7667511-10399-0-2305843944442877073-0](offset=54417,size=871,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:12288, nested_offset:217088, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:430, occupy_size:9202, data_checksum:0, max_merged_trans_version:1729061401350833000, upper_trans_version:1729061401350833000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:430, occupy_size_:9202, original_size_:19893, data_checksum_:944109238, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401350833000, max_merged_trans_version_:1729061401350833000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334e1418, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), orig_block_buf:0x2b0f334e1458, block_data:{buf:0x2b0f334e1418, size:64, extra_buf:0x2b0f334e154c, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), orig_block_buf:0x2b0f334e191b, block_data:{buf:0x2b0f334e191b, size:265, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334e1a24, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:217088, nested_size:12288}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.496096] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=28] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e72050, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0f36e72050, addr:[7667511-10399-0-2305843921894299640-0](offset=55288,size=850,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2080768, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:510, data_checksum:227810858, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:12, occupy_size_:510, original_size_:577, data_checksum_:227810858, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e72368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), orig_block_buf:0x2b0f36e723a8, block_data:{buf:0x2b0f36e72368, size:64, extra_buf:0x2b0f36e7249c, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0f36e7286b, block_data:{buf:0x2b0f36e7286b, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e7296a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2080768, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.496135] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=38] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e729a0, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328401377000, timestamp:0}, this:0x2b0f36e729a0, addr:[7667511-10399-0-2305843903640689482-0](offset=56138,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:217088, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328401377000, upper_trans_version:1717560328401377000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:965475280, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328401377000, max_merged_trans_version_:1717560328401377000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f36e72cb8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f36e72cf8, block_data:{buf:0x2b0f36e72cb8, size:64, extra_buf:0x2b0f36e72de5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0f36e731b4, block_data:{buf:0x2b0f36e731b4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e732b0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:217088, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.496196] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.496471] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9A279FA9BA-0-0] [lt=15] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.054802083969116211e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=302, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.073788187274310076e+03) [2024-10-16 14:57:32.497152] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9BA-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061852496588000, start_id=1729061852496588000, end_id=1729061852496588001) [2024-10-16 14:57:32.498048] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.501973] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:32.507067] WDIAG [MDS] ~ObOccamTimeGuard (ob_occam_time_guard.h:269) [21236][DBMS_SCHEDULER][T1002][YB4A0A000407-00061E99757960C1-0-0] [lt=14][errcode=0] cost too much time:ob_i_tablet_mds_interface.ipp:int oceanbase::storage::ObITabletMdsInterface::get_mds_data_from_tablet(const mds::DummyKey &, const share::SCN &, const int64_t, const common::ObFunction &) c..., (*this=|threshold=10.00ms|start at 14:57:32.493|137=0us|total=14.10ms) [2024-10-16 14:57:32.507108] WDIAG [MDS] ~ObOccamTimeGuard (ob_occam_time_guard.h:269) [21236][DBMS_SCHEDULER][T1002][YB4A0A000407-00061E99757960C1-0-0] [lt=38][errcode=0] cost too much time:ob_i_tablet_mds_interface.ipp:int oceanbase::storage::ObITabletMdsInterface::get_latest(OP &&, bool &) const [T = oceanbase::storage::ObTabletBindingMdsUserData, OP = oceanbase::storage::ReadBindingInfoOp], (*this=|threshold=10.00ms|start at 14:57:32.492|430=0us|442=1us|444=0us|451=1us|461=0us|472=14.11ms|total=14.12ms) [2024-10-16 14:57:32.507116] WDIAG [MDS] ~ObOccamTimeGuard (ob_occam_time_guard.h:269) [21236][DBMS_SCHEDULER][T1002][YB4A0A000407-00061E99757960C1-0-0] [lt=8][errcode=0] cost too much time:ob_i_tablet_mds_interface.cpp:int oceanbase::storage::ObITabletMdsInterface::get_latest_ddl_data(oceanbase::storage::ObTabletBindingMdsUserData &, bool &) const, (*this=|threshold=10.00ms|start at 14:57:32.492|123=0us|total=14.12ms) [2024-10-16 14:57:32.507122] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [21236][DBMS_SCHEDULER][T1002][YB4A0A000407-00061E99757960C1-0-0] [lt=5] refresh ddl data cache(ret=0, ls_id={id:1}, tablet_id={id:377}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:32.507423] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6b6ee0a0, start_pos=11686, remain=53762) [2024-10-16 14:57:32.507516] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.507733] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846983}, table_store_addr_.addr_=[7667511-10399-0-2305843630910267392-0](offset=57344,size=579,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843227183342147-0](offset=57923,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964976128-0](offset=1024000,size=726,type=4,seq=0), this=0x2b0e6b6ee0a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f36e73760, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f36e73780, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.507759] INFO [STORAGE] print_time_stats (ob_tablet_persister.cpp:262) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=22] [TABLET PERSISTER TIME STATS] (multi_stats={stats[0]: owner:'persist_and_transform_tablet' total=14830, time_dist: persist_and_fill_tablet=14567, check_tablet_meta_ids=1, persist_aggregated_meta=262 stats[1]: owner:'persist_and_fill_tablet' total=14506, time_dist: convert_tablet_to_disk_arg=5436, init_tabelt_macro_info=27, transform=9043 stats[2]: owner:'convert_tablet_to_disk_arg' total=5419, time_dist: fetch_table_store_and_write_info=3230, load_storage_schema=9, write_and_fill_args=2180 stats[3]: owner:'persist_and_fill_tablet' total=3215, time_dist: fetch_table_store=0, get_all_sstable=0, fetch_and_persist_sstable=3057, fill_write_info=158 stats[4]: owner:'fetch_and_persist_sstable' total=3053, time_dist: fill_all_sstable_write_info=159, batch_write_sstable_info=2883, init_new_table_store=11 extra_info:large_co_sst_cnt:0,small_co_sst_cnt:0,normal_sst_cnt:4,cg_sst_cnt:0 stats[5]: owner:'transform' total=9037, time_dist: before_load_table_store=4, load_table_store=8463, cache_table_store=559, cache_macro_info=10, init_table_store_cache=1 stats[6]: owner:'load_table_store' total=8458, time_dist: read_io=8449, deserialize_table_store=9}) [2024-10-16 14:57:32.508931] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601354, offset=9615452}, end_cursor:ObLogCursor{file_id=192, log_id=47601355, offset=9615568}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cb0050, len:1956, log_data_len:116, seq:47601354, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.508988] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=18) [2024-10-16 14:57:32.509116] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:32.509128] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686797964976128-0](offset=1024000,size=726,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6b6ee0a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6b6ee0a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964976128-0](offset=1024000,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846983}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:443971405, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332168, space_usage:{shared_data_size:724992, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:724992}}, table_store_addr:{ptr:0x2b0e6b6ee6b0, addr:[7667511-10399-0-2305843630910267392-0](offset=57344,size=579,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843227183342147-0](offset=57923,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6b6f0e46, addr:[7663823-15172-0-4611686276126450150-0](offset=1024486,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.509213] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=68] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846983}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f1adbe060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f36e77a00, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f36e77a20, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.509231] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] reset tablet(this=0x2b0f1adbe060, ls_id={id:1001}, tablet_id={id:1152921504606846983}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.509587] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1003}, tablet_id:{id:49402}}) [2024-10-16 14:57:32.509800] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:7, rowkey_column_count:3, schema_rowkey_col_cnt:1, full_stored_col_cnt:7, col_desc_array:[cnt:7, column_id=18 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:7, 2197175160, 0, 0, 2197175160, 2197175160, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:0, index_type:0, row_store_type:1, schema_version:1717560289212768, column_cnt:5, store_column_cnt:5, tablet_size:134217728, pctfree:10, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:1, encryption:"", encrypt_key:"", rowkey_cnt:1, rowkey_array:[cnt:1, {column_idx:18, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:5, column_array:[cnt:5, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:1, row_store_type:1, block_size:16384, schema_column_cnt:5, rowkey_column_cnt:3, schema_rowkey_column_cnt:1, column_cnt:7, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:7, column_id=18 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.510782] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=54] Async write macro block(macro_id_=[14770](ver=0,mode=0,seq=7667497)) [2024-10-16 14:57:32.510995] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:2084864, macro_id:[14770](ver=0,mode=0,seq=7667497)}, offset_=2088960, old_block=[]) [2024-10-16 14:57:32.511022] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:3, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea175c4b0, hex: 507154C498DFF417, int: 1726250404411634000}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea175c4c0, hex: E8771207472401E8, int: -1729060894453631000}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea175c478, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:7, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:594, meta_block_size:255, last_macro_size:849, macro_metas:0x2b0ea175c100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.511073] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=16] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), buf:0x2b0f0bd0f060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.511086] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), buf:0x2b0f0bd0e560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), buf:0x2b0f0bd0f060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:7, data_column_checksums:[0, 0, 0, 0, 0, 0, 0], row_count:1, max_merged_trans_version:1729060894453631000, contain_uncommitted_row:false, occupy_size:352, original_size:55, data_checksum:971252252, use_old_macro_block_count:0, compressor_type:1, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.511129] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0ef9082060, key:{tablet_id:{id:394}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729015204842020000, v:0}, end_scn:{val:1729060940167357001, v:0}}}, ref_cnt:0, upper_trans_version:1729060894453631000, timestamp:0}, this:0x2b0ef9082060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2084864, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:352, data_checksum:971252252, max_merged_trans_version:1729060894453631000, upper_trans_version:1729060894453631000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:1, occupy_size_:352, original_size_:55, data_checksum_:971252252, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060894453631000, max_merged_trans_version_:1729060894453631000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060940167357001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0ef9082390, block_data:{buf:0x2b0ef90825b0, size:64, extra_buf:0x2b0ef90825f0, extra_size:839, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0ef9082490, block_data:{buf:0x2b0ef9082490, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9082590, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2084864, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.511176] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=46] finish set upper trans version(ret=0, key_={tablet_id:{id:394}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729015204842020000, v:0}, end_scn:{val:1729060940167357001, v:0}}}, meta=0x2b0ef9082130, old_val=1729060894453631000, upper_trans_version=1729060894453631000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2084864, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:352, data_checksum:971252252, max_merged_trans_version:1729060894453631000, upper_trans_version:1729060894453631000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.511187] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:2084864, macro_id:[14770](ver=0,mode=0,seq=7667497)}, new_sstable.get_key()={tablet_id:{id:394}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729015204842020000, v:0}, end_scn:{val:1729060940167357001, v:0}}}) [2024-10-16 14:57:32.511321] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f3d7245c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f3d724f90, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f3d7245c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f3d724f90, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0e8487c900, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1728928802547408001 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINI 1729015204842020000 1729015204842020000 1728974598297198000 1729015204842020000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729060894453631000 1729060894453631000 1729015204842020000 1729060940167357001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560287003454000 1717560287003454000 1 1728978212337222000 0 false }}) [2024-10-16 14:57:32.511505] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=23] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:394}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f3d724060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8487da00, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e8487da20, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.511523] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=16] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e695b40a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663348-6841-0-4611686789375889408-0](offset=1871872,size=718,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:394}, data_tablet_id:{id:394}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1728974598297198000, v:0}, clog_checkpoint_scn:{val:1729060940167357001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729060939970781000, multi_version_start:1729015259994179003, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3589429118, row_count:33, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:16384, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:16384}}, table_store_addr:{ptr:0x2b0e695b468c, addr:[7663656-3934-0-2305843626616446976-0](offset=1204224,size=575,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663656-3934-0-2305843285166547519-0](offset=1204799,size=257,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e695b6c26, addr:[7663348-6841-0-4611686276127298014-0](offset=1872350,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f3d724060, is_inited:true, wash_score:1729061852510768000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:394}, data_tablet_id:{id:394}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1728974598297198000, v:0}, clog_checkpoint_scn:{val:1729060940167357001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729060939970781000, multi_version_start:1729015259994179003, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3589429118, row_count:33, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2764, meta_size:0, occupy_bytes:16384}}, table_store_addr:{ptr:0x2b0f3d724450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0e8487d240, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0e8487d970, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.511632] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=55] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f3d7245d0, key:{tablet_id:{id:394}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f3d7245d0, addr:[7663656-3934-0-2305844189257158656-0](offset=1200128,size=1099,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:999424, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:33, occupy_size:1414, data_checksum:3589429118, max_merged_trans_version:1728928802547408001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:92, row_count_:33, occupy_size_:1414, original_size_:1848, data_checksum_:3589429118, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560289212768, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728928802547408001, max_merged_trans_version_:1728928802547408001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f3d7248e8, column_checksum_count:7, data_root_info:{addr:[0-0-0-3458764857417924608-0](offset=0,size=320,type=3,seq=0), orig_block_buf:0x2b0f3d724980, block_data:{buf:0x2b0f3d724920, size:96, extra_buf:0x2b0f3d724ac0, extra_size:807, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764929358626816-0](offset=0,size=387,type=3,seq=0), orig_block_buf:0x2b0f3d724de7, block_data:{buf:0x2b0f3d724de7, size:387, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f3d724f6a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:999424, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1011, old_pos=0, pos=1011) [2024-10-16 14:57:32.511674] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=38] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f3d724fb0, key:{tablet_id:{id:394}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1728974598297198000, v:0}, end_scn:{val:1729015204842020000, v:0}}}, ref_cnt:0, upper_trans_version:1729015204842020000, timestamp:0}, this:0x2b0f3d724fb0, addr:[7663656-3934-0-2305843910084285515-0](offset=1201227,size=839,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1331200, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:384, data_checksum:0, max_merged_trans_version:1729015204842020000, upper_trans_version:1729015204842020000, filled_tx_scn:{val:1729015204842020000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:1, occupy_size_:384, original_size_:87, data_checksum_:2177823339, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015204842020000, max_merged_trans_version_:1729015204842020000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729015204842020000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f3d7252c8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0f3d725308, block_data:{buf:0x2b0f3d7252c8, size:64, extra_buf:0x2b0f3d7253fa, extra_size:839, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f3d725741, block_data:{buf:0x2b0f3d725741, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f3d725837, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1331200, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=751, old_pos=0, pos=751) [2024-10-16 14:57:32.511702] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=27] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0e8487c050, key:{tablet_id:{id:394}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729015204842020000, v:0}, end_scn:{val:1729060940167357001, v:0}}}, ref_cnt:0, upper_trans_version:1729060894453631000, timestamp:0}, this:0x2b0e8487c050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2084864, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:352, data_checksum:971252252, max_merged_trans_version:1729060894453631000, upper_trans_version:1729060894453631000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:1, occupy_size_:352, original_size_:55, data_checksum_:971252252, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060894453631000, max_merged_trans_version_:1729060894453631000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060940167357001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0e8487c368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0e8487c3a8, block_data:{buf:0x2b0e8487c368, size:64, extra_buf:0x2b0e8487c49a, extra_size:839, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0e8487c7e1, block_data:{buf:0x2b0e8487c7e1, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e8487c8e0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2084864, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=760, old_pos=0, pos=760) [2024-10-16 14:57:32.511729] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0e8487c910, key:{tablet_id:{id:394}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0e8487c910, addr:[7663656-3934-0-2305843896125643490-0](offset=1202914,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1740800, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:626, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:626, original_size_:329, data_checksum_:1828613749, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0e8487cc28, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0e8487cc68, block_data:{buf:0x2b0e8487cc28, size:64, extra_buf:0x2b0e8487cd55, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0e8487d124, block_data:{buf:0x2b0e8487d124, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e8487d219, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1740800, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=738, old_pos=0, pos=738) [2024-10-16 14:57:32.511779] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.518239] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846983}, table_store_addr_.addr_=[7663857-6566-0-2305843630910943232-0](offset=733184,size=579,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843227184017987-0](offset=733763,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964406784-0](offset=454656,size=726,type=4,seq=0), this=0x2b0df1ca60a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0df1ca8ec6, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0df1ca8ede, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.518263] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=19] reset tablet(this=0x2b0df1ca60a0, ls_id={id:1001}, tablet_id={id:1152921504606846983}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.518280] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to gc tablet(ret=0, tablet=0x2b0df1ca60a0) [2024-10-16 14:57:32.518284] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:827) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=5] gc tablets in queue(gc_tablets_cnt=1, err_tablets_cnt=0, tablet_gc_queue_.count()=0) [2024-10-16 14:57:32.522092] INFO [ARCHIVE] do_check_switch_archive_ (ob_archive_service.cpp:293) [22806][T1004_ArcSrv][T1004][YB4A0A000407-00061E9972A960C1-0-0] [lt=32] check_log_archive_state_, no switch round op(attr={key:{tenant_id:1004, dest_no:0}, incarnation:1, dest_id:1004, round_id:3, state:{status:"DOING"}, start_scn:{val:1728967049237798000, v:0}, checkpoint_scn:{val:1729061733534082001, v:0}, max_scn:{val:1729061734410085003, v:0}, compatible:{version:1}, base_piece_id:73, used_piece_id:74, piece_switch_interval:86400000000, frozen_input_bytes:597788630, frozen_output_bytes:597788630, active_input_bytes:43582031, active_output_bytes:43582031, deleted_input_bytes:0, deleted_output_bytes:0, path:"file:///mnt/oceanbasedata/bddb/archive", comment:""}) [2024-10-16 14:57:32.522115] INFO [ARCHIVE] print_self (ob_ls_task.cpp:414) [22806][T1004_ArcSrv][T1004][YB4A0A000407-00061E9972A960C1-0-0] [lt=20] print ls archive task(id={id:1}, tenant_id=1004, station={key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, round_start_scn={val:1728967049237798000, v:0}, dest={has_encount_error:false, is_worm:false, max_no_limit_lsn:{lsn:17343057821}, max_archived_info:{offset:{lsn:17493272856}, scn:{val:1729061734410085003, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, max_seq_log_offset:{lsn:17514344448}, max_fetch_info:{offset:{lsn:17493272856}, scn:{val:1729061734410085003, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, piece_min_lsn:{lsn:17479894340}, archive_file_id:261, archive_file_offset:13378516, piece_dir_exist:true, wait_send_task_count:0, tasks:}) [2024-10-16 14:57:32.522336] INFO [ARCHIVE] print_self (ob_ls_task.cpp:414) [22806][T1004_ArcSrv][T1004][YB4A0A000407-00061E9972A960C1-0-0] [lt=20] print ls archive task(id={id:1001}, tenant_id=1004, station={key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, round_start_scn={val:1728967049237798000, v:0}, dest={has_encount_error:false, is_worm:false, max_no_limit_lsn:{lsn:13531789325}, max_archived_info:{offset:{lsn:13645266451}, scn:{val:1729061734410085002, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, max_seq_log_offset:{lsn:13689372672}, max_fetch_info:{offset:{lsn:13645266451}, scn:{val:1729061734410085002, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, piece_min_lsn:{lsn:13635346983}, archive_file_id:204, archive_file_offset:9919468, piece_dir_exist:true, wait_send_task_count:0, tasks:}) [2024-10-16 14:57:32.522469] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ea170e650, key:{tablet_id:{id:394}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ea170e650, addr:[7667511-10399-0-2305844189256019968-0](offset=61440,size=1099,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:999424, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:33, occupy_size:1414, data_checksum:3589429118, max_merged_trans_version:1728928802547408001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:92, row_count_:33, occupy_size_:1414, original_size_:1848, data_checksum_:3589429118, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560289212768, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728928802547408001, max_merged_trans_version_:1728928802547408001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea170e968, column_checksum_count:7, data_root_info:{addr:[0-0-0-3458764857417924608-0](offset=0,size=320,type=3,seq=0), orig_block_buf:0x2b0ea170ea00, block_data:{buf:0x2b0ea170e9a0, size:96, extra_buf:0x2b0ea170eb40, extra_size:807, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764929358626816-0](offset=0,size=387,type=3,seq=0), orig_block_buf:0x2b0ea170ee67, block_data:{buf:0x2b0ea170ee67, size:387, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea170efea, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:999424, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.522520] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=49] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ea170f030, key:{tablet_id:{id:394}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1728974598297198000, v:0}, end_scn:{val:1729015204842020000, v:0}}}, ref_cnt:0, upper_trans_version:1729015204842020000, timestamp:0}, this:0x2b0ea170f030, addr:[7667511-10399-0-2305843910083146827-0](offset=62539,size=839,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1331200, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:384, data_checksum:0, max_merged_trans_version:1729015204842020000, upper_trans_version:1729015204842020000, filled_tx_scn:{val:1729015204842020000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:1, occupy_size_:384, original_size_:87, data_checksum_:2177823339, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015204842020000, max_merged_trans_version_:1729015204842020000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729015204842020000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea170f348, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0ea170f388, block_data:{buf:0x2b0ea170f348, size:64, extra_buf:0x2b0ea170f47a, extra_size:839, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0ea170f7c1, block_data:{buf:0x2b0ea170f7c1, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea170f8b7, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1331200, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.522548] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=28] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef977c050, key:{tablet_id:{id:394}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729015204842020000, v:0}, end_scn:{val:1729060940167357001, v:0}}}, ref_cnt:0, upper_trans_version:1729060894453631000, timestamp:0}, this:0x2b0ef977c050, addr:[7667511-10399-0-2305843919746824082-0](offset=63378,size=848,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2084864, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:352, data_checksum:971252252, max_merged_trans_version:1729060894453631000, upper_trans_version:1729060894453631000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:1, occupy_size_:352, original_size_:55, data_checksum_:971252252, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060894453631000, max_merged_trans_version_:1729060894453631000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060940167357001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef977c368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0ef977c3a8, block_data:{buf:0x2b0ef977c368, size:64, extra_buf:0x2b0ef977c49a, extra_size:839, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0ef977c7e1, block_data:{buf:0x2b0ef977c7e1, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef977c8e0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2084864, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.522610] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=62] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef977c910, key:{tablet_id:{id:394}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0ef977c910, addr:[7667511-10399-0-2305843896124504802-0](offset=64226,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1740800, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:626, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:626, original_size_:329, data_checksum_:1828613749, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ef977cc28, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ef977cc68, block_data:{buf:0x2b0ef977cc28, size:64, extra_buf:0x2b0ef977cd55, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0ef977d124, block_data:{buf:0x2b0ef977d124, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef977d219, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1740800, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.522662] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.524404] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=19] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a6bc70, tx_id:{txid:199664072}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061852523840, active_ts:1729061852523840, commit_ts:-1, finish_ts:1729061852523840, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061882523835, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061852522672, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:32.525195] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=34] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a65d90, tx_id:{txid:199664073}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061852524945, active_ts:1729061852524945, commit_ts:-1, finish_ts:1729061852524945, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061882524940, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061852524944, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:32.525673] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6a6a60a0, start_pos=11142, remain=54306) [2024-10-16 14:57:32.525789] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.526621] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C47-0-0] [lt=10] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:32.526641] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C47-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061852134673002, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061852134673002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:32.529337] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:394}, table_store_addr_.addr_=[7667511-10399-0-2305843626615308288-0](offset=65536,size=575,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843285165408831-0](offset=66111,size=257,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686789375045632-0](offset=1028096,size=718,type=4,seq=0), this=0x2b0e6a6a60a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef977d760, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ef977d780, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.529681] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601355, offset=9615568}, end_cursor:ObLogCursor{file_id=192, log_id=47601356, offset=9615684}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bb4050, len:1840, log_data_len:116, seq:47601355, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.529720] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=23] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686789375045632-0](offset=1028096,size=718,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6a6a60a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6a6a60a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686789375045632-0](offset=1028096,size=718,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:394}, data_tablet_id:{id:394}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1728974598297198000, v:0}, clog_checkpoint_scn:{val:1729060940167357001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729060939970781000, multi_version_start:1729015259994179003, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3589429118, row_count:33, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:16384, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:16384}}, table_store_addr:{ptr:0x2b0e6a6a668c, addr:[7667511-10399-0-2305843626615308288-0](offset=65536,size=575,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843285165408831-0](offset=66111,size=257,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a6a8c26, addr:[7663823-15172-0-4611686276126454238-0](offset=1028574,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.529781] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=46] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:394}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f3d724060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8487da00, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e8487da20, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.529798] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] reset tablet(this=0x2b0f3d724060, ls_id={id:1}, tablet_id={id:394}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.530471] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:7, rowkey_column_count:3, schema_rowkey_col_cnt:1, full_stored_col_cnt:7, col_desc_array:[cnt:7, column_id=18 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:7, 2197175160, 0, 0, 2197175160, 2197175160, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:0, index_type:0, row_store_type:1, schema_version:1717560289326944, column_cnt:5, store_column_cnt:5, tablet_size:134217728, pctfree:10, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:1, encryption:"", encrypt_key:"", rowkey_cnt:1, rowkey_array:[cnt:1, {column_idx:18, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:5, column_array:[cnt:5, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:1, row_store_type:1, block_size:16384, schema_column_cnt:5, rowkey_column_cnt:3, schema_rowkey_column_cnt:1, column_cnt:7, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:7, column_id=18 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.530636] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=54] Async write macro block(macro_id_=[14770](ver=0,mode=0,seq=7667497)) [2024-10-16 14:57:32.530890] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:2088960, macro_id:[14770](ver=0,mode=0,seq=7667497)}, offset_=2093056, old_block=[]) [2024-10-16 14:57:32.530920] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:3, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0e931ea4b0, hex: 60D0FD018DDBFE17, int: 1729060705627132000}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0e931ea4c0, hex: 90210DF8722401E8, int: -1729060705727078000}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0e931ea478, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:7, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:626, meta_block_size:246, last_macro_size:872, macro_metas:0x2b0e931ea100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.530969] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), buf:0x2b0f33469060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.530980] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), buf:0x2b0f33468560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), buf:0x2b0f33469060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:7, data_column_checksums:[0, 0, 0, 0, 0, 0, 0], row_count:1, max_merged_trans_version:1729060705727078000, contain_uncommitted_row:false, occupy_size:384, original_size:87, data_checksum:1604906901, use_old_macro_block_count:0, compressor_type:1, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.531006] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0eda24a060, key:{tablet_id:{id:431}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1728974598297198000, v:0}, end_scn:{val:1729060940167357001, v:0}}}, ref_cnt:0, upper_trans_version:1729060705727078000, timestamp:0}, this:0x2b0eda24a060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2088960, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:384, data_checksum:1604906901, max_merged_trans_version:1729060705727078000, upper_trans_version:1729060705727078000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:1, occupy_size_:384, original_size_:87, data_checksum_:1604906901, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060705727078000, max_merged_trans_version_:1729060705727078000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060940167357001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0eda24a390, block_data:{buf:0x2b0eda24a5b0, size:64, extra_buf:0x2b0eda24a5f0, extra_size:839, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0eda24a490, block_data:{buf:0x2b0eda24a490, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0eda24a590, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2088960, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.531049] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=41] finish set upper trans version(ret=0, key_={tablet_id:{id:431}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1728974598297198000, v:0}, end_scn:{val:1729060940167357001, v:0}}}, meta=0x2b0eda24a130, old_val=1729060705727078000, upper_trans_version=1729060705727078000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2088960, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:384, data_checksum:1604906901, max_merged_trans_version:1729060705727078000, upper_trans_version:1729060705727078000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.531059] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:2088960, macro_id:[14770](ver=0,mode=0,seq=7667497)}, new_sstable.get_key()={tablet_id:{id:431}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1728974598297198000, v:0}, end_scn:{val:1729060940167357001, v:0}}}) [2024-10-16 14:57:32.531201] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0ea17fe5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:1, sstable_array:0x2b0ea17fef90, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0ea17fe5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:1, sstable_array:0x2b0ea17fef90, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0ea17ff840, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1728974305586836000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINI 1729060705727078000 1729060705727078000 1728974598297198000 1729060940167357001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560287003454000 1717560287003454000 1 1728978212337222000 0 false }}) [2024-10-16 14:57:32.531368] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=21] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:431}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ea17fe060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f177d7140, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0f177d7160, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.531384] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6a35a0a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663348-6841-0-4611686756089896960-0](offset=1875968,size=687,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:431}, data_tablet_id:{id:431}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1728974598297198000, v:0}, clog_checkpoint_scn:{val:1729060940167357001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729060939970781000, multi_version_start:1729015264820413000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:434666986, row_count:130, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:16384, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:16384}}, table_store_addr:{ptr:0x2b0e6a35a68c, addr:[7663656-3934-0-2305843505283629056-0](offset=1212416,size=462,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663656-3934-0-2305843285166555598-0](offset=1212878,size=257,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a35c378, addr:[7663348-6841-0-4611686241767563743-0](offset=1876447,size=208,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0ea17fe060, is_inited:true, wash_score:1729061852530592000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:431}, data_tablet_id:{id:431}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1728974598297198000, v:0}, clog_checkpoint_scn:{val:1729060940167357001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729060939970781000, multi_version_start:1729015264820413000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:434666986, row_count:130, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:6291456, data_size:0, shared_meta_size:1929, meta_size:0, occupy_bytes:16384}}, table_store_addr:{ptr:0x2b0ea17fe450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0f177d6980, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0f177d70b0, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.531467] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=57] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea17fe5d0, key:{tablet_id:{id:431}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ea17fe5d0, addr:[7663656-3934-0-2305844193552134144-0](offset=1208320,size=1103,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:778240, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:130, occupy_size:3619, data_checksum:434666986, max_merged_trans_version:1728974305586836000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:93, row_count_:130, occupy_size_:3619, original_size_:7280, data_checksum_:434666986, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560289326944, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728974305586836000, max_merged_trans_version_:1728974305586836000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea17fe8e8, column_checksum_count:7, data_root_info:{addr:[0-0-0-3458764857417924608-0](offset=0,size=320,type=3,seq=0), orig_block_buf:0x2b0ea17fe980, block_data:{buf:0x2b0ea17fe920, size:96, extra_buf:0x2b0ea17feac0, extra_size:807, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764931506110464-0](offset=0,size=389,type=3,seq=0), orig_block_buf:0x2b0ea17fede7, block_data:{buf:0x2b0ea17fede7, size:389, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea17fef6c, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:778240, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1015, old_pos=0, pos=1015) [2024-10-16 14:57:32.531507] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea17fefa0, key:{tablet_id:{id:431}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1728974598297198000, v:0}, end_scn:{val:1729060940167357001, v:0}}}, ref_cnt:0, upper_trans_version:1729060705727078000, timestamp:0}, this:0x2b0ea17fefa0, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2088960, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:384, data_checksum:1604906901, max_merged_trans_version:1729060705727078000, upper_trans_version:1729060705727078000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:1, occupy_size_:384, original_size_:87, data_checksum_:1604906901, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060705727078000, max_merged_trans_version_:1729060705727078000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060940167357001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea17ff2b8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0ea17ff2f8, block_data:{buf:0x2b0ea17ff2b8, size:64, extra_buf:0x2b0ea17ff3ea, extra_size:839, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0ea17ff731, block_data:{buf:0x2b0ea17ff731, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea17ff827, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2088960, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=751, old_pos=0, pos=751) [2024-10-16 14:57:32.531535] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f177d6050, key:{tablet_id:{id:431}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0f177d6050, addr:[7663656-3934-0-2305843896125650838-0](offset=1210262,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1351680, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:626, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:626, original_size_:329, data_checksum_:2901109421, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f177d6368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f177d63a8, block_data:{buf:0x2b0f177d6368, size:64, extra_buf:0x2b0f177d6495, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0f177d6864, block_data:{buf:0x2b0f177d6864, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177d6959, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1351680, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=738, old_pos=0, pos=738) [2024-10-16 14:57:32.531578] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.531700] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e2c350, key:{tablet_id:{id:431}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f36e2c350, addr:[7667511-10399-0-2305844193550995456-0](offset=69632,size=1103,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:778240, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:130, occupy_size:3619, data_checksum:434666986, max_merged_trans_version:1728974305586836000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:93, row_count_:130, occupy_size_:3619, original_size_:7280, data_checksum_:434666986, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560289326944, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728974305586836000, max_merged_trans_version_:1728974305586836000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e2c668, column_checksum_count:7, data_root_info:{addr:[0-0-0-3458764857417924608-0](offset=0,size=320,type=3,seq=0), orig_block_buf:0x2b0f36e2c700, block_data:{buf:0x2b0f36e2c6a0, size:96, extra_buf:0x2b0f36e2c840, extra_size:807, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764931506110464-0](offset=0,size=389,type=3,seq=0), orig_block_buf:0x2b0f36e2cb67, block_data:{buf:0x2b0f36e2cb67, size:389, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e2ccec, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:778240, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.531733] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=32] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e2cd20, key:{tablet_id:{id:431}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1728974598297198000, v:0}, end_scn:{val:1729060940167357001, v:0}}}, ref_cnt:0, upper_trans_version:1729060705727078000, timestamp:0}, this:0x2b0f36e2cd20, addr:[7667511-10399-0-2305843910083155023-0](offset=70735,size=839,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2088960, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:384, data_checksum:1604906901, max_merged_trans_version:1729060705727078000, upper_trans_version:1729060705727078000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:84, row_count_:1, occupy_size_:384, original_size_:87, data_checksum_:1604906901, index_type_:0, rowkey_column_count_:3, column_cnt_:7, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060705727078000, max_merged_trans_version_:1729060705727078000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060940167357001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e2d038, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764773666062336-0](offset=0,size=242,type=3,seq=0), orig_block_buf:0x2b0f36e2d078, block_data:{buf:0x2b0f36e2d038, size:64, extra_buf:0x2b0f36e2d16a, extra_size:839, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f36e2d4b1, block_data:{buf:0x2b0f36e2d4b1, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e2d5a7, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2088960, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.531759] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e2d5d0, key:{tablet_id:{id:431}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0f36e2d5d0, addr:[7667511-10399-0-2305843896124512150-0](offset=71574,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1351680, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:626, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:626, original_size_:329, data_checksum_:2901109421, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f36e2d8e8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f36e2d928, block_data:{buf:0x2b0f36e2d8e8, size:64, extra_buf:0x2b0f36e2da15, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0f36e2dde4, block_data:{buf:0x2b0f36e2dde4, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e2ded9, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1351680, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.531798] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.532009] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.532074] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061852531625000, start_id=1729061852531625000, end_id=1729061852531625001) [2024-10-16 14:57:32.532092] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=15] ts response task statistics(alloc_count=261364070, free_count=130682034) [2024-10-16 14:57:32.532102] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=4] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:32.532408] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=10] ====== check clog disk timer task ====== [2024-10-16 14:57:32.532419] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:32.532433] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444476256, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:32.532725] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e60a4c0a0, start_pos=8920, remain=56528) [2024-10-16 14:57:32.532805] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.532894] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:431}, table_store_addr_.addr_=[7667511-10399-0-2305843505282490368-0](offset=73728,size=462,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843285165416910-0](offset=74190,size=257,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686756089053184-0](offset=1032192,size=687,type=4,seq=0), this=0x2b0e60a4c0a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f56f9a500, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0f56f9a520, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.535765] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601356, offset=9615684}, end_cursor:ObLogCursor{file_id=192, log_id=47601357, offset=9615800}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b84050, len:1724, log_data_len:116, seq:47601356, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.535796] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=16] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686756089053184-0](offset=1032192,size=687,type=4,seq=0), new_tablet_hdl={obj:0x2b0e60a4c0a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e60a4c0a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686756089053184-0](offset=1032192,size=687,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:431}, data_tablet_id:{id:431}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1728974598297198000, v:0}, clog_checkpoint_scn:{val:1729060940167357001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729060939970781000, multi_version_start:1729015264820413000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:434666986, row_count:130, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:16384, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:16384}}, table_store_addr:{ptr:0x2b0e60a4c68c, addr:[7667511-10399-0-2305843505282490368-0](offset=73728,size=462,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843285165416910-0](offset=74190,size=257,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e60a4e378, addr:[7663823-15172-0-4611686241766719967-0](offset=1032671,size=208,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.535853] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=44] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:431}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ea17fe060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f177d7140, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0f177d7160, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.535873] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=17] reset tablet(this=0x2b0ea17fe060, ls_id={id:1}, tablet_id={id:431}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.536068] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:6, col_desc_array:[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:6, 2197175160, 2197175160, 0, 0, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:3, index_type:0, row_store_type:1, schema_version:1729008951550040, column_cnt:4, store_column_cnt:4, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:4, column_array:[cnt:4, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"binary", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TINYINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:4, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:6, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.536196] INFO [STORAGE.BLKMGR] alloc_block (ob_block_manager.cpp:334) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=44] successfully alloc block(macro_id=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.536207] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.536299] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.536368] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] successfully write small sstable(ret=0, block_info={nested_size:20480, nested_offset:4096, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=24576, old_block=[]) [2024-10-16 14:57:32.536389] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f334e04e0, hex: 00D0F9A906438009, int: 684620839260901376}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f334e0440, hex: 44630F6700000000, int: 1729061700}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f334e04f0, hex: D8AD4D6E7A2301E8, int: -1729061773189993000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f334e04b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:6, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:18659, meta_block_size:233, last_macro_size:18892, macro_metas:0x2b0f334e0100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.536432] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0f36f31060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.536441] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), buf:0x2b0f36f30560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0f36f31060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:3, data_column_cnt:6, data_column_checksums:[0, 0, 0, 0, 0, 0], row_count:917, max_merged_trans_version:1729061774299396000, contain_uncommitted_row:false, occupy_size:18362, original_size:78874, data_checksum:3676636677, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.536464] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0ef9082060, key:{tablet_id:{id:209257}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, ref_cnt:0, upper_trans_version:1729061774299396000, timestamp:0}, this:0x2b0ef9082060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:917, occupy_size:18362, data_checksum:3676636677, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:917, occupy_size_:18362, original_size_:78874, data_checksum_:3676636677, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061774299396000, max_merged_trans_version_:1729061774299396000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778310817000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ef9082390, block_data:{buf:0x2b0ef90825c0, size:64, extra_buf:0x2b0ef9082600, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0ef9082490, block_data:{buf:0x2b0ef9082490, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef90825a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.536505] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=40] finish set upper trans version(ret=0, key_={tablet_id:{id:209257}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, meta=0x2b0ef9082130, old_val=1729061774299396000, upper_trans_version=1729061774299396000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:917, occupy_size:18362, data_checksum:3676636677, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.536515] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:20480, nested_offset:4096, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:209257}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}) [2024-10-16 14:57:32.536648] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0ea17fa5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0ea17fb070, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0ea17fa5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0ea17fb070, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f36e769a0, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729045392735383000 0 1729045392735383000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061413096686000 1729061413096686000 1729045336369978001 1729061413753243003 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061774299396000 1729061774299396000 1729061413753243003 1729061778310817000 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778310817000 9223372036854775807 3 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836143346544000 1728836143346544000 1 1729036755796258000 0 false }}) [2024-10-16 14:57:32.536807] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=22] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1003}, tablet_id={id:209257}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ea17fa060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f36e77ab0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f36e77ad0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.536824] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6aa4c0a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964312576-0](offset=360448,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1003}, tablet_id:{id:209257}, data_tablet_id:{id:209257}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836143268709001, v:0}, start_scn:{val:1729045336369978001, v:0}, clog_checkpoint_scn:{val:1729061778310817000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505000, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729045392735383000, cur_report_version:1729045392735383000, data_checksum:1963743088, row_count:89064, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755796258000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729045392735383000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836143346544000, v:0}, create_commit_version:1728836143346544000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836143165304, space_usage:{shared_data_size:1519616, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:1519616}}, table_store_addr:{ptr:0x2b0e6aa4c6b0, addr:[7663857-6566-0-2305843636279427072-0](offset=507904,size=584,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843270133465672-0](offset=508488,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6aa4ee7e, addr:[7663823-15172-0-4611686276125786598-0](offset=360934,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0e44a3a080, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}, this={this:0x2b0ea17fa060, is_inited:true, wash_score:1729061852535973000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1003}, tablet_id:{id:209257}, data_tablet_id:{id:209257}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836143268709001, v:0}, start_scn:{val:1729045336369978001, v:0}, clog_checkpoint_scn:{val:1729061778310817000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505000, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729045392735383000, cur_report_version:1729045392735383000, data_checksum:1963743088, row_count:89064, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755796258000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729045392735383000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836143346544000, v:0}, create_commit_version:1728836143346544000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836143165304, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2903, meta_size:0, occupy_bytes:1519616}}, table_store_addr:{ptr:0x2b0ea17fa450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0f36e772e0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0f36e77a20, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0e44a3a080, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.536904] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=58] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea17fa5d0, key:{tablet_id:{id:209257}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729045392735383000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ea17fa5d0, addr:[7663857-6566-0-2305844288040710144-0](offset=503808,size=1191,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:749568, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:89064, occupy_size:743575, data_checksum:1963743088, max_merged_trans_version:1729045392735383000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:89064, occupy_size_:743575, original_size_:7125120, data_checksum_:1963743088, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:62, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008951550040, create_snapshot_version_:1729029617350074000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729045392735383000, max_merged_trans_version_:1729045392735383000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea17fa8e8, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0ea17fa980, block_data:{buf:0x2b0ea17fa918, size:104, extra_buf:0x2b0ea17faae6, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764980898234368-0](offset=0,size=435,type=3,seq=0), orig_block_buf:0x2b0ea17fae9d, block_data:{buf:0x2b0ea17fae9d, size:435, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea17fb050, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:749568}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1103, old_pos=0, pos=1103) [2024-10-16 14:57:32.536942] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea17fb090, key:{tablet_id:{id:209257}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729045336369978001, v:0}, end_scn:{val:1729061413753243003, v:0}}}, ref_cnt:0, upper_trans_version:1729061413096686000, timestamp:0}, this:0x2b0ea17fb090, addr:[7663857-6566-0-2305843959475713191-0](offset=504999,size=885,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:745472, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:38265, occupy_size:739056, data_checksum:0, max_merged_trans_version:1729061413096686000, upper_trans_version:1729061413096686000, filled_tx_scn:{val:1729061413753243003, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:98, row_count_:38265, occupy_size_:739056, original_size_:3291002, data_checksum_:1445620112, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:53, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061413096686000, max_merged_trans_version_:1729061413096686000, recycle_version:1729045392735383000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413753243003, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea17fb3a8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ea17fb3e8, block_data:{buf:0x2b0ea17fb3a8, size:64, extra_buf:0x2b0ea17fb4e0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0ea17fb8b7, block_data:{buf:0x2b0ea17fb8b7, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea17fb9c7, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:745472}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=797, old_pos=0, pos=797) [2024-10-16 14:57:32.536971] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f36e76050, key:{tablet_id:{id:209257}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, ref_cnt:0, upper_trans_version:1729061774299396000, timestamp:0}, this:0x2b0f36e76050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:917, occupy_size:18362, data_checksum:3676636677, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:917, occupy_size_:18362, original_size_:78874, data_checksum_:3676636677, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061774299396000, max_merged_trans_version_:1729061774299396000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778310817000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e76368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f36e763a8, block_data:{buf:0x2b0f36e76368, size:64, extra_buf:0x2b0f36e764a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f36e76877, block_data:{buf:0x2b0f36e76877, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e7697d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=777, old_pos=0, pos=777) [2024-10-16 14:57:32.536998] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f36e769b0, key:{tablet_id:{id:209257}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755796258000, v:0}}}, ref_cnt:0, upper_trans_version:1728836143346544000, timestamp:0}, this:0x2b0f36e769b0, addr:[7663857-6566-0-2305843897198689149-0](offset=506749,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:147456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836143346544000, upper_trans_version:1728836143346544000, filled_tx_scn:{val:1729036755796258000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:3884449645, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836143346544000, max_merged_trans_version_:1728836143346544000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755796258000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f36e76cc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f36e76d08, block_data:{buf:0x2b0f36e76cc8, size:64, extra_buf:0x2b0f36e76df5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f36e771c4, block_data:{buf:0x2b0f36e771c4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e772ba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:147456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=739, old_pos=0, pos=739) [2024-10-16 14:57:32.537044] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.537233] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f56f0a660, key:{tablet_id:{id:209257}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729045392735383000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f56f0a660, addr:[7667511-10399-0-2305844288040284160-0](offset=77824,size=1191,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:749568, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:89064, occupy_size:743575, data_checksum:1963743088, max_merged_trans_version:1729045392735383000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:89064, occupy_size_:743575, original_size_:7125120, data_checksum_:1963743088, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:62, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008951550040, create_snapshot_version_:1729029617350074000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729045392735383000, max_merged_trans_version_:1729045392735383000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f0a978, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f56f0aa10, block_data:{buf:0x2b0f56f0a9a8, size:104, extra_buf:0x2b0f56f0ab76, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764980898234368-0](offset=0,size=435,type=3,seq=0), orig_block_buf:0x2b0f56f0af2d, block_data:{buf:0x2b0f56f0af2d, size:435, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f0b0e0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:749568}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.537270] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f56f0b120, key:{tablet_id:{id:209257}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729045336369978001, v:0}, end_scn:{val:1729061413753243003, v:0}}}, ref_cnt:0, upper_trans_version:1729061413096686000, timestamp:0}, this:0x2b0f56f0b120, addr:[7667511-10399-0-2305843959475287207-0](offset=79015,size=885,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:745472, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:38265, occupy_size:739056, data_checksum:0, max_merged_trans_version:1729061413096686000, upper_trans_version:1729061413096686000, filled_tx_scn:{val:1729061413753243003, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:98, row_count_:38265, occupy_size_:739056, original_size_:3291002, data_checksum_:1445620112, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:53, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061413096686000, max_merged_trans_version_:1729061413096686000, recycle_version:1729045392735383000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413753243003, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f0b438, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f56f0b478, block_data:{buf:0x2b0f56f0b438, size:64, extra_buf:0x2b0f56f0b570, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0f56f0b947, block_data:{buf:0x2b0f56f0b947, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f0ba57, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:745472}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.537295] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ebd3fe050, key:{tablet_id:{id:209257}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, ref_cnt:0, upper_trans_version:1729061774299396000, timestamp:0}, this:0x2b0ebd3fe050, addr:[7667511-10399-0-2305843938000451612-0](offset=79900,size=865,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:917, occupy_size:18362, data_checksum:3676636677, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:917, occupy_size_:18362, original_size_:78874, data_checksum_:3676636677, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061774299396000, max_merged_trans_version_:1729061774299396000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778310817000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ebd3fe368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ebd3fe3a8, block_data:{buf:0x2b0ebd3fe368, size:64, extra_buf:0x2b0ebd3fe4a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0ebd3fe877, block_data:{buf:0x2b0ebd3fe877, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ebd3fe97d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.537320] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ebd3fe9b0, key:{tablet_id:{id:209257}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755796258000, v:0}}}, ref_cnt:0, upper_trans_version:1728836143346544000, timestamp:0}, this:0x2b0ebd3fe9b0, addr:[7667511-10399-0-2305843897198263165-0](offset=80765,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:147456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836143346544000, upper_trans_version:1728836143346544000, filled_tx_scn:{val:1729036755796258000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:3884449645, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836143346544000, max_merged_trans_version_:1728836143346544000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755796258000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ebd3fecc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ebd3fed08, block_data:{buf:0x2b0ebd3fecc8, size:64, extra_buf:0x2b0ebd3fedf5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0ebd3ff1c4, block_data:{buf:0x2b0ebd3ff1c4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ebd3ff2ba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:147456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.537365] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=28] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.537543] INFO [STORAGE] deserialize (ob_tablet_table_store.cpp:153) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to deserialize table store(ret=0, ls_id={id:1003}, tablet_id={id:209257}, major_tables_={cnt:1, sstable_array:0x2b0f1adbe1d0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f1adbe2b0, serialize_table_type:false, is_inited:true}, meta_major_tables_={cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, mds_sstables_={cnt:1, sstable_array:0x2b0f1adbe470, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecd518, major_tables:{cnt:1, sstable_array:0x2b0f1adbe1d0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f1adbe2b0, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f1adbe470, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729045392735383000 0 1729045392735383000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061413096686000 1729061413096686000 1729045336369978001 1729061413753243003 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061774299396000 1729061774299396000 1729061413753243003 1729061778310817000 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778310817000 9223372036854775807 4 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836143346544000 1728836143346544000 1 1729036755796258000 0 false }}) [2024-10-16 14:57:32.537743] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6a64c0a0, start_pos=11742, remain=53706) [2024-10-16 14:57:32.537807] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.537892] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1003}, tablet_id={id:209257}, table_store_addr_.addr_=[7667511-10399-0-2305843635205259264-0](offset=81920,size=583,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843270133039687-0](offset=82503,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964988416-0](offset=1036288,size=726,type=4,seq=0), this=0x2b0e6a64c0a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ebd3ff7e0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ebd3ff800, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.538099] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601357, offset=9615800}, end_cursor:ObLogCursor{file_id=192, log_id=47601358, offset=9615916}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ad0050, len:1608, log_data_len:116, seq:47601357, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.538118] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686797964988416-0](offset=1036288,size=726,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6a64c0a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6a64c0a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964988416-0](offset=1036288,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1003}, tablet_id:{id:209257}, data_tablet_id:{id:209257}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836143268709001, v:0}, start_scn:{val:1729045336369978001, v:0}, clog_checkpoint_scn:{val:1729061778310817000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505000, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729045392735383000, cur_report_version:1729045392735383000, data_checksum:1963743088, row_count:89064, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755796258000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729045392735383000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836143346544000, v:0}, create_commit_version:1728836143346544000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836143165304, space_usage:{shared_data_size:1519616, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:1519616}}, table_store_addr:{ptr:0x2b0e6a64c6b0, addr:[7667511-10399-0-2305843635205259264-0](offset=81920,size=583,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843270133039687-0](offset=82503,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a64ee7e, addr:[7663823-15172-0-4611686276126462438-0](offset=1036774,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0e44a3a080, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.538168] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=41] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1003}, tablet_meta_.tablet_id_={id:209257}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ea17fa060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f36e77ab0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f36e77ad0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.538183] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] reset tablet(this=0x2b0ea17fa060, ls_id={id:1003}, tablet_id={id:209257}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.538458] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1002}, tablet_id:{id:49402}}) [2024-10-16 14:57:32.538790] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:32.539221] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:394}, table_store_addr_.addr_=[7663656-3934-0-2305843626616446976-0](offset=1204224,size=575,type=2,seq=0), storage_schema_addr_.addr_=[7663656-3934-0-2305843285166547519-0](offset=1204799,size=257,type=2,seq=0), tablet_addr_=[7663348-6841-0-4611686789375889408-0](offset=1871872,size=718,type=4,seq=0), this=0x2b0e695b40a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e695b6ca6, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e695b6cbe, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.539242] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=17] reset tablet(this=0x2b0e695b40a0, ls_id={id:1}, tablet_id={id:394}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.539255] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to gc tablet(ret=0, tablet=0x2b0e695b40a0) [2024-10-16 14:57:32.539263] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:431}, table_store_addr_.addr_=[7663656-3934-0-2305843505283629056-0](offset=1212416,size=462,type=2,seq=0), storage_schema_addr_.addr_=[7663656-3934-0-2305843285166555598-0](offset=1212878,size=257,type=2,seq=0), tablet_addr_=[7663348-6841-0-4611686756089896960-0](offset=1875968,size=687,type=4,seq=0), this=0x2b0e6a35a0a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6a35c3f8, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e6a35c410, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.539274] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=9] reset tablet(this=0x2b0e6a35a0a0, ls_id={id:1}, tablet_id={id:431}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.539284] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e6a35a0a0) [2024-10-16 14:57:32.539292] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1003}, tablet_meta_.tablet_id_={id:209257}, table_store_addr_.addr_=[7663857-6566-0-2305843636279427072-0](offset=507904,size=584,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843270133465672-0](offset=508488,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964312576-0](offset=360448,size=726,type=4,seq=0), this=0x2b0e6aa4c0a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6aa4eefe, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e6aa4ef16, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.539303] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=9] reset tablet(this=0x2b0e6aa4c0a0, ls_id={id:1003}, tablet_id={id:209257}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.539314] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e6aa4c0a0) [2024-10-16 14:57:32.539317] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:827) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] gc tablets in queue(gc_tablets_cnt=3, err_tablets_cnt=0, tablet_gc_queue_.count()=0) [2024-10-16 14:57:32.539674] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C47-0-0] [lt=39] get number(ret=0, service_type_=0, range=1, base_id=1729061852539191000, start_id=1729061852539191000, end_id=1729061852539191001) [2024-10-16 14:57:32.539836] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:27, rowkey_column_count:6, schema_rowkey_col_cnt:4, full_stored_col_cnt:27, col_desc_array:[cnt:27, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"DOUBLE", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"DOUBLE", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"DOUBLE", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=33 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=40 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:27, 2197175160, 2197175160, 2197175160, 2197175160, 0, 0, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2388842353], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:0, index_type:0, row_store_type:1, schema_version:1717560289055056, column_cnt:25, store_column_cnt:25, tablet_size:134217728, pctfree:10, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:1, encryption:"", encrypt_key:"", rowkey_cnt:4, rowkey_array:[cnt:4, {column_idx:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:21, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:25, column_array:[cnt:25, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DOUBLE", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DOUBLE", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DOUBLE", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TINYINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:1, row_store_type:1, block_size:16384, schema_column_cnt:25, rowkey_column_cnt:6, schema_rowkey_column_cnt:4, column_cnt:27, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:27, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"DOUBLE", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"DOUBLE", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"DOUBLE", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=33 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=40 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.541166] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=147] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.541273] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] successfully write small sstable(ret=0, block_info={nested_size:20480, nested_offset:24576, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=45056, old_block=[]) [2024-10-16 14:57:32.541295] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:6, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f0bdf4408, hex: 0000000000000000, int: 0}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f0bdf4440, hex: 7FA1070000000000, int: 500095}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f0bdf4478, hex: 7FA1070000000000, int: 500095}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f0bdf4550, hex: 639C2E781B220600, int: 1726351236045923}, idx=4:{len: 8, flag: 0, null: 0, ptr: 0x2b0f0bdf4560, hex: B777E5E2D32C01E8, int: -1729051493377149001}, idx=5:{len: 8, flag: 0, null: 0, ptr: 0x2b0f0bdf4520, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:27, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:18873, meta_block_size:298, last_macro_size:19171, macro_metas:0x2b0f0bdf4100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.541338] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=16] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764833795604480-0](offset=0,size=298,type=3,seq=0), buf:0x2b0f36e05070, height:1, is_meta_root:true}) [2024-10-16 14:57:32.541347] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764788698447872-0](offset=0,size=256,type=3,seq=0), buf:0x2b0f36e04560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764833795604480-0](offset=0,size=298,type=3,seq=0), buf:0x2b0f36e05070, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:2, data_column_cnt:27, data_column_checksums:[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], row_count:199, max_merged_trans_version:1729051493377149001, contain_uncommitted_row:false, occupy_size:18429, original_size:17884, data_checksum:1358583443, use_old_macro_block_count:0, compressor_type:1, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.541394] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=18] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0eda24a060, key:{tablet_id:{id:332}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}, ref_cnt:0, upper_trans_version:1729051493377149001, timestamp:0}, this:0x2b0eda24a060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:24576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:199, occupy_size:18429, data_checksum:1358583443, max_merged_trans_version:1729051493377149001, upper_trans_version:1729051493377149001, filled_tx_scn:{val:1729051758122012000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:199, occupy_size_:18429, original_size_:17884, data_checksum_:1358583443, index_type_:0, rowkey_column_count_:6, column_cnt_:27, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729051493377149001, max_merged_trans_version_:1729051493377149001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729051758122012000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764788698447872-0](offset=0,size=256,type=3,seq=0), orig_block_buf:0x2b0eda24a390, block_data:{buf:0x2b0eda24a5f0, size:64, extra_buf:0x2b0eda24a630, extra_size:1271, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764833795604480-0](offset=0,size=298,type=3,seq=0), orig_block_buf:0x2b0eda24a4a0, block_data:{buf:0x2b0eda24a4a0, size:298, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0eda24a5d0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:24576, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.541431] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=54] finish set upper trans version(ret=0, key_={tablet_id:{id:332}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}, meta=0x2b0eda24a130, old_val=1729051493377149001, upper_trans_version=1729051493377149001, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:24576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:199, occupy_size:18429, data_checksum:1358583443, max_merged_trans_version:1729051493377149001, upper_trans_version:1729051493377149001, filled_tx_scn:{val:1729051758122012000, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.541441] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:20480, nested_offset:24576, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:332}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}) [2024-10-16 14:57:32.541572] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f009fc5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:1, sstable_array:0x2b0f009fd370, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f009fc5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:1, sstable_array:0x2b0f009fd370, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f009fde20, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729000842428471000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINI 1729051493377149001 1729051493377149001 1729001176236986000 1729051758122012000 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560287003454000 1717560287003454000 1 1728978212337222000 0 false }}) [2024-10-16 14:57:32.541731] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=18] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:332}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f009fc060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef9787850, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0ef9787870, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.541748] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6b1d80a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663348-6841-0-4611686775417253888-0](offset=1880064,size=705,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:332}, data_tablet_id:{id:332}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729001176236986000, v:0}, clog_checkpoint_scn:{val:1729051758122012000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729051758122012000, multi_version_start:1729015254497626003, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:705749951, row_count:5577, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:102400, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:102400}}, table_store_addr:{ptr:0x2b0e6b1d86f8, addr:[7663656-3934-0-2305843510652346368-0](offset=1220608,size=467,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663656-3934-0-2305843735064388051-0](offset=1221075,size=676,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6b1da9b8, addr:[7663348-6841-0-4611686241767567857-0](offset=1880561,size=208,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f009fc060, is_inited:true, wash_score:1729061852541117000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:332}, data_tablet_id:{id:332}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729001176236986000, v:0}, clog_checkpoint_scn:{val:1729051758122012000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729051758122012000, multi_version_start:1729015254497626003, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:705749951, row_count:5577, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:6291456, data_size:0, shared_meta_size:2454, meta_size:0, occupy_bytes:102400}}, table_store_addr:{ptr:0x2b0f009fc450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0ef9786980, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0ef97877c0, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.541821] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=53] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f009fc5d0, key:{tablet_id:{id:332}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f009fc5d0, addr:[7663656-3934-0-2305844757266599936-0](offset=1216512,size=1628,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:77824, nested_offset:1773568, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5577, occupy_size:73693, data_checksum:705749951, max_merged_trans_version:1729000842428471000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:5577, occupy_size_:73693, original_size_:1204632, data_checksum_:705749951, index_type_:0, rowkey_column_count_:6, column_cnt_:27, data_macro_block_count_:1, data_micro_block_count_:7, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560288989864, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729000842428471000, max_merged_trans_version_:1729000842428471000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f009fc8e8, column_checksum_count:27, data_root_info:{addr:[0-0-0-3458764979824492544-0](offset=0,size=434,type=3,seq=0), orig_block_buf:0x2b0f009fca38, block_data:{buf:0x2b0f009fc9c0, size:120, extra_buf:0x2b0f009fcbea, extra_size:1239, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765214973952000-0](offset=0,size=653,type=3,seq=0), orig_block_buf:0x2b0f009fd0c1, block_data:{buf:0x2b0f009fd0c1, size:653, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f009fd34e, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1773568, nested_size:77824}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1540, old_pos=0, pos=1540) [2024-10-16 14:57:32.541859] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f009fd380, key:{tablet_id:{id:332}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}, ref_cnt:0, upper_trans_version:1729051493377149001, timestamp:0}, this:0x2b0f009fd380, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:24576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:199, occupy_size:18429, data_checksum:1358583443, max_merged_trans_version:1729051493377149001, upper_trans_version:1729051493377149001, filled_tx_scn:{val:1729051758122012000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:199, occupy_size_:18429, original_size_:17884, data_checksum_:1358583443, index_type_:0, rowkey_column_count_:6, column_cnt_:27, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729051493377149001, max_merged_trans_version_:1729051493377149001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729051758122012000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f009fd698, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764788698447872-0](offset=0,size=256,type=3,seq=0), orig_block_buf:0x2b0f009fd6d8, block_data:{buf:0x2b0f009fd698, size:64, extra_buf:0x2b0f009fd7d8, extra_size:1271, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764833795604480-0](offset=0,size=298,type=3,seq=0), orig_block_buf:0x2b0f009fdccf, block_data:{buf:0x2b0f009fdccf, size:298, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f009fddf9, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:24576, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=821, old_pos=0, pos=821) [2024-10-16 14:57:32.541887] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef9786050, key:{tablet_id:{id:332}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0ef9786050, addr:[7663656-3934-0-2305843896125659625-0](offset=1219049,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1945600, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:626, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:626, original_size_:329, data_checksum_:2210493174, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ef9786368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ef97863a8, block_data:{buf:0x2b0ef9786368, size:64, extra_buf:0x2b0ef9786495, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0ef9786864, block_data:{buf:0x2b0ef9786864, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9786959, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1945600, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=738, old_pos=0, pos=738) [2024-10-16 14:57:32.541931] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.542047] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ec821e690, key:{tablet_id:{id:332}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ec821e690, addr:[7667511-10399-0-2305844757265469440-0](offset=86016,size=1628,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:77824, nested_offset:1773568, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5577, occupy_size:73693, data_checksum:705749951, max_merged_trans_version:1729000842428471000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:5577, occupy_size_:73693, original_size_:1204632, data_checksum_:705749951, index_type_:0, rowkey_column_count_:6, column_cnt_:27, data_macro_block_count_:1, data_micro_block_count_:7, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560288989864, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729000842428471000, max_merged_trans_version_:1729000842428471000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ec821e9a8, column_checksum_count:27, data_root_info:{addr:[0-0-0-3458764979824492544-0](offset=0,size=434,type=3,seq=0), orig_block_buf:0x2b0ec821eaf8, block_data:{buf:0x2b0ec821ea80, size:120, extra_buf:0x2b0ec821ecaa, extra_size:1239, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765214973952000-0](offset=0,size=653,type=3,seq=0), orig_block_buf:0x2b0ec821f181, block_data:{buf:0x2b0ec821f181, size:653, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec821f40e, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1773568, nested_size:77824}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.542079] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=32] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ec821f440, key:{tablet_id:{id:332}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729051758122012000, v:0}}}, ref_cnt:0, upper_trans_version:1729051493377149001, timestamp:0}, this:0x2b0ec821f440, addr:[7667511-10399-0-2305843985245099612-0](offset=87644,size=909,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:24576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:199, occupy_size:18429, data_checksum:1358583443, max_merged_trans_version:1729051493377149001, upper_trans_version:1729051493377149001, filled_tx_scn:{val:1729051758122012000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:199, occupy_size_:18429, original_size_:17884, data_checksum_:1358583443, index_type_:0, rowkey_column_count_:6, column_cnt_:27, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729051493377149001, max_merged_trans_version_:1729051493377149001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729051758122012000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ec821f758, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764788698447872-0](offset=0,size=256,type=3,seq=0), orig_block_buf:0x2b0ec821f798, block_data:{buf:0x2b0ec821f758, size:64, extra_buf:0x2b0ec821f898, extra_size:1271, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764833795604480-0](offset=0,size=298,type=3,seq=0), orig_block_buf:0x2b0ec821fd8f, block_data:{buf:0x2b0ec821fd8f, size:298, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec821feb9, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:24576, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.542104] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f12cb8050, key:{tablet_id:{id:332}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0f12cb8050, addr:[7667511-10399-0-2305843896124529129-0](offset=88553,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1945600, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:626, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:626, original_size_:329, data_checksum_:2210493174, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f12cb8368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f12cb83a8, block_data:{buf:0x2b0f12cb8368, size:64, extra_buf:0x2b0f12cb8495, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0f12cb8864, block_data:{buf:0x2b0f12cb8864, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f12cb8959, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1945600, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.542150] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=29] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.542465] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6a5360a0, start_pos=10520, remain=54928) [2024-10-16 14:57:32.542534] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.542638] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:332}, table_store_addr_.addr_=[7667511-10399-0-2305843510651215872-0](offset=90112,size=467,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843735063257555-0](offset=90579,size=676,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686775416414208-0](offset=1040384,size=705,type=4,seq=0), this=0x2b0e6a5360a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f12cb9320, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0f12cb9340, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.542878] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601358, offset=9615916}, end_cursor:ObLogCursor{file_id=192, log_id=47601359, offset=9616032}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c40050, len:1492, log_data_len:116, seq:47601358, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.542898] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=17] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686775416414208-0](offset=1040384,size=705,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6a5360a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6a5360a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686775416414208-0](offset=1040384,size=705,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:332}, data_tablet_id:{id:332}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729001176236986000, v:0}, clog_checkpoint_scn:{val:1729051758122012000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729051758122012000, multi_version_start:1729015254497626003, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:705749951, row_count:5577, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:102400, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:102400}}, table_store_addr:{ptr:0x2b0e6a5366f8, addr:[7667511-10399-0-2305843510651215872-0](offset=90112,size=467,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843735063257555-0](offset=90579,size=676,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a5389b8, addr:[7663823-15172-0-4611686241766728177-0](offset=1040881,size=208,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.542943] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=37] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:332}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f009fc060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef9787850, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0ef9787870, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.542957] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] reset tablet(this=0x2b0f009fc060, ls_id={id:1}, tablet_id={id:332}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.543628] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:4, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:4, col_desc_array:[cnt:4, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:4, 2197175160, 2197175160, 0, 0], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:5, index_type:7, row_store_type:1, schema_version:1717560328508224, column_cnt:2, store_column_cnt:2, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:2, column_array:[cnt:2, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:2, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:4, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:4, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.543773] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=37] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.543853] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully write small sstable(ret=0, block_info={nested_size:20480, nested_offset:45056, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=65536, old_block=[]) [2024-10-16 14:57:32.543873] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12dda408, hex: 8A9C2E0000000000, int: 3054730}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12dda440, hex: 54722F0000000000, int: 3109460}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12dda4e0, hex: 900A58247D2301E8, int: -1729061761545926000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12dda4b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:4, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:16581, meta_block_size:236, last_macro_size:16817, macro_metas:0x2b0f12dda100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.543909] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), buf:0x2b0f33469060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.543918] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), buf:0x2b0f33468560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), buf:0x2b0f33469060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:2, data_column_cnt:4, data_column_checksums:[0, 0, 0, 0], row_count:776, max_merged_trans_version:1729061761545926000, contain_uncommitted_row:false, occupy_size:16334, original_size:35704, data_checksum:986605367, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.543938] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0ef9082060, key:{tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761545926000, timestamp:0}, this:0x2b0ef9082060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:45056, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:776, occupy_size:16334, data_checksum:986605367, max_merged_trans_version:1729061761545926000, upper_trans_version:1729061761545926000, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:776, occupy_size_:16334, original_size_:35704, data_checksum_:986605367, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761545926000, max_merged_trans_version_:1729061761545926000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778311878001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), orig_block_buf:0x2b0ef9082390, block_data:{buf:0x2b0ef90825c0, size:64, extra_buf:0x2b0ef9082600, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), orig_block_buf:0x2b0ef9082490, block_data:{buf:0x2b0ef9082490, size:265, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef90825a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:45056, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.543973] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] finish set upper trans version(ret=0, key_={tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, meta=0x2b0ef9082130, old_val=1729061761545926000, upper_trans_version=1729061761545926000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:45056, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:776, occupy_size:16334, data_checksum:986605367, max_merged_trans_version:1729061761545926000, upper_trans_version:1729061761545926000, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.543983] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:20480, nested_offset:45056, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729061778311878001, v:0}}}) [2024-10-16 14:57:32.544100] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0ec83c25c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:1, sstable_array:0x2b0ec83c3330, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0ec83c25c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:1, sstable_array:0x2b0ec83c3330, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0ec83c3c80, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015201117438000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061761545926000 1729061761545926000 1729015204449389004 1729061778311878001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328534252000 1717560328534252000 1 1729036755768426000 0 false }}) [2024-10-16 14:57:32.544235] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1002}, tablet_id={id:1152921504606846986}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ec83c2060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:2, arr:0x2b0e8499f0b0, capacity:2}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8499f0f0, capacity:1}, shared_data_block_info_arr:{cnt:2, arr:0x2b0e8499f110, capacity:2}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.544250] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6a2040a0, is_inited:true, wash_score:1729061852010132000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686780784492544-0](offset=409600,size=710,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1002}, tablet_id:{id:1152921504606846986}, data_tablet_id:{id:200068}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1717560328237935004, v:0}, start_scn:{val:1729015204449389004, v:0}, clog_checkpoint_scn:{val:1729061778311878001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3291875008, row_count:181042, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755768426000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4353, compat:1, last_compaction_type:1, wait_check_flag:1, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328534252000, v:0}, create_commit_version:1717560328534252000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328508216, space_usage:{shared_data_size:24576, data_size:4194304, shared_meta_size:12288, meta_size:0, occupy_bytes:1622114}}, table_store_addr:{ptr:0x2b0e6a2046b0, addr:[7663857-6566-0-2305843509578018816-0](offset=634880,size=466,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843227183919570-0](offset=635346,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a2067e6, addr:[7663823-15172-0-4611686258945966566-0](offset=410086,size=224,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0ec83c2060, is_inited:true, wash_score:1729061852543508000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1002}, tablet_id:{id:1152921504606846986}, data_tablet_id:{id:200068}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1717560328237935004, v:0}, start_scn:{val:1729015204449389004, v:0}, clog_checkpoint_scn:{val:1729061778311878001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3291875008, row_count:181042, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755768426000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4353, compat:1, last_compaction_type:1, wait_check_flag:1, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328534252000, v:0}, create_commit_version:1717560328534252000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328508216, space_usage:{shared_data_size:4194304, data_size:4194304, shared_meta_size:2292, meta_size:0, occupy_bytes:1622114}}, table_store_addr:{ptr:0x2b0ec83c2450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0e8499e980, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0e8499f020, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.544319] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=53] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ec83c25d0, key:{tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ec83c25d0, addr:[7663857-6566-0-2305844575803637760-0](offset=622592,size=1459,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:2, nested_size:2097152, nested_offset:0, total_macro_block_count:2, total_use_old_macro_block_count:1, row_count:181042, occupy_size:1597538, data_checksum:3291875008, max_merged_trans_version:1729015201117438000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:181042, occupy_size_:1597538, original_size_:5793344, data_checksum_:3291875008, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:2, data_micro_block_count_:98, use_old_macro_block_count_:1, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328508216, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201117438000, max_merged_trans_version_:1729015201117438000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ec83c28e8, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458765001299329024-0](offset=0,size=454,type=3,seq=0), orig_block_buf:0x2b0ec83c2970, block_data:{buf:0x2b0ec83c2908, size:104, extra_buf:0x2b0ec83c2b36, extra_size:1391, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765154844409856-0](offset=0,size=597,type=3,seq=0), orig_block_buf:0x2b0ec83c30a5, block_data:{buf:0x2b0ec83c30a5, size:597, extra_buf:null, extra_size:0, type:1}}, data_block_count:2, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec83c32fa, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:0, nested_size:2097152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1371, old_pos=0, pos=1371) [2024-10-16 14:57:32.544356] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ec83c3340, key:{tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761545926000, timestamp:0}, this:0x2b0ec83c3340, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:45056, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:776, occupy_size:16334, data_checksum:986605367, max_merged_trans_version:1729061761545926000, upper_trans_version:1729061761545926000, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:776, occupy_size_:16334, original_size_:35704, data_checksum_:986605367, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761545926000, max_merged_trans_version_:1729061761545926000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778311878001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ec83c3658, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), orig_block_buf:0x2b0ec83c3698, block_data:{buf:0x2b0ec83c3658, size:64, extra_buf:0x2b0ec83c378c, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), orig_block_buf:0x2b0ec83c3b5b, block_data:{buf:0x2b0ec83c3b5b, size:265, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec83c3c64, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:45056, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=783, old_pos=0, pos=783) [2024-10-16 14:57:32.544385] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0e8499e050, key:{tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755768426000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328534252000, timestamp:0}, this:0x2b0e8499e050, addr:[7663857-6566-0-2305843903641258266-0](offset=624922,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:167936, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328534252000, upper_trans_version:1717560328534252000, filled_tx_scn:{val:1729036755768426000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:1225173178, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328534252000, max_merged_trans_version_:1717560328534252000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755768426000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0e8499e368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0e8499e3a8, block_data:{buf:0x2b0e8499e368, size:64, extra_buf:0x2b0e8499e495, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0e8499e864, block_data:{buf:0x2b0e8499e864, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e8499e960, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:167936, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=745, old_pos=0, pos=745) [2024-10-16 14:57:32.544425] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.544552] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef9608350, key:{tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ef9608350, addr:[7667511-10399-0-2305844575803109376-0](offset=94208,size=1459,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:2, nested_size:2097152, nested_offset:0, total_macro_block_count:2, total_use_old_macro_block_count:1, row_count:181042, occupy_size:1597538, data_checksum:3291875008, max_merged_trans_version:1729015201117438000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:181042, occupy_size_:1597538, original_size_:5793344, data_checksum_:3291875008, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:2, data_micro_block_count_:98, use_old_macro_block_count_:1, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328508216, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201117438000, max_merged_trans_version_:1729015201117438000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef9608668, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458765001299329024-0](offset=0,size=454,type=3,seq=0), orig_block_buf:0x2b0ef96086f0, block_data:{buf:0x2b0ef9608688, size:104, extra_buf:0x2b0ef96088b6, extra_size:1391, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765154844409856-0](offset=0,size=597,type=3,seq=0), orig_block_buf:0x2b0ef9608e25, block_data:{buf:0x2b0ef9608e25, size:597, extra_buf:null, extra_size:0, type:1}}, data_block_count:2, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef960907a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:0, nested_size:2097152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.544589] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef96090c0, key:{tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761545926000, timestamp:0}, this:0x2b0ef96090c0, addr:[7667511-10399-0-2305843944442918323-0](offset=95667,size=871,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:45056, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:776, occupy_size:16334, data_checksum:986605367, max_merged_trans_version:1729061761545926000, upper_trans_version:1729061761545926000, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:776, occupy_size_:16334, original_size_:35704, data_checksum_:986605367, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761545926000, max_merged_trans_version_:1729061761545926000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778311878001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef96093d8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764775813545984-0](offset=0,size=244,type=3,seq=0), orig_block_buf:0x2b0ef9609418, block_data:{buf:0x2b0ef96093d8, size:64, extra_buf:0x2b0ef960950c, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), orig_block_buf:0x2b0ef96098db, block_data:{buf:0x2b0ef96098db, size:265, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef96099e4, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:45056, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.544620] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=31] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0093a050, key:{tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755768426000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328534252000, timestamp:0}, this:0x2b0f0093a050, addr:[7667511-10399-0-2305843903640729882-0](offset=96538,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:167936, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328534252000, upper_trans_version:1717560328534252000, filled_tx_scn:{val:1729036755768426000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:1225173178, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328534252000, max_merged_trans_version_:1717560328534252000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755768426000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f0093a368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f0093a3a8, block_data:{buf:0x2b0f0093a368, size:64, extra_buf:0x2b0f0093a495, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0f0093a864, block_data:{buf:0x2b0f0093a864, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0093a960, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:167936, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.544659] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.545701] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061852544670000, start_id=1729061852544670000, end_id=1729061852544670001) [2024-10-16 14:57:32.546296] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6b6b80a0, start_pos=10054, remain=55394) [2024-10-16 14:57:32.546364] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.546773] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1002}, tablet_id={id:1152921504606846986}, table_store_addr_.addr_=[7667511-10399-0-2305843509577482240-0](offset=98304,size=466,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843227183382994-0](offset=98770,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686780785127424-0](offset=1044480,size=710,type=4,seq=0), this=0x2b0e6b6b80a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:2, arr:0x2b0f0093ada0, capacity:2}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f0093ade0, capacity:1}, shared_data_block_info_arr:{cnt:2, arr:0x2b0f0093ae00, capacity:2}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.547014] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601359, offset=9616032}, end_cursor:ObLogCursor{file_id=192, log_id=47601360, offset=9616148}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d54050, len:1376, log_data_len:116, seq:47601359, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.547034] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686780785127424-0](offset=1044480,size=710,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6b6b80a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6b6b80a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686780785127424-0](offset=1044480,size=710,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1002}, tablet_id:{id:1152921504606846986}, data_tablet_id:{id:200068}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1717560328237935004, v:0}, start_scn:{val:1729015204449389004, v:0}, clog_checkpoint_scn:{val:1729061778311878001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3291875008, row_count:181042, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755768426000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4353, compat:1, last_compaction_type:1, wait_check_flag:1, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328534252000, v:0}, create_commit_version:1717560328534252000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328508216, space_usage:{shared_data_size:24576, data_size:4194304, shared_meta_size:12288, meta_size:0, occupy_bytes:1622114}}, table_store_addr:{ptr:0x2b0e6b6b86b0, addr:[7667511-10399-0-2305843509577482240-0](offset=98304,size=466,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843227183382994-0](offset=98770,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6b6ba7e6, addr:[7663823-15172-0-4611686258946601446-0](offset=1044966,size=224,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.547085] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=41] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1002}, tablet_meta_.tablet_id_={id:1152921504606846986}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ec83c2060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:2, arr:0x2b0e8499f0b0, capacity:2}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8499f0f0, capacity:1}, shared_data_block_info_arr:{cnt:2, arr:0x2b0e8499f110, capacity:2}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.547100] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] reset tablet(this=0x2b0ec83c2060, ls_id={id:1002}, tablet_id={id:1152921504606846986}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.547517] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:4, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:4, col_desc_array:[cnt:4, column_id=17 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:4, 2197175160, 2197175160, 0, 0], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:5, index_type:7, row_store_type:1, schema_version:1717560328798944, column_cnt:2, store_column_cnt:2, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:17, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0}, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:2, column_array:[cnt:2, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:2, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:4, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:4, column_id=17 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.547675] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=42] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.547741] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:65536, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=69632, old_block=[]) [2024-10-16 14:57:32.547758] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 10, flag: 0, null: 0, ptr: 0x2b0f36e084e0, hex: 5355434345535346554C, cstr: SUCCESSFUL}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36e08440, hex: 8A9C2E0000000000, int: 3054730}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36e084f0, hex: 889B47247D2301E8, int: -1729061761547003000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36e084b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:4, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:863, meta_block_size:231, last_macro_size:1094, macro_metas:0x2b0f36e08100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.547796] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), buf:0x2b0f12d29060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.547804] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), buf:0x2b0f12d28560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), buf:0x2b0f12d29060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:4, data_column_checksums:[0, 0, 0, 0], row_count:21, max_merged_trans_version:1729061761547003000, contain_uncommitted_row:false, occupy_size:671, original_size:1066, data_checksum:3946811441, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.547824] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0eda24a060, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0eda24a060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:65536, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:21, occupy_size:671, data_checksum:3946811441, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:21, occupy_size_:671, original_size_:1066, data_checksum_:3946811441, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0eda24a390, block_data:{buf:0x2b0eda24a5c0, size:64, extra_buf:0x2b0eda24a600, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), orig_block_buf:0x2b0eda24a490, block_data:{buf:0x2b0eda24a490, size:261, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0eda24a5a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:65536, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.547860] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] finish set upper trans version(ret=0, key_={tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0eda24a130, old_val=1729061761547003000, upper_trans_version=1729061761547003000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:65536, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:21, occupy_size:671, data_checksum:3946811441, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.547870] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:65536, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.547997] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0ea17fa5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0ea17fb060, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0ea17fa5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0ea17fb060, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0e8481a9a0, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015201157706000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061401350833000 1729061401350833000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061761547003000 1729061761547003000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328401377000 1717560328401377000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.548128] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846984}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ea17fa060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8481ba10, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e8481ba30, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.548144] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6656c0a0, is_inited:true, wash_score:1729061852526526000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964427264-0](offset=475136,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846984}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3136814811, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332176, space_usage:{shared_data_size:724992, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:724992}}, table_store_addr:{ptr:0x2b0e6656c6b0, addr:[7663857-6566-0-2305843633058435072-0](offset=741376,size=581,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843227184026181-0](offset=741957,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6656ee83, addr:[7663823-15172-0-4611686276125901286-0](offset=475622,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0ea17fa060, is_inited:true, wash_score:1729061852547011000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846984}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3136814811, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332176, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2893, meta_size:0, occupy_bytes:724992}}, table_store_addr:{ptr:0x2b0ea17fa450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0e8481b2e0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0e8481b980, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.548212] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=52] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea17fa5d0, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ea17fa5d0, addr:[7663857-6566-0-2305844268713590784-0](offset=737280,size=1173,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90523, occupy_size:692820, data_checksum:3136814811, max_merged_trans_version:1729015201157706000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90523, occupy_size_:692820, original_size_:3801959, data_checksum_:3136814811, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:44, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328332176, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201157706000, max_merged_trans_version_:1729015201157706000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea17fa8e8, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764900367597568-0](offset=0,size=360,type=3,seq=0), orig_block_buf:0x2b0ea17fa970, block_data:{buf:0x2b0ea17fa908, size:104, extra_buf:0x2b0ea17faad8, extra_size:961, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764974455783424-0](offset=0,size=429,type=3,seq=0), orig_block_buf:0x2b0ea17fae99, block_data:{buf:0x2b0ea17fae99, size:429, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea17fb046, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1085, old_pos=0, pos=1085) [2024-10-16 14:57:32.548249] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea17fb080, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401350833000, timestamp:0}, this:0x2b0ea17fb080, addr:[7663857-6566-0-2305843961623430293-0](offset=738453,size=887,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:348160, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1215, occupy_size:18788, data_checksum:0, max_merged_trans_version:1729061401350833000, upper_trans_version:1729061401350833000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:96, row_count_:1215, occupy_size_:18788, original_size_:60078, data_checksum_:2100735505, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401350833000, max_merged_trans_version_:1729061401350833000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea17fb398, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0ea17fb3d8, block_data:{buf:0x2b0ea17fb398, size:64, extra_buf:0x2b0ea17fb4d2, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764808025800704-0](offset=0,size=274,type=3,seq=0), orig_block_buf:0x2b0ea17fb8ab, block_data:{buf:0x2b0ea17fb8ab, size:274, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea17fb9bd, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:348160, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=799, old_pos=0, pos=799) [2024-10-16 14:57:32.548278] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=27] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0e8481a050, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0e8481a050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:65536, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:21, occupy_size:671, data_checksum:3946811441, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:21, occupy_size_:671, original_size_:1066, data_checksum_:3946811441, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0e8481a368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0e8481a3a8, block_data:{buf:0x2b0e8481a368, size:64, extra_buf:0x2b0e8481a4a2, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), orig_block_buf:0x2b0e8481a87b, block_data:{buf:0x2b0e8481a87b, size:261, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e8481a980, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:65536, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=775, old_pos=0, pos=775) [2024-10-16 14:57:32.548304] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0e8481a9b0, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328401377000, timestamp:0}, this:0x2b0e8481a9b0, addr:[7663857-6566-0-2305843903641373547-0](offset=740203,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1179648, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328401377000, upper_trans_version:1717560328401377000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:965475280, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328401377000, max_merged_trans_version_:1717560328401377000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0e8481acc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0e8481ad08, block_data:{buf:0x2b0e8481acc8, size:64, extra_buf:0x2b0e8481adf5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0e8481b1c4, block_data:{buf:0x2b0e8481b1c4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e8481b2c0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1179648, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=745, old_pos=0, pos=745) [2024-10-16 14:57:32.548345] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.548468] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f177d6660, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f177d6660, addr:[7667511-10399-0-2305844268712955904-0](offset=102400,size=1173,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90523, occupy_size:692820, data_checksum:3136814811, max_merged_trans_version:1729015201157706000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90523, occupy_size_:692820, original_size_:3801959, data_checksum_:3136814811, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:44, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328332176, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201157706000, max_merged_trans_version_:1729015201157706000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177d6978, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764900367597568-0](offset=0,size=360,type=3,seq=0), orig_block_buf:0x2b0f177d6a00, block_data:{buf:0x2b0f177d6998, size:104, extra_buf:0x2b0f177d6b68, extra_size:961, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764974455783424-0](offset=0,size=429,type=3,seq=0), orig_block_buf:0x2b0f177d6f29, block_data:{buf:0x2b0f177d6f29, size:429, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177d70d6, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.548499] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=30] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f177d7110, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401350833000, timestamp:0}, this:0x2b0f177d7110, addr:[7667511-10399-0-2305843961622795413-0](offset=103573,size=887,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:348160, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1215, occupy_size:18788, data_checksum:0, max_merged_trans_version:1729061401350833000, upper_trans_version:1729061401350833000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:96, row_count_:1215, occupy_size_:18788, original_size_:60078, data_checksum_:2100735505, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401350833000, max_merged_trans_version_:1729061401350833000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177d7428, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0f177d7468, block_data:{buf:0x2b0f177d7428, size:64, extra_buf:0x2b0f177d7562, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764808025800704-0](offset=0,size=274,type=3,seq=0), orig_block_buf:0x2b0f177d793b, block_data:{buf:0x2b0f177d793b, size:274, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177d7a4d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:348160, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.548529] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=29] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f56f9a050, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0f56f9a050, addr:[7667511-10399-0-2305843935852992524-0](offset=104460,size=863,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:65536, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:21, occupy_size:671, data_checksum:3946811441, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:21, occupy_size_:671, original_size_:1066, data_checksum_:3946811441, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f9a368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0f56f9a3a8, block_data:{buf:0x2b0f56f9a368, size:64, extra_buf:0x2b0f56f9a4a2, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), orig_block_buf:0x2b0f56f9a87b, block_data:{buf:0x2b0f56f9a87b, size:261, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f9a980, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:65536, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.548554] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f56f9a9b0, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328401377000, timestamp:0}, this:0x2b0f56f9a9b0, addr:[7667511-10399-0-2305843903640738667-0](offset=105323,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1179648, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328401377000, upper_trans_version:1717560328401377000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:965475280, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328401377000, max_merged_trans_version_:1717560328401377000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f56f9acc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f56f9ad08, block_data:{buf:0x2b0f56f9acc8, size:64, extra_buf:0x2b0f56f9adf5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0f56f9b1c4, block_data:{buf:0x2b0f56f9b1c4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f9b2c0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1179648, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.548591] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.549358] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0df1d900a0, start_pos=11747, remain=53701) [2024-10-16 14:57:32.549448] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.550888] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846984}, table_store_addr_.addr_=[7667511-10399-0-2305843633057800192-0](offset=106496,size=581,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843227183391301-0](offset=107077,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797965000704-0](offset=1048576,size=726,type=4,seq=0), this=0x2b0df1d900a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f56f9b770, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f56f9b790, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.551251] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C47-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061852550814000, start_id=1729061852550814000, end_id=1729061852550814001) [2024-10-16 14:57:32.551582] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601360, offset=9616148}, end_cursor:ObLogCursor{file_id=192, log_id=47601361, offset=9616264}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2be0050, len:1260, log_data_len:116, seq:47601360, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.551676] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=20] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686797965000704-0](offset=1048576,size=726,type=4,seq=0), new_tablet_hdl={obj:0x2b0df1d900a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0df1d900a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797965000704-0](offset=1048576,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846984}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3136814811, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332176, space_usage:{shared_data_size:724992, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:724992}}, table_store_addr:{ptr:0x2b0df1d906b0, addr:[7667511-10399-0-2305843633057800192-0](offset=106496,size=581,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843227183391301-0](offset=107077,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0df1d92e83, addr:[7663823-15172-0-4611686276126474726-0](offset=1049062,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.551747] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=59] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846984}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ea17fa060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8481ba10, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e8481ba30, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.551764] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] reset tablet(this=0x2b0ea17fa060, ls_id={id:1001}, tablet_id={id:1152921504606846984}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.552087] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:4, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:4, col_desc_array:[cnt:4, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:4, 2197175160, 2197175160, 0, 0], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:5, index_type:7, row_store_type:1, schema_version:1717560328798944, column_cnt:2, store_column_cnt:2, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:2, column_array:[cnt:2, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:2, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:4, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:4, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.552246] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.552313] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:69632, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=73728, old_block=[]) [2024-10-16 14:57:32.552331] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:null, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f177ae440, hex: 8A9C2E0000000000, int: 3054730}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f177ae4e0, hex: E80D715F7D2301E8, int: -1729061760554431000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f177ae4b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:4, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:539, meta_block_size:220, last_macro_size:759, macro_metas:0x2b0f177ae100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.552367] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), buf:0x2b0ee4bd5060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.552375] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764771518578688-0](offset=0,size=240,type=3,seq=0), buf:0x2b0ee4bd4560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), buf:0x2b0ee4bd5060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:4, data_column_checksums:[0, 0, 0, 0], row_count:3, max_merged_trans_version:1729061760554431000, contain_uncommitted_row:false, occupy_size:360, original_size:130, data_checksum:3140516469, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.552396] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0f278da060, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0f278da060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:69632, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:360, data_checksum:3140516469, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:360, original_size_:130, data_checksum_:3140516469, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764771518578688-0](offset=0,size=240,type=3,seq=0), orig_block_buf:0x2b0f278da390, block_data:{buf:0x2b0f278da5b0, size:64, extra_buf:0x2b0f278da5f0, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0f278da490, block_data:{buf:0x2b0f278da490, size:250, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f278da590, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:69632, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.552433] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] finish set upper trans version(ret=0, key_={tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0f278da130, old_val=1729061760554431000, upper_trans_version=1729061760554431000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:69632, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:360, data_checksum:3140516469, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.552443] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:69632, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.552566] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f56f045c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f56f05050, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f56f045c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f56f05050, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0ef9608980, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015200571456000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061400920852000 1729061400920852000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061760554431000 1729061760554431000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328873906000 1717560328873906000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.552701] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=20] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846991}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f56f04060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef96099f0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ef9609a10, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.552716] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e699480a0, is_inited:true, wash_score:1729061852012281000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964390400-0](offset=438272,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846991}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:524967456, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798920, space_usage:{shared_data_size:712704, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:712704}}, table_store_addr:{ptr:0x2b0e699486b0, addr:[7663857-6566-0-2305843631984635904-0](offset=684032,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843227183968836-0](offset=684612,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6994ae23, addr:[7663823-15172-0-4611686276125864422-0](offset=438758,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f56f04060, is_inited:true, wash_score:1729061852551971000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846991}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:524967456, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798920, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2846, meta_size:0, occupy_bytes:712704}}, table_store_addr:{ptr:0x2b0f56f04450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0ef96092c0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0ef9609960, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.552787] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=53] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f045d0, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f56f045d0, addr:[7663857-6566-0-2305844247238696960-0](offset=679936,size=1153,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:1392640, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90516, occupy_size:691788, data_checksum:524967456, max_merged_trans_version:1729015200571456000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:691788, original_size_:2896512, data_checksum_:524967456, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:43, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798920, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015200571456000, max_merged_trans_version_:1729015200571456000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f048e8, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764889630179328-0](offset=0,size=350,type=3,seq=0), orig_block_buf:0x2b0f56f04970, block_data:{buf:0x2b0f56f04908, size:104, extra_buf:0x2b0f56f04ace, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764963718365184-0](offset=0,size=419,type=3,seq=0), orig_block_buf:0x2b0f56f04e85, block_data:{buf:0x2b0f56f04e85, size:419, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f05028, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1392640, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1065, old_pos=0, pos=1065) [2024-10-16 14:57:32.552824] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f05070, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061400920852000, timestamp:0}, this:0x2b0f56f05070, addr:[7663857-6566-0-2305843932632343681-0](offset=681089,size=860,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:385024, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:385, occupy_size:6890, data_checksum:0, max_merged_trans_version:1729061400920852000, upper_trans_version:1729061400920852000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:94, row_count_:385, occupy_size_:6890, original_size_:16174, data_checksum_:2575509336, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061400920852000, max_merged_trans_version_:1729061400920852000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f05388, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764771518578688-0](offset=0,size=240,type=3,seq=0), orig_block_buf:0x2b0f56f053c8, block_data:{buf:0x2b0f56f05388, size:64, extra_buf:0x2b0f56f054b8, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764791919673344-0](offset=0,size=259,type=3,seq=0), orig_block_buf:0x2b0f56f05887, block_data:{buf:0x2b0f56f05887, size:259, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f0598a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:385024, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=772, old_pos=0, pos=772) [2024-10-16 14:57:32.552858] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef9608050, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0ef9608050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:69632, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:360, data_checksum:3140516469, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:360, original_size_:130, data_checksum_:3140516469, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef9608368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764771518578688-0](offset=0,size=240,type=3,seq=0), orig_block_buf:0x2b0ef96083a8, block_data:{buf:0x2b0ef9608368, size:64, extra_buf:0x2b0ef9608498, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0ef9608867, block_data:{buf:0x2b0ef9608867, size:250, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9608961, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:69632, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=754, old_pos=0, pos=754) [2024-10-16 14:57:32.552888] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=29] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef9608990, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0ef9608990, addr:[7663857-6566-0-2305843903641316135-0](offset=682791,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1044480, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ef9608ca8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ef9608ce8, block_data:{buf:0x2b0ef9608ca8, size:64, extra_buf:0x2b0ef9608dd5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0ef96091a4, block_data:{buf:0x2b0ef96091a4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef96092a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1044480, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=745, old_pos=0, pos=745) [2024-10-16 14:57:32.552930] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.553072] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ebd3fe650, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ebd3fe650, addr:[7667511-10399-0-2305844247238127616-0](offset=110592,size=1153,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:1392640, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90516, occupy_size:691788, data_checksum:524967456, max_merged_trans_version:1729015200571456000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:691788, original_size_:2896512, data_checksum_:524967456, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:43, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798920, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015200571456000, max_merged_trans_version_:1729015200571456000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ebd3fe968, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764889630179328-0](offset=0,size=350,type=3,seq=0), orig_block_buf:0x2b0ebd3fe9f0, block_data:{buf:0x2b0ebd3fe988, size:104, extra_buf:0x2b0ebd3feb4e, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764963718365184-0](offset=0,size=419,type=3,seq=0), orig_block_buf:0x2b0ebd3fef05, block_data:{buf:0x2b0ebd3fef05, size:419, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ebd3ff0a8, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1392640, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.553108] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ebd3ff0f0, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061400920852000, timestamp:0}, this:0x2b0ebd3ff0f0, addr:[7667511-10399-0-2305843932631774337-0](offset=111745,size=860,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:385024, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:385, occupy_size:6890, data_checksum:0, max_merged_trans_version:1729061400920852000, upper_trans_version:1729061400920852000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:94, row_count_:385, occupy_size_:6890, original_size_:16174, data_checksum_:2575509336, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061400920852000, max_merged_trans_version_:1729061400920852000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ebd3ff408, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764771518578688-0](offset=0,size=240,type=3,seq=0), orig_block_buf:0x2b0ebd3ff448, block_data:{buf:0x2b0ebd3ff408, size:64, extra_buf:0x2b0ebd3ff538, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764791919673344-0](offset=0,size=259,type=3,seq=0), orig_block_buf:0x2b0ebd3ff907, block_data:{buf:0x2b0ebd3ff907, size:259, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ebd3ffa0a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:385024, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.553133] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ec785e050, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0ec785e050, addr:[7667511-10399-0-2305843913304422365-0](offset=112605,size=842,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:69632, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:360, data_checksum:3140516469, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:360, original_size_:130, data_checksum_:3140516469, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ec785e368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764771518578688-0](offset=0,size=240,type=3,seq=0), orig_block_buf:0x2b0ec785e3a8, block_data:{buf:0x2b0ec785e368, size:64, extra_buf:0x2b0ec785e498, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0ec785e867, block_data:{buf:0x2b0ec785e867, size:250, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec785e961, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:69632, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.553157] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ec785e990, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0ec785e990, addr:[7667511-10399-0-2305843903640746791-0](offset=113447,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1044480, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ec785eca8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ec785ece8, block_data:{buf:0x2b0ec785eca8, size:64, extra_buf:0x2b0ec785edd5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0ec785f1a4, block_data:{buf:0x2b0ec785f1a4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec785f2a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1044480, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.553195] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=23] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.553534] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6b1a20a0, start_pos=11651, remain=53797) [2024-10-16 14:57:32.553607] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.553703] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846991}, table_store_addr_.addr_=[7667511-10399-0-2305843631984066560-0](offset=114688,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843227183399492-0](offset=115268,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797965004800-0](offset=1052672,size=726,type=4,seq=0), this=0x2b0e6b1a20a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ec785f750, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ec785f770, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.553934] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601361, offset=9616264}, end_cursor:ObLogCursor{file_id=192, log_id=47601362, offset=9616380}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c04050, len:1144, log_data_len:116, seq:47601361, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.553957] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686797965004800-0](offset=1052672,size=726,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6b1a20a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6b1a20a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797965004800-0](offset=1052672,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846991}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:524967456, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798920, space_usage:{shared_data_size:712704, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:712704}}, table_store_addr:{ptr:0x2b0e6b1a26b0, addr:[7667511-10399-0-2305843631984066560-0](offset=114688,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843227183399492-0](offset=115268,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6b1a4e23, addr:[7663823-15172-0-4611686276126478822-0](offset=1053158,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.554005] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846991}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f56f04060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef96099f0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ef9609a10, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.554020] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] reset tablet(this=0x2b0f56f04060, ls_id={id:1001}, tablet_id={id:1152921504606846991}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.554053] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1001}, tablet_id:{id:49402}}) [2024-10-16 14:57:32.554219] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:6, col_desc_array:[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:6, 2197175160, 2197175160, 0, 0, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:3, index_type:0, row_store_type:1, schema_version:1729008951550040, column_cnt:4, store_column_cnt:4, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:4, column_array:[cnt:4, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"binary", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TINYINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:4, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:6, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.554353] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=40] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.554434] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] successfully write small sstable(ret=0, block_info={nested_size:20480, nested_offset:73728, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=94208, old_block=[]) [2024-10-16 14:57:32.554455] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12ca24e0, hex: 00D0B9C4DA428009, int: 684620650731130880}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12ca2440, hex: 44630F6700000000, int: 1729061700}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12ca24f0, hex: D8AD4D6E7A2301E8, int: -1729061773189993000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12ca24b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:6, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:19250, meta_block_size:233, last_macro_size:19483, macro_metas:0x2b0f12ca2100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.554488] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0f283a1060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.554496] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), buf:0x2b0f283a0560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0f283a1060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:3, data_column_cnt:6, data_column_checksums:[0, 0, 0, 0, 0, 0], row_count:962, max_merged_trans_version:1729061774299396000, contain_uncommitted_row:false, occupy_size:18952, original_size:82744, data_checksum:3998557750, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.554516] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0ef9082060, key:{tablet_id:{id:209251}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, ref_cnt:0, upper_trans_version:1729061774299396000, timestamp:0}, this:0x2b0ef9082060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:73728, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:962, occupy_size:18952, data_checksum:3998557750, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:962, occupy_size_:18952, original_size_:82744, data_checksum_:3998557750, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061774299396000, max_merged_trans_version_:1729061774299396000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778310817000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ef9082390, block_data:{buf:0x2b0ef90825c0, size:64, extra_buf:0x2b0ef9082600, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0ef9082490, block_data:{buf:0x2b0ef9082490, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef90825a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:73728, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.554551] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] finish set upper trans version(ret=0, key_={tablet_id:{id:209251}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, meta=0x2b0ef9082130, old_val=1729061774299396000, upper_trans_version=1729061774299396000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:73728, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:962, occupy_size:18952, data_checksum:3998557750, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.554560] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:20480, nested_offset:73728, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:209251}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}) [2024-10-16 14:57:32.554694] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0ef7ed05c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0ef7ed1070, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0ef7ed05c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0ef7ed1070, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f177d69a0, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729045392735383000 0 1729045392735383000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061413096686000 1729061413096686000 1729045336369978001 1729061413753243003 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061774299396000 1729061774299396000 1729061413753243003 1729061778310817000 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778310817000 9223372036854775807 3 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836143346544000 1728836143346544000 1 1729036755796258000 0 false }}) [2024-10-16 14:57:32.554828] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=22] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1003}, tablet_id={id:209251}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ef7ed0060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f177d7ab0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f177d7ad0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.554864] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6b2700a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964328960-0](offset=376832,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1003}, tablet_id:{id:209251}, data_tablet_id:{id:209251}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836143268709001, v:0}, start_scn:{val:1729045336369978001, v:0}, clog_checkpoint_scn:{val:1729061778310817000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505000, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729045392735383000, cur_report_version:1729045392735383000, data_checksum:2515473192, row_count:93425, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755796258000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729045392735383000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836143346544000, v:0}, create_commit_version:1728836143346544000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836143165304, space_usage:{shared_data_size:1552384, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:1552384}}, table_store_addr:{ptr:0x2b0e6b2706b0, addr:[7663857-6566-0-2305843637353197568-0](offset=536576,size=585,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843270133494345-0](offset=537161,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6b272e7e, addr:[7663823-15172-0-4611686276125802982-0](offset=377318,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0ee72d19b0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}, this={this:0x2b0ef7ed0060, is_inited:true, wash_score:1729061852554136000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1003}, tablet_id:{id:209251}, data_tablet_id:{id:209251}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836143268709001, v:0}, start_scn:{val:1729045336369978001, v:0}, clog_checkpoint_scn:{val:1729061778310817000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505000, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729045392735383000, cur_report_version:1729045392735383000, data_checksum:2515473192, row_count:93425, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755796258000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729045392735383000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836143346544000, v:0}, create_commit_version:1728836143346544000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836143165304, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2903, meta_size:0, occupy_bytes:1552384}}, table_store_addr:{ptr:0x2b0ef7ed0450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0f177d72e0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0f177d7a20, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0ee72d19b0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.554938] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=58] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef7ed05d0, key:{tablet_id:{id:209251}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729045392735383000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ef7ed05d0, addr:[7663857-6566-0-2305844288040730624-0](offset=524288,size=1191,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:737280, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:93425, occupy_size:733388, data_checksum:2515473192, max_merged_trans_version:1729045392735383000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:93425, occupy_size_:733388, original_size_:7474000, data_checksum_:2515473192, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:57, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008951550040, create_snapshot_version_:1729029617350074000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729045392735383000, max_merged_trans_version_:1729045392735383000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef7ed08e8, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0ef7ed0980, block_data:{buf:0x2b0ef7ed0918, size:104, extra_buf:0x2b0ef7ed0ae6, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764980898234368-0](offset=0,size=435,type=3,seq=0), orig_block_buf:0x2b0ef7ed0e9d, block_data:{buf:0x2b0ef7ed0e9d, size:435, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef7ed1050, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:737280}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1103, old_pos=0, pos=1103) [2024-10-16 14:57:32.554975] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef7ed1090, key:{tablet_id:{id:209251}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729045336369978001, v:0}, end_scn:{val:1729061413753243003, v:0}}}, ref_cnt:0, upper_trans_version:1729061413096686000, timestamp:0}, this:0x2b0ef7ed1090, addr:[7663857-6566-0-2305843959475733671-0](offset=525479,size=885,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:790528, nested_offset:839680, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:40138, occupy_size:784263, data_checksum:0, max_merged_trans_version:1729061413096686000, upper_trans_version:1729061413096686000, filled_tx_scn:{val:1729061413753243003, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:98, row_count_:40138, occupy_size_:784263, original_size_:3452084, data_checksum_:3537926867, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:54, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061413096686000, max_merged_trans_version_:1729061413096686000, recycle_version:1729045392735383000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413753243003, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef7ed13a8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ef7ed13e8, block_data:{buf:0x2b0ef7ed13a8, size:64, extra_buf:0x2b0ef7ed14e0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0ef7ed18b7, block_data:{buf:0x2b0ef7ed18b7, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef7ed19c7, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:839680, nested_size:790528}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=797, old_pos=0, pos=797) [2024-10-16 14:57:32.555004] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f177d6050, key:{tablet_id:{id:209251}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, ref_cnt:0, upper_trans_version:1729061774299396000, timestamp:0}, this:0x2b0f177d6050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:73728, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:962, occupy_size:18952, data_checksum:3998557750, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:962, occupy_size_:18952, original_size_:82744, data_checksum_:3998557750, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061774299396000, max_merged_trans_version_:1729061774299396000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778310817000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177d6368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f177d63a8, block_data:{buf:0x2b0f177d6368, size:64, extra_buf:0x2b0f177d64a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f177d6877, block_data:{buf:0x2b0f177d6877, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177d697d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:73728, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=777, old_pos=0, pos=777) [2024-10-16 14:57:32.555035] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=30] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f177d69b0, key:{tablet_id:{id:209251}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755796258000, v:0}}}, ref_cnt:0, upper_trans_version:1728836143346544000, timestamp:0}, this:0x2b0f177d69b0, addr:[7663857-6566-0-2305843897198709629-0](offset=527229,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2072576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836143346544000, upper_trans_version:1728836143346544000, filled_tx_scn:{val:1729036755796258000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:3884449645, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836143346544000, max_merged_trans_version_:1728836143346544000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755796258000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f177d6cc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f177d6d08, block_data:{buf:0x2b0f177d6cc8, size:64, extra_buf:0x2b0f177d6df5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f177d71c4, block_data:{buf:0x2b0f177d71c4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177d72ba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2072576, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=739, old_pos=0, pos=739) [2024-10-16 14:57:32.555077] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.555383] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f56fae660, key:{tablet_id:{id:209251}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729045392735383000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f56fae660, addr:[7667511-10399-0-2305844288040325120-0](offset=118784,size=1191,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:737280, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:93425, occupy_size:733388, data_checksum:2515473192, max_merged_trans_version:1729045392735383000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:93425, occupy_size_:733388, original_size_:7474000, data_checksum_:2515473192, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:57, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008951550040, create_snapshot_version_:1729029617350074000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729045392735383000, max_merged_trans_version_:1729045392735383000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56fae978, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f56faea10, block_data:{buf:0x2b0f56fae9a8, size:104, extra_buf:0x2b0f56faeb76, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764980898234368-0](offset=0,size=435,type=3,seq=0), orig_block_buf:0x2b0f56faef2d, block_data:{buf:0x2b0f56faef2d, size:435, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56faf0e0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:737280}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.555422] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=38] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f56faf120, key:{tablet_id:{id:209251}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729045336369978001, v:0}, end_scn:{val:1729061413753243003, v:0}}}, ref_cnt:0, upper_trans_version:1729061413096686000, timestamp:0}, this:0x2b0f56faf120, addr:[7667511-10399-0-2305843959475328167-0](offset=119975,size=885,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:790528, nested_offset:839680, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:40138, occupy_size:784263, data_checksum:0, max_merged_trans_version:1729061413096686000, upper_trans_version:1729061413096686000, filled_tx_scn:{val:1729061413753243003, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:98, row_count_:40138, occupy_size_:784263, original_size_:3452084, data_checksum_:3537926867, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:54, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061413096686000, max_merged_trans_version_:1729061413096686000, recycle_version:1729045392735383000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413753243003, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56faf438, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f56faf478, block_data:{buf:0x2b0f56faf438, size:64, extra_buf:0x2b0f56faf570, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0f56faf947, block_data:{buf:0x2b0f56faf947, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56fafa57, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:839680, nested_size:790528}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.555448] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef9786050, key:{tablet_id:{id:209251}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, ref_cnt:0, upper_trans_version:1729061774299396000, timestamp:0}, this:0x2b0ef9786050, addr:[7667511-10399-0-2305843938000492572-0](offset=120860,size=865,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:73728, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:962, occupy_size:18952, data_checksum:3998557750, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:962, occupy_size_:18952, original_size_:82744, data_checksum_:3998557750, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061774299396000, max_merged_trans_version_:1729061774299396000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778310817000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef9786368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ef97863a8, block_data:{buf:0x2b0ef9786368, size:64, extra_buf:0x2b0ef97864a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0ef9786877, block_data:{buf:0x2b0ef9786877, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef978697d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:73728, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.555473] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef97869b0, key:{tablet_id:{id:209251}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755796258000, v:0}}}, ref_cnt:0, upper_trans_version:1728836143346544000, timestamp:0}, this:0x2b0ef97869b0, addr:[7667511-10399-0-2305843897198304125-0](offset=121725,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:2072576, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836143346544000, upper_trans_version:1728836143346544000, filled_tx_scn:{val:1729036755796258000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:3884449645, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836143346544000, max_merged_trans_version_:1728836143346544000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755796258000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ef9786cc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ef9786d08, block_data:{buf:0x2b0ef9786cc8, size:64, extra_buf:0x2b0ef9786df5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0ef97871c4, block_data:{buf:0x2b0ef97871c4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef97872ba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2072576, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.555514] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.555708] INFO [STORAGE] deserialize (ob_tablet_table_store.cpp:153) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to deserialize table store(ret=0, ls_id={id:1003}, tablet_id={id:209251}, major_tables_={cnt:1, sstable_array:0x2b0f177fe1d0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f177fe2b0, serialize_table_type:false, is_inited:true}, meta_major_tables_={cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, mds_sstables_={cnt:1, sstable_array:0x2b0f177fe470, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecd518, major_tables:{cnt:1, sstable_array:0x2b0f177fe1d0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f177fe2b0, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f177fe470, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729045392735383000 0 1729045392735383000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061413096686000 1729061413096686000 1729045336369978001 1729061413753243003 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061774299396000 1729061774299396000 1729061413753243003 1729061778310817000 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778310817000 9223372036854775807 4 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836143346544000 1728836143346544000 1 1729036755796258000 0 false }}) [2024-10-16 14:57:32.555904] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=27] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6984c0a0, start_pos=11742, remain=53706) [2024-10-16 14:57:32.555983] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.556132] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1003}, tablet_id={id:209251}, table_store_addr_.addr_=[7667511-10399-0-2305843637352783872-0](offset=122880,size=585,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843270133080649-0](offset=123465,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797965008896-0](offset=1056768,size=726,type=4,seq=0), this=0x2b0e6984c0a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef97877f0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ef9787810, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.556765] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601362, offset=9616380}, end_cursor:ObLogCursor{file_id=192, log_id=47601363, offset=9616496}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c68050, len:1028, log_data_len:116, seq:47601362, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.556790] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=16] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686797965008896-0](offset=1056768,size=726,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6984c0a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6984c0a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797965008896-0](offset=1056768,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1003}, tablet_id:{id:209251}, data_tablet_id:{id:209251}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836143268709001, v:0}, start_scn:{val:1729045336369978001, v:0}, clog_checkpoint_scn:{val:1729061778310817000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505000, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729045392735383000, cur_report_version:1729045392735383000, data_checksum:2515473192, row_count:93425, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755796258000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729045392735383000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836143346544000, v:0}, create_commit_version:1728836143346544000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836143165304, space_usage:{shared_data_size:1552384, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:1552384}}, table_store_addr:{ptr:0x2b0e6984c6b0, addr:[7667511-10399-0-2305843637352783872-0](offset=122880,size=585,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843270133080649-0](offset=123465,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6984ee7e, addr:[7663823-15172-0-4611686276126482918-0](offset=1057254,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0ee72d19b0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.556842] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=41] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1003}, tablet_meta_.tablet_id_={id:209251}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ef7ed0060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f177d7ab0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f177d7ad0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.556857] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] reset tablet(this=0x2b0ef7ed0060, ls_id={id:1003}, tablet_id={id:209251}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.557418] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:6, col_desc_array:[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:6, 2197175160, 2197175160, 0, 0, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:3, index_type:0, row_store_type:1, schema_version:1729008951550040, column_cnt:4, store_column_cnt:4, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:4, column_array:[cnt:4, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"binary", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TINYINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:4, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:6, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.557568] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=42] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.560542] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:332}, table_store_addr_.addr_=[7663656-3934-0-2305843510652346368-0](offset=1220608,size=467,type=2,seq=0), storage_schema_addr_.addr_=[7663656-3934-0-2305843735064388051-0](offset=1221075,size=676,type=2,seq=0), tablet_addr_=[7663348-6841-0-4611686775417253888-0](offset=1880064,size=705,type=4,seq=0), this=0x2b0e6b1d80a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6b1daa38, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e6b1daa50, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560565] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=19] reset tablet(this=0x2b0e6b1d80a0, ls_id={id:1}, tablet_id={id:332}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560581] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to gc tablet(ret=0, tablet=0x2b0e6b1d80a0) [2024-10-16 14:57:32.560590] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1002}, tablet_meta_.tablet_id_={id:1152921504606846986}, table_store_addr_.addr_=[7663857-6566-0-2305843509578018816-0](offset=634880,size=466,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843227183919570-0](offset=635346,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686780784492544-0](offset=409600,size=710,type=4,seq=0), this=0x2b0e6a2040a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:2, arr:0x2b0e6a206866, capacity:2}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6a206896, capacity:1}, shared_data_block_info_arr:{cnt:2, arr:0x2b0e6a2068ae, capacity:2}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560613] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=19] reset tablet(this=0x2b0e6a2040a0, ls_id={id:1002}, tablet_id={id:1152921504606846986}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560663] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=38] succeed to gc tablet(ret=0, tablet=0x2b0e6a2040a0) [2024-10-16 14:57:32.560672] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846984}, table_store_addr_.addr_=[7663857-6566-0-2305843633058435072-0](offset=741376,size=581,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843227184026181-0](offset=741957,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964427264-0](offset=475136,size=726,type=4,seq=0), this=0x2b0e6656c0a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6656ef03, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e6656ef1b, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560689] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=13] reset tablet(this=0x2b0e6656c0a0, ls_id={id:1001}, tablet_id={id:1152921504606846984}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560699] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to gc tablet(ret=0, tablet=0x2b0e6656c0a0) [2024-10-16 14:57:32.560706] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846991}, table_store_addr_.addr_=[7663857-6566-0-2305843631984635904-0](offset=684032,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843227183968836-0](offset=684612,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964390400-0](offset=438272,size=726,type=4,seq=0), this=0x2b0e699480a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6994aea3, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e6994aebb, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560717] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=8] reset tablet(this=0x2b0e699480a0, ls_id={id:1001}, tablet_id={id:1152921504606846991}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560726] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e699480a0) [2024-10-16 14:57:32.560733] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1003}, tablet_meta_.tablet_id_={id:209251}, table_store_addr_.addr_=[7663857-6566-0-2305843637353197568-0](offset=536576,size=585,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843270133494345-0](offset=537161,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964328960-0](offset=376832,size=726,type=4,seq=0), this=0x2b0e6b2700a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6b272efe, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e6b272f16, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560744] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=8] reset tablet(this=0x2b0e6b2700a0, ls_id={id:1003}, tablet_id={id:209251}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.560752] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e6b2700a0) [2024-10-16 14:57:32.560755] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:827) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] gc tablets in queue(gc_tablets_cnt=5, err_tablets_cnt=0, tablet_gc_queue_.count()=0) [2024-10-16 14:57:32.561748] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully write small sstable(ret=0, block_info={nested_size:20480, nested_offset:94208, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=114688, old_block=[]) [2024-10-16 14:57:32.561788] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12cb84e0, hex: 00D0394DFB408009, int: 684618591436918784}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12cb8440, hex: 44630F6700000000, int: 1729061700}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12cb84f0, hex: D8AD4D6E7A2301E8, int: -1729061773189993000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12cb84b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:6, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:18096, meta_block_size:233, last_macro_size:18329, macro_metas:0x2b0f12cb8100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.561871] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0ea175d060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.561881] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), buf:0x2b0ea175c560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0ea175d060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:3, data_column_cnt:6, data_column_checksums:[0, 0, 0, 0, 0, 0], row_count:884, max_merged_trans_version:1729061774299396000, contain_uncommitted_row:false, occupy_size:17798, original_size:76036, data_checksum:2915815781, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.561907] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0eda24a060, key:{tablet_id:{id:209245}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, ref_cnt:0, upper_trans_version:1729061774299396000, timestamp:0}, this:0x2b0eda24a060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:94208, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:884, occupy_size:17798, data_checksum:2915815781, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:884, occupy_size_:17798, original_size_:76036, data_checksum_:2915815781, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061774299396000, max_merged_trans_version_:1729061774299396000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778310817000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0eda24a390, block_data:{buf:0x2b0eda24a5c0, size:64, extra_buf:0x2b0eda24a600, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0eda24a490, block_data:{buf:0x2b0eda24a490, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0eda24a5a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:94208, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.561958] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=49] finish set upper trans version(ret=0, key_={tablet_id:{id:209245}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, meta=0x2b0eda24a130, old_val=1729061774299396000, upper_trans_version=1729061774299396000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:94208, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:884, occupy_size:17798, data_checksum:2915815781, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.561968] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:20480, nested_offset:94208, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:209245}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}) [2024-10-16 14:57:32.562226] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f334c45c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f334c5070, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f334c45c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f334c5070, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f3d7249a0, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729045392735383000 0 1729045392735383000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061413096686000 1729061413096686000 1729045336369978001 1729061413753243003 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061774299396000 1729061774299396000 1729061413753243003 1729061778310817000 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778310817000 9223372036854775807 3 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836143346544000 1728836143346544000 1 1729036755796258000 0 false }}) [2024-10-16 14:57:32.562433] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1003}, tablet_id={id:209245}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f334c4060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f3d725ab0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f3d725ad0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.562451] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e0c03e0a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964316672-0](offset=364544,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1003}, tablet_id:{id:209245}, data_tablet_id:{id:209245}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836143268709001, v:0}, start_scn:{val:1729045336369978001, v:0}, clog_checkpoint_scn:{val:1729061778310817000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505000, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729045392735383000, cur_report_version:1729045392735383000, data_checksum:1315166190, row_count:85949, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755796258000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729045392735383000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836143346544000, v:0}, create_commit_version:1728836143346544000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836143165304, space_usage:{shared_data_size:1490944, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:1490944}}, table_store_addr:{ptr:0x2b0e0c03e6b0, addr:[7663857-6566-0-2305843637353181184-0](offset=520192,size=585,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843270133477961-0](offset=520777,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e0c040e7e, addr:[7663823-15172-0-4611686276125790694-0](offset=365030,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0e44a3d0e0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}, this={this:0x2b0f334c4060, is_inited:true, wash_score:1729061852561713000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1003}, tablet_id:{id:209245}, data_tablet_id:{id:209245}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836143268709001, v:0}, start_scn:{val:1729045336369978001, v:0}, clog_checkpoint_scn:{val:1729061778310817000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505000, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729045392735383000, cur_report_version:1729045392735383000, data_checksum:1315166190, row_count:85949, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755796258000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729045392735383000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836143346544000, v:0}, create_commit_version:1728836143346544000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836143165304, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2903, meta_size:0, occupy_bytes:1490944}}, table_store_addr:{ptr:0x2b0f334c4450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0f3d7252e0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0f3d725a20, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0e44a3d0e0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.562530] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=56] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334c45d0, key:{tablet_id:{id:209245}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729045392735383000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f334c45d0, addr:[7663857-6566-0-2305844288040722432-0](offset=516096,size=1191,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:724992, nested_offset:753664, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:85949, occupy_size:721672, data_checksum:1315166190, max_merged_trans_version:1729045392735383000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:85949, occupy_size_:721672, original_size_:6875920, data_checksum_:1315166190, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:63, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008951550040, create_snapshot_version_:1729029617350074000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729045392735383000, max_merged_trans_version_:1729045392735383000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334c48e8, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f334c4980, block_data:{buf:0x2b0f334c4918, size:104, extra_buf:0x2b0f334c4ae6, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764980898234368-0](offset=0,size=435,type=3,seq=0), orig_block_buf:0x2b0f334c4e9d, block_data:{buf:0x2b0f334c4e9d, size:435, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334c5050, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:753664, nested_size:724992}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1103, old_pos=0, pos=1103) [2024-10-16 14:57:32.562580] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=47] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334c5090, key:{tablet_id:{id:209245}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729045336369978001, v:0}, end_scn:{val:1729061413753243003, v:0}}}, ref_cnt:0, upper_trans_version:1729061413096686000, timestamp:0}, this:0x2b0f334c5090, addr:[7663857-6566-0-2305843959475725479-0](offset=517287,size=885,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:741376, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:36928, occupy_size:735250, data_checksum:0, max_merged_trans_version:1729061413096686000, upper_trans_version:1729061413096686000, filled_tx_scn:{val:1729061413753243003, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:98, row_count_:36928, occupy_size_:735250, original_size_:3176024, data_checksum_:2901880491, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:54, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061413096686000, max_merged_trans_version_:1729061413096686000, recycle_version:1729045392735383000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413753243003, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334c53a8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f334c53e8, block_data:{buf:0x2b0f334c53a8, size:64, extra_buf:0x2b0f334c54e0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0f334c58b7, block_data:{buf:0x2b0f334c58b7, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334c59c7, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:741376}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=797, old_pos=0, pos=797) [2024-10-16 14:57:32.562617] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f3d724050, key:{tablet_id:{id:209245}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, ref_cnt:0, upper_trans_version:1729061774299396000, timestamp:0}, this:0x2b0f3d724050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:94208, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:884, occupy_size:17798, data_checksum:2915815781, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:884, occupy_size_:17798, original_size_:76036, data_checksum_:2915815781, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061774299396000, max_merged_trans_version_:1729061774299396000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778310817000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f3d724368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f3d7243a8, block_data:{buf:0x2b0f3d724368, size:64, extra_buf:0x2b0f3d7244a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f3d724877, block_data:{buf:0x2b0f3d724877, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f3d72497d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:94208, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=777, old_pos=0, pos=777) [2024-10-16 14:57:32.562649] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=30] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f3d7249b0, key:{tablet_id:{id:209245}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755796258000, v:0}}}, ref_cnt:0, upper_trans_version:1728836143346544000, timestamp:0}, this:0x2b0f3d7249b0, addr:[7663857-6566-0-2305843897198701437-0](offset=519037,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1806336, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836143346544000, upper_trans_version:1728836143346544000, filled_tx_scn:{val:1729036755796258000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:3884449645, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836143346544000, max_merged_trans_version_:1728836143346544000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755796258000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f3d724cc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f3d724d08, block_data:{buf:0x2b0f3d724cc8, size:64, extra_buf:0x2b0f3d724df5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f3d7251c4, block_data:{buf:0x2b0f3d7251c4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f3d7252ba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1806336, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=739, old_pos=0, pos=739) [2024-10-16 14:57:32.562700] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.562929] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0093a660, key:{tablet_id:{id:209245}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729045392735383000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f0093a660, addr:[7667511-10399-0-2305844288040333312-0](offset=126976,size=1191,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:724992, nested_offset:753664, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:85949, occupy_size:721672, data_checksum:1315166190, max_merged_trans_version:1729045392735383000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:85949, occupy_size_:721672, original_size_:6875920, data_checksum_:1315166190, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:63, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008951550040, create_snapshot_version_:1729029617350074000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729045392735383000, max_merged_trans_version_:1729045392735383000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f0093a978, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f0093aa10, block_data:{buf:0x2b0f0093a9a8, size:104, extra_buf:0x2b0f0093ab76, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764980898234368-0](offset=0,size=435,type=3,seq=0), orig_block_buf:0x2b0f0093af2d, block_data:{buf:0x2b0f0093af2d, size:435, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0093b0e0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:753664, nested_size:724992}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.562972] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=42] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0093b120, key:{tablet_id:{id:209245}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729045336369978001, v:0}, end_scn:{val:1729061413753243003, v:0}}}, ref_cnt:0, upper_trans_version:1729061413096686000, timestamp:0}, this:0x2b0f0093b120, addr:[7667511-10399-0-2305843959475336359-0](offset=128167,size=885,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:741376, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:36928, occupy_size:735250, data_checksum:0, max_merged_trans_version:1729061413096686000, upper_trans_version:1729061413096686000, filled_tx_scn:{val:1729061413753243003, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:98, row_count_:36928, occupy_size_:735250, original_size_:3176024, data_checksum_:2901880491, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:54, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061413096686000, max_merged_trans_version_:1729061413096686000, recycle_version:1729045392735383000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413753243003, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f0093b438, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f0093b478, block_data:{buf:0x2b0f0093b438, size:64, extra_buf:0x2b0f0093b570, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0f0093b947, block_data:{buf:0x2b0f0093b947, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0093ba57, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:741376}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.562998] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f12c9e050, key:{tablet_id:{id:209245}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413753243003, v:0}, end_scn:{val:1729061778310817000, v:0}}}, ref_cnt:0, upper_trans_version:1729061774299396000, timestamp:0}, this:0x2b0f12c9e050, addr:[7667511-10399-0-2305843938000500764-0](offset=129052,size=865,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:94208, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:884, occupy_size:17798, data_checksum:2915815781, max_merged_trans_version:1729061774299396000, upper_trans_version:1729061774299396000, filled_tx_scn:{val:1729061778310817000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:884, occupy_size_:17798, original_size_:76036, data_checksum_:2915815781, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061774299396000, max_merged_trans_version_:1729061774299396000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778310817000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f12c9e368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f12c9e3a8, block_data:{buf:0x2b0f12c9e368, size:64, extra_buf:0x2b0f12c9e4a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f12c9e877, block_data:{buf:0x2b0f12c9e877, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f12c9e97d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:94208, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.563022] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f12c9e9b0, key:{tablet_id:{id:209245}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755796258000, v:0}}}, ref_cnt:0, upper_trans_version:1728836143346544000, timestamp:0}, this:0x2b0f12c9e9b0, addr:[7667511-10399-0-2305843897198312317-0](offset=129917,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1806336, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836143346544000, upper_trans_version:1728836143346544000, filled_tx_scn:{val:1729036755796258000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:3884449645, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836143346544000, max_merged_trans_version_:1728836143346544000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755796258000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f12c9ecc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f12c9ed08, block_data:{buf:0x2b0f12c9ecc8, size:64, extra_buf:0x2b0f12c9edf5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f12c9f1c4, block_data:{buf:0x2b0f12c9f1c4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f12c9f2ba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1806336, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.563062] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.563400] INFO [STORAGE] deserialize (ob_tablet_table_store.cpp:153) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to deserialize table store(ret=0, ls_id={id:1003}, tablet_id={id:209245}, major_tables_={cnt:1, sstable_array:0x2b0ebd3fe1d0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0ebd3fe2b0, serialize_table_type:false, is_inited:true}, meta_major_tables_={cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, mds_sstables_={cnt:1, sstable_array:0x2b0ebd3fe470, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecd518, major_tables:{cnt:1, sstable_array:0x2b0ebd3fe1d0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0ebd3fe2b0, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0ebd3fe470, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729045392735383000 0 1729045392735383000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061413096686000 1729061413096686000 1729045336369978001 1729061413753243003 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061774299396000 1729061774299396000 1729061413753243003 1729061778310817000 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778310817000 9223372036854775807 4 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836143346544000 1728836143346544000 1 1729036755796258000 0 false }}) [2024-10-16 14:57:32.563611] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCA9F-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061852562846000, start_id=1729061852562846000, end_id=1729061852562846001) [2024-10-16 14:57:32.563686] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=37] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6a8a60a0, start_pos=11742, remain=53706) [2024-10-16 14:57:32.563777] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.563968] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1003}, tablet_id={id:209245}, table_store_addr_.addr_=[7667511-10399-0-2305843637352792064-0](offset=131072,size=585,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843270133088841-0](offset=131657,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797965012992-0](offset=1060864,size=726,type=4,seq=0), this=0x2b0e6a8a60a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f12c9f7f0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f12c9f810, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.564214] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601363, offset=9616496}, end_cursor:ObLogCursor{file_id=192, log_id=47601364, offset=9616612}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c24050, len:912, log_data_len:116, seq:47601363, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.564399] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=29] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686797965012992-0](offset=1060864,size=726,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6a8a60a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6a8a60a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797965012992-0](offset=1060864,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1003}, tablet_id:{id:209245}, data_tablet_id:{id:209245}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836143268709001, v:0}, start_scn:{val:1729045336369978001, v:0}, clog_checkpoint_scn:{val:1729061778310817000, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505000, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729045392735383000, cur_report_version:1729045392735383000, data_checksum:1315166190, row_count:85949, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755796258000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729045392735383000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836143346544000, v:0}, create_commit_version:1728836143346544000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836143165304, space_usage:{shared_data_size:1490944, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:1490944}}, table_store_addr:{ptr:0x2b0e6a8a66b0, addr:[7667511-10399-0-2305843637352792064-0](offset=131072,size=585,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843270133088841-0](offset=131657,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a8a8e7e, addr:[7663823-15172-0-4611686276126487014-0](offset=1061350,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0e44a3d0e0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.564459] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=49] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1003}, tablet_meta_.tablet_id_={id:209245}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f334c4060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f3d725ab0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f3d725ad0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.564476] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] reset tablet(this=0x2b0f334c4060, ls_id={id:1003}, tablet_id={id:209245}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.565047] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:17, rowkey_column_count:3, schema_rowkey_col_cnt:1, full_stored_col_cnt:17, col_desc_array:[cnt:17, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:17, 2197175160, 0, 0, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:3, index_type:0, row_store_type:1, schema_version:1717560328798944, column_cnt:15, store_column_cnt:15, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:1, rowkey_array:[cnt:1, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:15, column_array:[cnt:15, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"binary", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:15, rowkey_column_cnt:3, schema_rowkey_column_cnt:1, column_cnt:17, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:17, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.565210] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=82] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.565283] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:114688, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=118784, old_block=[]) [2024-10-16 14:57:32.565303] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:3, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0e8487c408, hex: 8A9C2E0000000000, int: 3054730}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0e8487c4b0, hex: E80D715F7D2301E8, int: -1729061760554431000}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0e8487c478, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:17, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:1213, meta_block_size:217, last_macro_size:1430, macro_metas:0x2b0e8487c100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.565343] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), buf:0x2b0f36e73050, height:1, is_meta_root:true}) [2024-10-16 14:57:32.565352] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), buf:0x2b0f36e72560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), buf:0x2b0f36e73050, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:17, data_column_checksums:[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], row_count:9, max_merged_trans_version:1729061762069428000, contain_uncommitted_row:false, occupy_size:1032, original_size:2350, data_checksum:435935718, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.565374] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0ef9082060, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061762069428000, timestamp:0}, this:0x2b0ef9082060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:114688, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:9, occupy_size:1032, data_checksum:435935718, max_merged_trans_version:1729061762069428000, upper_trans_version:1729061762069428000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:9, occupy_size_:1032, original_size_:2350, data_checksum_:435935718, index_type_:0, rowkey_column_count_:3, column_cnt_:17, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061762069428000, max_merged_trans_version_:1729061762069428000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0ef9082390, block_data:{buf:0x2b0ef90825a0, size:64, extra_buf:0x2b0ef90825e0, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0ef9082480, block_data:{buf:0x2b0ef9082480, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9082580, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:114688, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.565411] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] finish set upper trans version(ret=0, key_={tablet_id:{id:200070}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0ef9082130, old_val=1729061762069428000, upper_trans_version=1729061762069428000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:114688, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:9, occupy_size:1032, data_checksum:435935718, max_merged_trans_version:1729061762069428000, upper_trans_version:1729061762069428000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.565420] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:114688, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:200070}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.565557] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f56f9a5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f56f9b660, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f56f9a5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f56f9b660, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0e848848f0, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015201510676000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061401603832000 1729061401603832000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061762069428000 1729061762069428000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328873906000 1717560328873906000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.565708] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=20] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:200070}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f56f9a060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:3, arr:0x2b0e84885d10, capacity:3}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e84885d60, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e84885d80, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.565725] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6ae4c0a0, is_inited:true, wash_score:1729061852081028000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686834471915520-0](offset=741376,size=760,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:200070}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3777861405, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798896, space_usage:{shared_data_size:28672, data_size:6291456, shared_meta_size:16384, meta_size:0, occupy_bytes:2008708}}, table_store_addr:{ptr:0x2b0e6ae4c68c, addr:[7663857-6566-0-2305843633059241984-0](offset=1548288,size=581,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843499915256389-0](offset=1548869,size=457,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6ae4f2fc, addr:[7663823-15172-0-4611686319075840480-0](offset=741856,size=280,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f56f9a060, is_inited:true, wash_score:1729061852564969000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:200070}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3777861405, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798896, space_usage:{shared_data_size:6291456, data_size:6291456, shared_meta_size:3781, meta_size:0, occupy_bytes:2008708}}, table_store_addr:{ptr:0x2b0f56f9a450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0e84885230, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0e84885c80, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.565799] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=54] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f9a5d0, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f56f9a5d0, addr:[7663857-6566-0-2305845246893195264-0](offset=1540096,size=2084,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:3, nested_size:2097152, nested_offset:0, total_macro_block_count:3, total_use_old_macro_block_count:2, row_count:90516, occupy_size:1980036, data_checksum:3777861405, max_merged_trans_version:1729015201510676000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:1980036, original_size_:34125661, data_checksum_:3777861405, index_type_:0, rowkey_column_count_:3, column_cnt_:17, data_macro_block_count_:3, data_micro_block_count_:127, use_old_macro_block_count_:2, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798944, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201510676000, max_merged_trans_version_:1729015201510676000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f9a8e8, column_checksum_count:17, data_root_info:{addr:[0-0-0-3458765100083576832-0](offset=0,size=546,type=3,seq=0), orig_block_buf:0x2b0f56f9a9d0, block_data:{buf:0x2b0f56f9a970, size:96, extra_buf:0x2b0f56f9abf2, extra_size:1575, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765603668492288-0](offset=0,size=1015,type=3,seq=0), orig_block_buf:0x2b0f56f9b219, block_data:{buf:0x2b0f56f9b219, size:1015, extra_buf:null, extra_size:0, type:1}}, data_block_count:3, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f9b610, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:0, nested_size:2097152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1996, old_pos=0, pos=1996) [2024-10-16 14:57:32.565838] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f9b680, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401603832000, timestamp:0}, this:0x2b0f56f9b680, addr:[7663857-6566-0-2305843943370623012-0](offset=1542180,size=870,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:995328, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:415, occupy_size:17951, data_checksum:0, max_merged_trans_version:1729061401603832000, upper_trans_version:1729061401603832000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:96, row_count_:415, occupy_size_:17951, original_size_:142348, data_checksum_:724211031, index_type_:0, rowkey_column_count_:3, column_cnt_:17, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401603832000, max_merged_trans_version_:1729061401603832000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f9b998, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0f56f9b9d8, block_data:{buf:0x2b0f56f9b998, size:64, extra_buf:0x2b0f56f9bac6, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764802657091584-0](offset=0,size=269,type=3,seq=0), orig_block_buf:0x2b0f56f9be05, block_data:{buf:0x2b0f56f9be05, size:269, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f9bf12, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:995328, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=782, old_pos=0, pos=782) [2024-10-16 14:57:32.565867] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=27] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0e84884050, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061762069428000, timestamp:0}, this:0x2b0e84884050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:114688, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:9, occupy_size:1032, data_checksum:435935718, max_merged_trans_version:1729061762069428000, upper_trans_version:1729061762069428000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:9, occupy_size_:1032, original_size_:2350, data_checksum_:435935718, index_type_:0, rowkey_column_count_:3, column_cnt_:17, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061762069428000, max_merged_trans_version_:1729061762069428000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0e84884368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0e848843a8, block_data:{buf:0x2b0e84884368, size:64, extra_buf:0x2b0e84884496, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0e848847d5, block_data:{buf:0x2b0e848847d5, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e848848d4, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:114688, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=757, old_pos=0, pos=757) [2024-10-16 14:57:32.565894] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0e84884900, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0e84884900, addr:[7663857-6566-0-2305843897199726295-0](offset=1543895,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1568768, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0e84884c18, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0e84884c58, block_data:{buf:0x2b0e84884c18, size:64, extra_buf:0x2b0e84884d45, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0e84885114, block_data:{buf:0x2b0e84885114, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e8488520a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1568768, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=739, old_pos=0, pos=739) [2024-10-16 14:57:32.565937] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.566064] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f12ca2960, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f12ca2960, addr:[7667511-10399-0-2305845246891790336-0](offset=135168,size=2084,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:3, nested_size:2097152, nested_offset:0, total_macro_block_count:3, total_use_old_macro_block_count:2, row_count:90516, occupy_size:1980036, data_checksum:3777861405, max_merged_trans_version:1729015201510676000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:1980036, original_size_:34125661, data_checksum_:3777861405, index_type_:0, rowkey_column_count_:3, column_cnt_:17, data_macro_block_count_:3, data_micro_block_count_:127, use_old_macro_block_count_:2, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798944, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201510676000, max_merged_trans_version_:1729015201510676000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f12ca2c78, column_checksum_count:17, data_root_info:{addr:[0-0-0-3458765100083576832-0](offset=0,size=546,type=3,seq=0), orig_block_buf:0x2b0f12ca2d60, block_data:{buf:0x2b0f12ca2d00, size:96, extra_buf:0x2b0f12ca2f82, extra_size:1575, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765603668492288-0](offset=0,size=1015,type=3,seq=0), orig_block_buf:0x2b0f12ca35a9, block_data:{buf:0x2b0f12ca35a9, size:1015, extra_buf:null, extra_size:0, type:1}}, data_block_count:3, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f12ca39a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:0, nested_size:2097152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.566095] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=30] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ea175c050, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401603832000, timestamp:0}, this:0x2b0ea175c050, addr:[7667511-10399-0-2305843943369218084-0](offset=137252,size=870,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:995328, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:415, occupy_size:17951, data_checksum:0, max_merged_trans_version:1729061401603832000, upper_trans_version:1729061401603832000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:96, row_count_:415, occupy_size_:17951, original_size_:142348, data_checksum_:724211031, index_type_:0, rowkey_column_count_:3, column_cnt_:17, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401603832000, max_merged_trans_version_:1729061401603832000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea175c368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0ea175c3a8, block_data:{buf:0x2b0ea175c368, size:64, extra_buf:0x2b0ea175c496, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764802657091584-0](offset=0,size=269,type=3,seq=0), orig_block_buf:0x2b0ea175c7d5, block_data:{buf:0x2b0ea175c7d5, size:269, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea175c8e2, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:995328, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.566120] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ea175c900, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061762069428000, timestamp:0}, this:0x2b0ea175c900, addr:[7667511-10399-0-2305843916525673354-0](offset=138122,size=845,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:114688, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:9, occupy_size:1032, data_checksum:435935718, max_merged_trans_version:1729061762069428000, upper_trans_version:1729061762069428000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:9, occupy_size_:1032, original_size_:2350, data_checksum_:435935718, index_type_:0, rowkey_column_count_:3, column_cnt_:17, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061762069428000, max_merged_trans_version_:1729061762069428000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea175cc18, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0ea175cc58, block_data:{buf:0x2b0ea175cc18, size:64, extra_buf:0x2b0ea175cd46, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764787624706048-0](offset=0,size=255,type=3,seq=0), orig_block_buf:0x2b0ea175d085, block_data:{buf:0x2b0ea175d085, size:255, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea175d184, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:114688, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.566145] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ea175d1b0, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0ea175d1b0, addr:[7667511-10399-0-2305843897198321367-0](offset=138967,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1568768, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ea175d4c8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ea175d508, block_data:{buf:0x2b0ea175d4c8, size:64, extra_buf:0x2b0ea175d5f5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0ea175d9c4, block_data:{buf:0x2b0ea175d9c4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea175daba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1568768, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.566187] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.566535] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e0c0f20a0, start_pos=12892, remain=52556) [2024-10-16 14:57:32.566611] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.566703] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:200070}, table_store_addr_.addr_=[7667511-10399-0-2305843633057837056-0](offset=143360,size=581,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843499913851461-0](offset=143941,size=457,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686834472239104-0](offset=1064960,size=760,type=4,seq=0), this=0x2b0e0c0f20a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:3, arr:0x2b0ee4bd4450, capacity:3}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ee4bd44a0, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0ee4bd44c0, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.566913] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601364, offset=9616612}, end_cursor:ObLogCursor{file_id=192, log_id=47601365, offset=9616728}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ca4050, len:796, log_data_len:116, seq:47601364, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.566928] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686834472239104-0](offset=1064960,size=760,type=4,seq=0), new_tablet_hdl={obj:0x2b0e0c0f20a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e0c0f20a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686834472239104-0](offset=1064960,size=760,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:200070}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3777861405, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798896, space_usage:{shared_data_size:28672, data_size:6291456, shared_meta_size:16384, meta_size:0, occupy_bytes:2008708}}, table_store_addr:{ptr:0x2b0e0c0f268c, addr:[7667511-10399-0-2305843633057837056-0](offset=143360,size=581,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843499913851461-0](offset=143941,size=457,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e0c0f52fc, addr:[7663823-15172-0-4611686319076164064-0](offset=1065440,size=280,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.566973] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:200070}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f56f9a060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:3, arr:0x2b0e84885d10, capacity:3}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e84885d60, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e84885d80, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.566988] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] reset tablet(this=0x2b0f56f9a060, ls_id={id:1001}, tablet_id={id:200070}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.567314] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.567430] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:4, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:4, col_desc_array:[cnt:4, column_id=17 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:4, 2197175160, 2197175160, 0, 0], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:5, index_type:7, row_store_type:1, schema_version:1717560328798944, column_cnt:2, store_column_cnt:2, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:17, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0}, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:2, column_array:[cnt:2, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:2, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:4, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:4, column_id=17 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.567547] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=37] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.567660] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:118784, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=122880, old_block=[]) [2024-10-16 14:57:32.567677] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 13, flag: 0, null: 0, ptr: 0x2b0ec785e4e0, hex: 5359535F5343484544554C4544, cstr: SYS_SCHEDULED}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0ec785e440, hex: 8A9C2E0000000000, int: 3054730}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0ec785e4f0, hex: E80D715F7D2301E8, int: -1729061760554431000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0ec785e4b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:4, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:566, meta_block_size:233, last_macro_size:799, macro_metas:0x2b0ec785e100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.567713] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764796214640640-0](offset=0,size=263,type=3,seq=0), buf:0x2b0f283a1060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.567721] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764785477222400-0](offset=0,size=253,type=3,seq=0), buf:0x2b0f283a0560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764796214640640-0](offset=0,size=263,type=3,seq=0), buf:0x2b0f283a1060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:4, data_column_checksums:[0, 0, 0, 0], row_count:3, max_merged_trans_version:1729061760554431000, contain_uncommitted_row:false, occupy_size:374, original_size:169, data_checksum:619713868, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.567741] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0f278da060, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0f278da060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:118784, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:374, data_checksum:619713868, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:374, original_size_:169, data_checksum_:619713868, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764785477222400-0](offset=0,size=253,type=3,seq=0), orig_block_buf:0x2b0f278da390, block_data:{buf:0x2b0f278da5c0, size:64, extra_buf:0x2b0f278da600, extra_size:988, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764796214640640-0](offset=0,size=263,type=3,seq=0), orig_block_buf:0x2b0f278da490, block_data:{buf:0x2b0f278da490, size:263, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f278da5a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:118784, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.567776] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] finish set upper trans version(ret=0, key_={tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0f278da130, old_val=1729061760554431000, upper_trans_version=1729061760554431000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:118784, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:374, data_checksum:619713868, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.567786] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:118784, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.568083] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f12c9e5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f12c9f070, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f12c9e5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f12c9f070, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0ef96089b0, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015200571456000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061400920852000 1729061400920852000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061760554431000 1729061760554431000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328873906000 1717560328873906000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.568223] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=33] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846990}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f12c9e060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef9609a20, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ef9609a40, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.568239] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6a1000a0, is_inited:true, wash_score:1729061852015429000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964369920-0](offset=417792,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846990}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:581363308, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798912, space_usage:{shared_data_size:712704, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:712704}}, table_store_addr:{ptr:0x2b0e6a1006b0, addr:[7663857-6566-0-2305843631984607232-0](offset=655360,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843227183940164-0](offset=655940,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a102e9a, addr:[7663823-15172-0-4611686276125843942-0](offset=418278,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f12c9e060, is_inited:true, wash_score:1729061852567331000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846990}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:581363308, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798912, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2901, meta_size:0, occupy_bytes:712704}}, table_store_addr:{ptr:0x2b0f12c9e450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0ef96092f0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0ef9609990, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.568327] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=53] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f12c9e5d0, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f12c9e5d0, addr:[7663857-6566-0-2305844275155955712-0](offset=651264,size=1179,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:708608, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90516, occupy_size:692334, data_checksum:581363308, max_merged_trans_version:1729015200571456000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:692334, original_size_:4073213, data_checksum_:581363308, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:43, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798912, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015200571456000, max_merged_trans_version_:1729015200571456000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f12c9e8e8, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764903588823040-0](offset=0,size=363,type=3,seq=0), orig_block_buf:0x2b0f12c9e970, block_data:{buf:0x2b0f12c9e908, size:104, extra_buf:0x2b0f12c9eadb, extra_size:964, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764977677008896-0](offset=0,size=432,type=3,seq=0), orig_block_buf:0x2b0f12c9ee9f, block_data:{buf:0x2b0f12c9ee9f, size:432, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f12c9f04f, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:708608, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1091, old_pos=0, pos=1091) [2024-10-16 14:57:32.568365] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f12c9f090, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061400920852000, timestamp:0}, this:0x2b0f12c9f090, addr:[7663857-6566-0-2305843963770827931-0](offset=652443,size=889,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:286720, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:385, occupy_size:7010, data_checksum:0, max_merged_trans_version:1729061400920852000, upper_trans_version:1729061400920852000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:385, occupy_size_:7010, original_size_:21183, data_checksum_:2489115202, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061400920852000, max_merged_trans_version_:1729061400920852000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f12c9f3a8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764785477222400-0](offset=0,size=253,type=3,seq=0), orig_block_buf:0x2b0f12c9f3e8, block_data:{buf:0x2b0f12c9f3a8, size:64, extra_buf:0x2b0f12c9f4e5, extra_size:988, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764808025800704-0](offset=0,size=274,type=3,seq=0), orig_block_buf:0x2b0f12c9f8c1, block_data:{buf:0x2b0f12c9f8c1, size:274, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f12c9f9d3, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:286720, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=801, old_pos=0, pos=801) [2024-10-16 14:57:32.568399] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=31] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef9608050, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0ef9608050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:118784, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:374, data_checksum:619713868, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:374, original_size_:169, data_checksum_:619713868, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef9608368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764785477222400-0](offset=0,size=253,type=3,seq=0), orig_block_buf:0x2b0ef96083a8, block_data:{buf:0x2b0ef9608368, size:64, extra_buf:0x2b0ef96084a5, extra_size:988, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764796214640640-0](offset=0,size=263,type=3,seq=0), orig_block_buf:0x2b0ef9608881, block_data:{buf:0x2b0ef9608881, size:263, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9608988, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:118784, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=780, old_pos=0, pos=780) [2024-10-16 14:57:32.568426] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef96089c0, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0ef96089c0, addr:[7663857-6566-0-2305843903641287544-0](offset=654200,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:241664, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ef9608cd8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ef9608d18, block_data:{buf:0x2b0ef9608cd8, size:64, extra_buf:0x2b0ef9608e05, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0ef96091d4, block_data:{buf:0x2b0ef96091d4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef96092d0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:241664, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=745, old_pos=0, pos=745) [2024-10-16 14:57:32.568468] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.568654] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0bdfe660, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f0bdfe660, addr:[7667511-10399-0-2305844275155451904-0](offset=147456,size=1179,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:708608, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90516, occupy_size:692334, data_checksum:581363308, max_merged_trans_version:1729015200571456000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:692334, original_size_:4073213, data_checksum_:581363308, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:43, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798912, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015200571456000, max_merged_trans_version_:1729015200571456000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f0bdfe978, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764903588823040-0](offset=0,size=363,type=3,seq=0), orig_block_buf:0x2b0f0bdfea00, block_data:{buf:0x2b0f0bdfe998, size:104, extra_buf:0x2b0f0bdfeb6b, extra_size:964, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764977677008896-0](offset=0,size=432,type=3,seq=0), orig_block_buf:0x2b0f0bdfef2f, block_data:{buf:0x2b0f0bdfef2f, size:432, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0bdff0df, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:708608, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.568702] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=46] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0bdff120, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061400920852000, timestamp:0}, this:0x2b0f0bdff120, addr:[7667511-10399-0-2305843963770324123-0](offset=148635,size=889,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:286720, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:385, occupy_size:7010, data_checksum:0, max_merged_trans_version:1729061400920852000, upper_trans_version:1729061400920852000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:385, occupy_size_:7010, original_size_:21183, data_checksum_:2489115202, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061400920852000, max_merged_trans_version_:1729061400920852000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f0bdff438, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764785477222400-0](offset=0,size=253,type=3,seq=0), orig_block_buf:0x2b0f0bdff478, block_data:{buf:0x2b0f0bdff438, size:64, extra_buf:0x2b0f0bdff575, extra_size:988, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764808025800704-0](offset=0,size=274,type=3,seq=0), orig_block_buf:0x2b0f0bdff951, block_data:{buf:0x2b0f0bdff951, size:274, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0bdffa63, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:286720, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.568728] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef9786050, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0ef9786050, addr:[7667511-10399-0-2305843941221746708-0](offset=149524,size=868,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:118784, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:374, data_checksum:619713868, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:374, original_size_:169, data_checksum_:619713868, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef9786368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764785477222400-0](offset=0,size=253,type=3,seq=0), orig_block_buf:0x2b0ef97863a8, block_data:{buf:0x2b0ef9786368, size:64, extra_buf:0x2b0ef97864a5, extra_size:988, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764796214640640-0](offset=0,size=263,type=3,seq=0), orig_block_buf:0x2b0ef9786881, block_data:{buf:0x2b0ef9786881, size:263, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9786988, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:118784, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.568757] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=29] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef97869c0, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0ef97869c0, addr:[7667511-10399-0-2305843903640783736-0](offset=150392,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:241664, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ef9786cd8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ef9786d18, block_data:{buf:0x2b0ef9786cd8, size:64, extra_buf:0x2b0ef9786e05, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0ef97871d4, block_data:{buf:0x2b0ef97871d4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef97872d0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:241664, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.568797] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.569153] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0df1ca60a0, start_pos=11770, remain=53678) [2024-10-16 14:57:32.569221] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.569242] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.569325] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846990}, table_store_addr_.addr_=[7667511-10399-0-2305843631984103424-0](offset=151552,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843227183436356-0](offset=152132,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797965021184-0](offset=1069056,size=726,type=4,seq=0), this=0x2b0df1ca60a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef9787780, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ef97877a0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.569545] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=8] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601365, offset=9616728}, end_cursor:ObLogCursor{file_id=192, log_id=47601366, offset=9616844}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2a80050, len:680, log_data_len:116, seq:47601365, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.569568] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686797965021184-0](offset=1069056,size=726,type=4,seq=0), new_tablet_hdl={obj:0x2b0df1ca60a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0df1ca60a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797965021184-0](offset=1069056,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846990}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:581363308, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798912, space_usage:{shared_data_size:712704, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:712704}}, table_store_addr:{ptr:0x2b0df1ca66b0, addr:[7667511-10399-0-2305843631984103424-0](offset=151552,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843227183436356-0](offset=152132,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0df1ca8e9a, addr:[7663823-15172-0-4611686276126495206-0](offset=1069542,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.569618] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=40] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846990}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f12c9e060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef9609a20, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ef9609a40, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.569633] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] reset tablet(this=0x2b0f12c9e060, ls_id={id:1001}, tablet_id={id:1152921504606846990}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.569765] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:4, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:4, col_desc_array:[cnt:4, column_id=17 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:4, 2197175160, 2197175160, 0, 0], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:5, index_type:7, row_store_type:1, schema_version:1717560328798944, column_cnt:2, store_column_cnt:2, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:17, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0}, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:2, column_array:[cnt:2, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:2, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:4, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:4, column_id=17 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.569934] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=47] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.569997] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:122880, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=126976, old_block=[]) [2024-10-16 14:57:32.570013] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 10, flag: 0, null: 0, ptr: 0x2b0f0bdf44e0, hex: 5355434345535346554C, cstr: SUCCESSFUL}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f0bdf4440, hex: 8A9C2E0000000000, int: 3054730}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f0bdf44f0, hex: E00824057D2301E8, int: -1729061762069428000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f0bdf44b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:4, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:707, meta_block_size:231, last_macro_size:938, macro_metas:0x2b0f0bdf4100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.570047] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), buf:0x2b0f334c5060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.570060] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), buf:0x2b0f334c4560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), buf:0x2b0f334c5060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:4, data_column_checksums:[0, 0, 0, 0], row_count:12, max_merged_trans_version:1729061762069428000, contain_uncommitted_row:false, occupy_size:515, original_size:622, data_checksum:1251338713, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.570079] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0eda24a060, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061762069428000, timestamp:0}, this:0x2b0eda24a060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:122880, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:515, data_checksum:1251338713, max_merged_trans_version:1729061762069428000, upper_trans_version:1729061762069428000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:12, occupy_size_:515, original_size_:622, data_checksum_:1251338713, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061762069428000, max_merged_trans_version_:1729061762069428000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0eda24a390, block_data:{buf:0x2b0eda24a5c0, size:64, extra_buf:0x2b0eda24a600, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), orig_block_buf:0x2b0eda24a490, block_data:{buf:0x2b0eda24a490, size:261, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0eda24a5a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:122880, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.570113] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] finish set upper trans version(ret=0, key_={tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0eda24a130, old_val=1729061762069428000, upper_trans_version=1729061762069428000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:122880, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:515, data_checksum:1251338713, max_merged_trans_version:1729061762069428000, upper_trans_version:1729061762069428000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.570122] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:122880, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.570249] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0ea170e5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0ea170f060, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0ea170e5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0ea170f060, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0ea175c9a0, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015201510676000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061401603832000 1729061401603832000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061762069428000 1729061762069428000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328873906000 1717560328873906000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.570402] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=20] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846989}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ea170e060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ea175da10, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ea175da30, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.570418] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6aa700a0, is_inited:true, wash_score:1729061852069511000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964697600-0](offset=745472,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846989}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3286913364, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798904, space_usage:{shared_data_size:720896, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:720896}}, table_store_addr:{ptr:0x2b0e6aa706b0, addr:[7663857-6566-0-2305843631985508352-0](offset=1556480,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843227184841284-0](offset=1557060,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6aa72e80, addr:[7663823-15172-0-4611686276126171622-0](offset=745958,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0ea170e060, is_inited:true, wash_score:1729061852569562000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846989}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3286913364, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798904, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2889, meta_size:0, occupy_bytes:720896}}, table_store_addr:{ptr:0x2b0ea170e450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0ea175d2e0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0ea175d980, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.570510] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=65] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea170e5d0, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ea170e5d0, addr:[7663857-6566-0-2305844268714405888-0](offset=1552384,size=1173,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90516, occupy_size:692809, data_checksum:3286913364, max_merged_trans_version:1729015201510676000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:692809, original_size_:3801661, data_checksum_:3286913364, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:44, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798904, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201510676000, max_merged_trans_version_:1729015201510676000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea170e8e8, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764900367597568-0](offset=0,size=360,type=3,seq=0), orig_block_buf:0x2b0ea170e970, block_data:{buf:0x2b0ea170e908, size:104, extra_buf:0x2b0ea170ead8, extra_size:961, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764974455783424-0](offset=0,size=429,type=3,seq=0), orig_block_buf:0x2b0ea170ee99, block_data:{buf:0x2b0ea170ee99, size:429, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea170f046, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1085, old_pos=0, pos=1085) [2024-10-16 14:57:32.570548] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea170f080, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401603832000, timestamp:0}, this:0x2b0ea170f080, addr:[7663857-6566-0-2305843957329278101-0](offset=1553557,size=883,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:16384, nested_offset:1015808, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:800, occupy_size:13731, data_checksum:0, max_merged_trans_version:1729061401603832000, upper_trans_version:1729061401603832000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:800, occupy_size_:13731, original_size_:40468, data_checksum_:2891618477, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401603832000, max_merged_trans_version_:1729061401603832000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea170f398, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0ea170f3d8, block_data:{buf:0x2b0ea170f398, size:64, extra_buf:0x2b0ea170f4d2, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764804804575232-0](offset=0,size=271,type=3,seq=0), orig_block_buf:0x2b0ea170f8ab, block_data:{buf:0x2b0ea170f8ab, size:271, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea170f9ba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1015808, nested_size:16384}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=795, old_pos=0, pos=795) [2024-10-16 14:57:32.570583] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=33] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea175c050, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061762069428000, timestamp:0}, this:0x2b0ea175c050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:122880, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:515, data_checksum:1251338713, max_merged_trans_version:1729061762069428000, upper_trans_version:1729061762069428000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:12, occupy_size_:515, original_size_:622, data_checksum_:1251338713, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061762069428000, max_merged_trans_version_:1729061762069428000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea175c368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0ea175c3a8, block_data:{buf:0x2b0ea175c368, size:64, extra_buf:0x2b0ea175c4a2, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), orig_block_buf:0x2b0ea175c87b, block_data:{buf:0x2b0ea175c87b, size:261, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea175c980, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:122880, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=775, old_pos=0, pos=775) [2024-10-16 14:57:32.570616] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=32] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea175c9b0, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0ea175c9b0, addr:[7663857-6566-0-2305843903642188647-0](offset=1555303,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1306624, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ea175ccc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ea175cd08, block_data:{buf:0x2b0ea175ccc8, size:64, extra_buf:0x2b0ea175cdf5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0ea175d1c4, block_data:{buf:0x2b0ea175d1c4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea175d2c0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1306624, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=745, old_pos=0, pos=745) [2024-10-16 14:57:32.570658] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.571631] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ec821e660, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ec821e660, addr:[7667511-10399-0-2305844268713009152-0](offset=155648,size=1173,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90516, occupy_size:692809, data_checksum:3286913364, max_merged_trans_version:1729015201510676000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:692809, original_size_:3801661, data_checksum_:3286913364, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:44, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798904, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201510676000, max_merged_trans_version_:1729015201510676000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ec821e978, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764900367597568-0](offset=0,size=360,type=3,seq=0), orig_block_buf:0x2b0ec821ea00, block_data:{buf:0x2b0ec821e998, size:104, extra_buf:0x2b0ec821eb68, extra_size:961, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764974455783424-0](offset=0,size=429,type=3,seq=0), orig_block_buf:0x2b0ec821ef29, block_data:{buf:0x2b0ec821ef29, size:429, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec821f0d6, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.571672] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=40] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ec821f110, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401603832000, timestamp:0}, this:0x2b0ec821f110, addr:[7667511-10399-0-2305843957327881365-0](offset=156821,size=883,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:16384, nested_offset:1015808, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:800, occupy_size:13731, data_checksum:0, max_merged_trans_version:1729061401603832000, upper_trans_version:1729061401603832000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:800, occupy_size_:13731, original_size_:40468, data_checksum_:2891618477, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401603832000, max_merged_trans_version_:1729061401603832000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ec821f428, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0ec821f468, block_data:{buf:0x2b0ec821f428, size:64, extra_buf:0x2b0ec821f562, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764804804575232-0](offset=0,size=271,type=3,seq=0), orig_block_buf:0x2b0ec821f93b, block_data:{buf:0x2b0ec821f93b, size:271, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec821fa4a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1015808, nested_size:16384}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.571702] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=29] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e76050, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061762069428000, timestamp:0}, this:0x2b0f36e76050, addr:[7667511-10399-0-2305843935853045768-0](offset=157704,size=863,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:122880, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:515, data_checksum:1251338713, max_merged_trans_version:1729061762069428000, upper_trans_version:1729061762069428000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:12, occupy_size_:515, original_size_:622, data_checksum_:1251338713, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061762069428000, max_merged_trans_version_:1729061762069428000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e76368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764782255996928-0](offset=0,size=250,type=3,seq=0), orig_block_buf:0x2b0f36e763a8, block_data:{buf:0x2b0f36e76368, size:64, extra_buf:0x2b0f36e764a2, extra_size:985, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764794067156992-0](offset=0,size=261,type=3,seq=0), orig_block_buf:0x2b0f36e7687b, block_data:{buf:0x2b0f36e7687b, size:261, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e76980, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:122880, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.571728] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e769b0, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0f36e769b0, addr:[7667511-10399-0-2305843903640791911-0](offset=158567,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1306624, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f36e76cc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f36e76d08, block_data:{buf:0x2b0f36e76cc8, size:64, extra_buf:0x2b0f36e76df5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0f36e771c4, block_data:{buf:0x2b0f36e771c4, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e772c0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1306624, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.571774] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=29] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.572173] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e695b40a0, start_pos=11744, remain=53704) [2024-10-16 14:57:32.572237] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.572332] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846989}, table_store_addr_.addr_=[7667511-10399-0-2305843631984111616-0](offset=159744,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843227183444548-0](offset=160324,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797965025280-0](offset=1073152,size=726,type=4,seq=0), this=0x2b0e695b40a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f36e77770, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f36e77790, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.572536] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601366, offset=9616844}, end_cursor:ObLogCursor{file_id=192, log_id=47601367, offset=9616960}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2adc050, len:564, log_data_len:116, seq:47601366, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.572552] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686797965025280-0](offset=1073152,size=726,type=4,seq=0), new_tablet_hdl={obj:0x2b0e695b40a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e695b40a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797965025280-0](offset=1073152,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846989}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:3286913364, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798904, space_usage:{shared_data_size:720896, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:720896}}, table_store_addr:{ptr:0x2b0e695b46b0, addr:[7667511-10399-0-2305843631984111616-0](offset=159744,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843227183444548-0](offset=160324,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e695b6e80, addr:[7663823-15172-0-4611686276126499302-0](offset=1073638,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.572596] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846989}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ea170e060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ea175da10, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ea175da30, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.572616] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=17] reset tablet(this=0x2b0ea170e060, ls_id={id:1001}, tablet_id={id:1152921504606846989}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.572733] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:6, col_desc_array:[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:6, 2197175160, 2197175160, 0, 0, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:3, index_type:0, row_store_type:1, schema_version:1729008952878920, column_cnt:4, store_column_cnt:4, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:4, column_array:[cnt:4, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"binary", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TINYINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:4, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:6, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.575812] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.575834] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.575850] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.583138] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1003}, tablet_meta_.tablet_id_={id:209245}, table_store_addr_.addr_=[7663857-6566-0-2305843637353181184-0](offset=520192,size=585,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843270133477961-0](offset=520777,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964316672-0](offset=364544,size=726,type=4,seq=0), this=0x2b0e0c03e0a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e0c040efe, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e0c040f16, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.583163] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=20] reset tablet(this=0x2b0e0c03e0a0, ls_id={id:1003}, tablet_id={id:209245}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.583177] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to gc tablet(ret=0, tablet=0x2b0e0c03e0a0) [2024-10-16 14:57:32.583188] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:200070}, table_store_addr_.addr_=[7663857-6566-0-2305843633059241984-0](offset=1548288,size=581,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843499915256389-0](offset=1548869,size=457,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686834471915520-0](offset=741376,size=760,type=4,seq=0), this=0x2b0e6ae4c0a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:3, arr:0x2b0e6ae4f37c, capacity:3}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6ae4f3c4, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e6ae4f3dc, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.583220] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=30] reset tablet(this=0x2b0e6ae4c0a0, ls_id={id:1001}, tablet_id={id:200070}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.583232] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to gc tablet(ret=0, tablet=0x2b0e6ae4c0a0) [2024-10-16 14:57:32.583240] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846990}, table_store_addr_.addr_=[7663857-6566-0-2305843631984607232-0](offset=655360,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843227183940164-0](offset=655940,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964369920-0](offset=417792,size=726,type=4,seq=0), this=0x2b0e6a1000a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6a102f1a, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e6a102f32, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.583251] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=9] reset tablet(this=0x2b0e6a1000a0, ls_id={id:1001}, tablet_id={id:1152921504606846990}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.583261] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e6a1000a0) [2024-10-16 14:57:32.583268] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846989}, table_store_addr_.addr_=[7663857-6566-0-2305843631985508352-0](offset=1556480,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843227184841284-0](offset=1557060,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964697600-0](offset=745472,size=726,type=4,seq=0), this=0x2b0e6aa700a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6aa72f00, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e6aa72f18, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.583279] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=8] reset tablet(this=0x2b0e6aa700a0, ls_id={id:1001}, tablet_id={id:1152921504606846989}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.583288] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e6aa700a0) [2024-10-16 14:57:32.583291] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:827) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] gc tablets in queue(gc_tablets_cnt=4, err_tablets_cnt=0, tablet_gc_queue_.count()=0) [2024-10-16 14:57:32.584037] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1}, {id:1003}]) [2024-10-16 14:57:32.586611] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [22451][T1003_ObXAHbWor][T1003][Y0-0000000000000000-0-0] [lt=16] XA scheduler heartbeat task statistics(avg_time=0) [2024-10-16 14:57:32.586669] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:32.586681] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:32.586905] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:14538477222}, scn={val:1729061852134673005, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:32.586926] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:19, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061852586864, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538477278}, next_to_submit_scn_:{val:1729061852134673006, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538477166}, end_lsn:{lsn:14538477278}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061852134673005, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061852134673005, v:0}, data_checksum:1346344266, flag:1}}, init_mode_version:0, accumulate_checksum:2227248186, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061852134673005, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650124, miss_cnt:32, cache_read_size:8050406363, hit_ratio:"9.999994892271297786e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}, replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14538477222}, scn_:{val:1729061852134673005, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061852574033, first_handle_ts_:1729061852586925, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0ef5ffc370, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:32.587467] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=62] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060041826930000) [2024-10-16 14:57:32.587476] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14538477222}, scn_:{val:1729061852134673005, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061852574033, first_handle_ts_:1729061852587458, replay_cost_:16, retry_cost_:-1, read_log_buf_:0x2b0ef5ffc380, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061852586864, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538477278}, next_to_submit_scn_:{val:1729061852134673006, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538477166}, end_lsn:{lsn:14538477278}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061852134673005, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061852134673005, v:0}, data_checksum:1346344266, flag:1}}, init_mode_version:0, accumulate_checksum:2227248186, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061852134673005, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650124, miss_cnt:32, cache_read_size:8050406363, hit_ratio:"9.999994892271297786e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}, replay_queue_idx=17, ret=0) [2024-10-16 14:57:32.591389] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:32.591665] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41D9-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:32.591674] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41D9-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3211, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:32.597901] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061852596860000, start_id=1729061852596860000, end_id=1729061852596860001) [2024-10-16 14:57:32.599725] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.608115] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4796, mem_used=10398720) [2024-10-16 14:57:32.619857] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061852618212000, start_id=1729061852618212000, end_id=1729061852618212001) [2024-10-16 14:57:32.620347] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=41] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.620727] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] successfully write small sstable(ret=0, block_info={nested_size:327680, nested_offset:126976, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=454656, old_block=[]) [2024-10-16 14:57:32.620763] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0e931ea4e0, hex: 00D03922D8338909, int: 687137421530877952}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0e931ea440, hex: 60560F6700000000, int: 1729058400}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0e931ea4f0, hex: 78E2148A792601E8, int: -1729058478484037000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0e931ea4b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:6, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:326541, meta_block_size:232, last_macro_size:326773, macro_metas:0x2b0e931ea100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.620823] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=18] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764803730833408-0](offset=0,size=270,type=3,seq=0), buf:0x2b0f56f0b060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.620835] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), buf:0x2b0f56f0a560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764803730833408-0](offset=0,size=270,type=3,seq=0), buf:0x2b0f56f0b060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:28, data_column_cnt:6, data_column_checksums:[0, 0, 0, 0, 0, 0], row_count:7534, max_merged_trans_version:1729060098301862001, contain_uncommitted_row:false, occupy_size:324958, original_size:4070838, data_checksum:3794951257, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.620867] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0f278da060, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729060098301862001, v:0}}}, ref_cnt:0, upper_trans_version:1729060098301862001, timestamp:0}, this:0x2b0f278da060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:327680, nested_offset:126976, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:7534, occupy_size:324958, data_checksum:3794951257, max_merged_trans_version:1729060098301862001, upper_trans_version:1729060098301862001, filled_tx_scn:{val:1729060098301862001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:7534, occupy_size_:324958, original_size_:4070838, data_checksum_:3794951257, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:28, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060098301862001, max_merged_trans_version_:1729060098301862001, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060098301862001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f278da390, block_data:{buf:0x2b0f278da5c0, size:64, extra_buf:0x2b0f278da600, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764803730833408-0](offset=0,size=270,type=3,seq=0), orig_block_buf:0x2b0f278da490, block_data:{buf:0x2b0f278da490, size:270, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f278da5a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:126976, nested_size:327680}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.620917] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=48] finish set upper trans version(ret=0, key_={tablet_id:{id:209283}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729060098301862001, v:0}}}, meta=0x2b0f278da130, old_val=1729060098301862001, upper_trans_version=1729060098301862001, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:327680, nested_offset:126976, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:7534, occupy_size:324958, data_checksum:3794951257, max_merged_trans_version:1729060098301862001, upper_trans_version:1729060098301862001, filled_tx_scn:{val:1729060098301862001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.620928] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] successfully rebuild one sstable(ret=0, block_info={nested_size:327680, nested_offset:126976, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:209283}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729060098301862001, v:0}}}) [2024-10-16 14:57:32.621078] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:6, col_desc_array:[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:6, 2197175160, 2197175160, 0, 0, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:3, index_type:0, row_store_type:1, schema_version:1729008952878920, column_cnt:4, store_column_cnt:4, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:4, column_array:[cnt:4, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"binary", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TINYINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:4, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:6, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.621255] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=45] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.621353] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully write small sstable(ret=0, block_info={nested_size:49152, nested_offset:454656, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=503808, old_block=[]) [2024-10-16 14:57:32.621372] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36e044e0, hex: 00D03922D8338909, int: 687137421530877952}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36e04440, hex: 60560F6700000000, int: 1729058400}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36e044f0, hex: 3038F1F4CC2301E8, int: -1729061418743810000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f36e044b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:6, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:47852, meta_block_size:233, last_macro_size:48085, macro_metas:0x2b0f36e04100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.621408] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0f12cb9060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.621417] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), buf:0x2b0f12cb8560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0f12cb9060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:5, data_column_cnt:6, data_column_checksums:[0, 0, 0, 0, 0, 0], row_count:1274, max_merged_trans_version:1729061778265412002, contain_uncommitted_row:false, occupy_size:47423, original_size:689163, data_checksum:1308093999, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.621437] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0f278daa00, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, ref_cnt:0, upper_trans_version:1729061778265412002, timestamp:0}, this:0x2b0f278daa00, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:454656, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1274, occupy_size:47423, data_checksum:1308093999, max_merged_trans_version:1729061778265412002, upper_trans_version:1729061778265412002, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:1274, occupy_size_:47423, original_size_:689163, data_checksum_:1308093999, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:5, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061778265412002, max_merged_trans_version_:1729061778265412002, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778311878001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f278dad30, block_data:{buf:0x2b0f278daf60, size:64, extra_buf:0x2b0f278dafa0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f278dae30, block_data:{buf:0x2b0f278dae30, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f278daf40, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:454656, nested_size:49152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.621473] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] finish set upper trans version(ret=0, key_={tablet_id:{id:209283}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, meta=0x2b0f278daad0, old_val=1729061778265412002, upper_trans_version=1729061778265412002, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:454656, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1274, occupy_size:47423, data_checksum:1308093999, max_merged_trans_version:1729061778265412002, upper_trans_version:1729061778265412002, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.621483] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:49152, nested_offset:454656, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:209283}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}) [2024-10-16 14:57:32.621616] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:2, sstable_array:0x2b0ef96085c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:4, sstable_array:0x2b0ef9609b20, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:2, sstable_array:0x2b0ef96085c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:4, sstable_array:0x2b0ef9609b20, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f334689a0, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015160270278000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MAJOR 0 1729060581893153000 0 1729060581893153000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729060098301862001 1729060098301862001 1729015204449389004 1729060098301862001 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729060578364001001 1729060578364001001 1729060098301862001 1729060578365064001 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR 1729061363352338000 1729061363352338000 1729060578365064001 1729061413762206004 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061778265412002 1729061778265412002 1729061413762206004 1729061778311878001 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778311878001 9223372036854775807 3 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836144450006000 1728836144450006000 1 1729036755768426000 0 false }}) [2024-10-16 14:57:32.621806] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=27] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1002}, tablet_id={id:209283}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ef9608060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f33469ab0, capacity:1}, shared_data_block_info_arr:{cnt:6, arr:0x2b0f33469ad0, capacity:6}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.621827] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=18] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e664940a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686899969847296-0](offset=421888,size=821,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1002}, tablet_id:{id:209283}, data_tablet_id:{id:209283}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836144284549001, v:0}, start_scn:{val:1729015204449389004, v:0}, clog_checkpoint_scn:{val:1729061778311878001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729060581893153000, cur_report_version:1729060581893153000, data_checksum:2854395054, row_count:2535, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755768426000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729060581893153000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836144450006000, v:0}, create_commit_version:1728836144450006000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836144376720, space_usage:{shared_data_size:659456, data_size:0, shared_meta_size:16384, meta_size:0, occupy_bytes:659456}}, table_store_addr:{ptr:0x2b0e664946b0, addr:[7663857-6566-0-2305843987393167360-0](offset=671744,size=911,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843270133629839-0](offset=672655,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e66498bb1, addr:[7663823-15172-0-4611686379205063141-0](offset=422373,size=336,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0efb1f5220, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}, this={this:0x2b0ef9608060, is_inited:true, wash_score:1729061852621140000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1002}, tablet_id:{id:209283}, data_tablet_id:{id:209283}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836144284549001, v:0}, start_scn:{val:1729015204449389004, v:0}, clog_checkpoint_scn:{val:1729061778311878001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729060581893153000, cur_report_version:1729060581893153000, data_checksum:2854395054, row_count:2535, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755768426000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729060581893153000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836144450006000, v:0}, create_commit_version:1728836144450006000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836144376720, space_usage:{shared_data_size:12582912, data_size:0, shared_meta_size:4929, meta_size:0, occupy_bytes:659456}}, table_store_addr:{ptr:0x2b0ef9608450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0f334692e0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0f33469a20, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0efb1f5220, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.621918] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=61] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef96085e0, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ef96085e0, addr:[7663857-6566-0-2305844269787258880-0](offset=663552,size=1174,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:1675264, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:338, occupy_size:19068, data_checksum:55651879, max_merged_trans_version:1729015160270278000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:94, row_count_:338, occupy_size_:19068, original_size_:178464, data_checksum_:55651879, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:4, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008952878920, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015160270278000, max_merged_trans_version_:1729015160270278000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef96088f8, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0ef9608990, block_data:{buf:0x2b0ef9608928, size:104, extra_buf:0x2b0ef9608af6, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764972308299776-0](offset=0,size=427,type=3,seq=0), orig_block_buf:0x2b0ef9608ead, block_data:{buf:0x2b0ef9608ead, size:427, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9609058, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1675264, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1086, old_pos=0, pos=1086) [2024-10-16 14:57:32.621957] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef9609080, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729060581893153000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ef9609080, addr:[7663857-6566-0-2305844281598420118-0](offset=664726,size=1185,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:122880, nested_offset:1413120, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2535, occupy_size:120275, data_checksum:2854395054, max_merged_trans_version:1729060581893153000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:2535, occupy_size_:120275, original_size_:1338480, data_checksum_:2854395054, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:13, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008952878920, create_snapshot_version_:1729015204839740000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060581893153000, max_merged_trans_version_:1729060581893153000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef9609398, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0ef9609430, block_data:{buf:0x2b0ef96093c8, size:104, extra_buf:0x2b0ef9609596, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764977677008896-0](offset=0,size=432,type=3,seq=0), orig_block_buf:0x2b0ef960994d, block_data:{buf:0x2b0ef960994d, size:432, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9609afd, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1413120, nested_size:122880}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1097, old_pos=0, pos=1097) [2024-10-16 14:57:32.621987] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334e0050, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729060098301862001, v:0}}}, ref_cnt:0, upper_trans_version:1729060098301862001, timestamp:0}, this:0x2b0f334e0050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:327680, nested_offset:126976, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:7534, occupy_size:324958, data_checksum:3794951257, max_merged_trans_version:1729060098301862001, upper_trans_version:1729060098301862001, filled_tx_scn:{val:1729060098301862001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:7534, occupy_size_:324958, original_size_:4070838, data_checksum_:3794951257, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:28, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060098301862001, max_merged_trans_version_:1729060098301862001, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060098301862001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334e0368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f334e03a8, block_data:{buf:0x2b0f334e0368, size:64, extra_buf:0x2b0f334e04a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764803730833408-0](offset=0,size=270,type=3,seq=0), orig_block_buf:0x2b0f334e0877, block_data:{buf:0x2b0f334e0877, size:270, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334e0985, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:126976, nested_size:327680}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=794, old_pos=0, pos=794) [2024-10-16 14:57:32.622014] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334e09a0, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729060098301862001, v:0}, end_scn:{val:1729060578365064001, v:0}}}, ref_cnt:0, upper_trans_version:1729060578364001001, timestamp:0}, this:0x2b0f334e09a0, addr:[7663857-6566-0-2305843936927296681-0](offset=666793,size=864,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:53248, nested_offset:823296, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1495, occupy_size:49265, data_checksum:0, max_merged_trans_version:1729060578364001001, upper_trans_version:1729060578364001001, filled_tx_scn:{val:1729060578365064001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:87, row_count_:1495, occupy_size_:49265, original_size_:808503, data_checksum_:166832269, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:5, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060578364001001, max_merged_trans_version_:1729060578364001001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060578365064001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334e0cb8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f334e0cf8, block_data:{buf:0x2b0f334e0cb8, size:64, extra_buf:0x2b0f334e0df0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f334e11c7, block_data:{buf:0x2b0f334e11c7, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334e12cd, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:823296, nested_size:53248}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=776, old_pos=0, pos=776) [2024-10-16 14:57:32.622041] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334e12f0, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729060578365064001, v:0}, end_scn:{val:1729061413762206004, v:0}}}, ref_cnt:0, upper_trans_version:1729061363352338000, timestamp:0}, this:0x2b0f334e12f0, addr:[7663857-6566-0-2305843953033424905-0](offset=667657,size=879,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:81920, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2463, occupy_size:78983, data_checksum:0, max_merged_trans_version:1729061363352338000, upper_trans_version:1729061363352338000, filled_tx_scn:{val:1729061413762206004, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:96, row_count_:2463, occupy_size_:78983, original_size_:1331231, data_checksum_:3757155896, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:7, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061363352338000, max_merged_trans_version_:1729061363352338000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413762206004, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334e1608, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f334e1648, block_data:{buf:0x2b0f334e1608, size:64, extra_buf:0x2b0f334e1740, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764801583349760-0](offset=0,size=268,type=3,seq=0), orig_block_buf:0x2b0f334e1b17, block_data:{buf:0x2b0f334e1b17, size:268, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334e1c23, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:81920}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=791, old_pos=0, pos=791) [2024-10-16 14:57:32.622067] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f33468050, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, ref_cnt:0, upper_trans_version:1729061778265412002, timestamp:0}, this:0x2b0f33468050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:454656, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1274, occupy_size:47423, data_checksum:1308093999, max_merged_trans_version:1729061778265412002, upper_trans_version:1729061778265412002, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:1274, occupy_size_:47423, original_size_:689163, data_checksum_:1308093999, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:5, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061778265412002, max_merged_trans_version_:1729061778265412002, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778311878001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f33468368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f334683a8, block_data:{buf:0x2b0f33468368, size:64, extra_buf:0x2b0f334684a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f33468877, block_data:{buf:0x2b0f33468877, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f3346897d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:454656, nested_size:49152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=777, old_pos=0, pos=777) [2024-10-16 14:57:32.622092] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334689b0, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755768426000, v:0}}}, ref_cnt:0, upper_trans_version:1728836144450006000, timestamp:0}, this:0x2b0f334689b0, addr:[7663857-6566-0-2305843897198851801-0](offset=669401,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:630784, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836144450006000, upper_trans_version:1728836144450006000, filled_tx_scn:{val:1729036755768426000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:1744735392, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836144450006000, max_merged_trans_version_:1728836144450006000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755768426000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f33468cc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f33468d08, block_data:{buf:0x2b0f33468cc8, size:64, extra_buf:0x2b0f33468df5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f334691c4, block_data:{buf:0x2b0f334691c4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334692ba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:630784, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=739, old_pos=0, pos=739) [2024-10-16 14:57:32.622147] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.622280] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f334c4fc0, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f334c4fc0, addr:[7667511-10399-0-2305844269786759168-0](offset=163840,size=1174,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:1675264, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:338, occupy_size:19068, data_checksum:55651879, max_merged_trans_version:1729015160270278000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:94, row_count_:338, occupy_size_:19068, original_size_:178464, data_checksum_:55651879, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:4, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008952878920, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015160270278000, max_merged_trans_version_:1729015160270278000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334c52d8, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f334c5370, block_data:{buf:0x2b0f334c5308, size:104, extra_buf:0x2b0f334c54d6, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764972308299776-0](offset=0,size=427,type=3,seq=0), orig_block_buf:0x2b0f334c588d, block_data:{buf:0x2b0f334c588d, size:427, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334c5a38, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1675264, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.622312] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=31] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef960e050, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729060581893153000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ef960e050, addr:[7667511-10399-0-2305844281597920406-0](offset=165014,size=1185,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:122880, nested_offset:1413120, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2535, occupy_size:120275, data_checksum:2854395054, max_merged_trans_version:1729060581893153000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:2535, occupy_size_:120275, original_size_:1338480, data_checksum_:2854395054, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:13, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008952878920, create_snapshot_version_:1729015204839740000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060581893153000, max_merged_trans_version_:1729060581893153000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef960e368, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0ef960e400, block_data:{buf:0x2b0ef960e398, size:104, extra_buf:0x2b0ef960e566, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764977677008896-0](offset=0,size=432,type=3,seq=0), orig_block_buf:0x2b0ef960e91d, block_data:{buf:0x2b0ef960e91d, size:432, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef960eacd, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1413120, nested_size:122880}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.622336] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef960eb20, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729060098301862001, v:0}}}, ref_cnt:0, upper_trans_version:1729060098301862001, timestamp:0}, this:0x2b0ef960eb20, addr:[7667511-10399-0-2305843956254148919-0](offset=166199,size=882,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:327680, nested_offset:126976, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:7534, occupy_size:324958, data_checksum:3794951257, max_merged_trans_version:1729060098301862001, upper_trans_version:1729060098301862001, filled_tx_scn:{val:1729060098301862001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:7534, occupy_size_:324958, original_size_:4070838, data_checksum_:3794951257, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:28, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060098301862001, max_merged_trans_version_:1729060098301862001, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060098301862001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef960ee38, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ef960ee78, block_data:{buf:0x2b0ef960ee38, size:64, extra_buf:0x2b0ef960ef70, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764803730833408-0](offset=0,size=270,type=3,seq=0), orig_block_buf:0x2b0ef960f347, block_data:{buf:0x2b0ef960f347, size:270, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef960f455, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:126976, nested_size:327680}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.622361] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef960f470, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729060098301862001, v:0}, end_scn:{val:1729060578365064001, v:0}}}, ref_cnt:0, upper_trans_version:1729060578364001001, timestamp:0}, this:0x2b0ef960f470, addr:[7667511-10399-0-2305843936926796969-0](offset=167081,size=864,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:53248, nested_offset:823296, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1495, occupy_size:49265, data_checksum:0, max_merged_trans_version:1729060578364001001, upper_trans_version:1729060578364001001, filled_tx_scn:{val:1729060578365064001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:87, row_count_:1495, occupy_size_:49265, original_size_:808503, data_checksum_:166832269, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:5, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060578364001001, max_merged_trans_version_:1729060578364001001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060578365064001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef960f788, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ef960f7c8, block_data:{buf:0x2b0ef960f788, size:64, extra_buf:0x2b0ef960f8c0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0ef960fc97, block_data:{buf:0x2b0ef960fc97, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef960fd9d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:823296, nested_size:53248}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.622386] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0093a050, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729060578365064001, v:0}, end_scn:{val:1729061413762206004, v:0}}}, ref_cnt:0, upper_trans_version:1729061363352338000, timestamp:0}, this:0x2b0f0093a050, addr:[7667511-10399-0-2305843953032925193-0](offset=167945,size=879,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:81920, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2463, occupy_size:78983, data_checksum:0, max_merged_trans_version:1729061363352338000, upper_trans_version:1729061363352338000, filled_tx_scn:{val:1729061413762206004, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:96, row_count_:2463, occupy_size_:78983, original_size_:1331231, data_checksum_:3757155896, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:7, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061363352338000, max_merged_trans_version_:1729061363352338000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413762206004, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f0093a368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f0093a3a8, block_data:{buf:0x2b0f0093a368, size:64, extra_buf:0x2b0f0093a4a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764801583349760-0](offset=0,size=268,type=3,seq=0), orig_block_buf:0x2b0f0093a877, block_data:{buf:0x2b0f0093a877, size:268, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0093a983, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:81920}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.622410] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=23] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0093a9a0, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, ref_cnt:0, upper_trans_version:1729061778265412002, timestamp:0}, this:0x2b0f0093a9a0, addr:[7667511-10399-0-2305843938000540536-0](offset=168824,size=865,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:454656, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1274, occupy_size:47423, data_checksum:1308093999, max_merged_trans_version:1729061778265412002, upper_trans_version:1729061778265412002, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:1274, occupy_size_:47423, original_size_:689163, data_checksum_:1308093999, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:5, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061778265412002, max_merged_trans_version_:1729061778265412002, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778311878001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f0093acb8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f0093acf8, block_data:{buf:0x2b0f0093acb8, size:64, extra_buf:0x2b0f0093adf0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f0093b1c7, block_data:{buf:0x2b0f0093b1c7, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0093b2cd, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:454656, nested_size:49152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.622434] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=23] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0093b300, key:{tablet_id:{id:209283}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755768426000, v:0}}}, ref_cnt:0, upper_trans_version:1728836144450006000, timestamp:0}, this:0x2b0f0093b300, addr:[7667511-10399-0-2305843897198352089-0](offset=169689,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:630784, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836144450006000, upper_trans_version:1728836144450006000, filled_tx_scn:{val:1729036755768426000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:1744735392, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836144450006000, max_merged_trans_version_:1728836144450006000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755768426000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f0093b618, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f0093b658, block_data:{buf:0x2b0f0093b618, size:64, extra_buf:0x2b0f0093b745, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f0093bb14, block_data:{buf:0x2b0f0093bb14, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0093bc0a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:630784, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.622474] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.622679] INFO [STORAGE] deserialize (ob_tablet_table_store.cpp:153) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to deserialize table store(ret=0, ls_id={id:1002}, tablet_id={id:209283}, major_tables_={cnt:2, sstable_array:0x2b0f36e721d0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:4, sstable_array:0x2b0f36e72390, serialize_table_type:false, is_inited:true}, meta_major_tables_={cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, mds_sstables_={cnt:1, sstable_array:0x2b0f36e72700, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecd518, major_tables:{cnt:2, sstable_array:0x2b0f36e721d0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:4, sstable_array:0x2b0f36e72390, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f36e72700, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015160270278000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MAJOR 0 1729060581893153000 0 1729060581893153000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729060098301862001 1729060098301862001 1729015204449389004 1729060098301862001 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729060578364001001 1729060578364001001 1729060098301862001 1729060578365064001 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR 1729061363352338000 1729061363352338000 1729060578365064001 1729061413762206004 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061778265412002 1729061778265412002 1729061413762206004 1729061778311878001 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778311878001 9223372036854775807 4 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836144450006000 1728836144450006000 1 1729036755768426000 0 false }}) [2024-10-16 14:57:32.623079] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=30] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6a35a0a0, start_pos=19217, remain=46231) [2024-10-16 14:57:32.623158] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.623246] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1002}, tablet_id={id:209283}, table_store_addr_.addr_=[7667511-10399-0-2305843987392667648-0](offset=172032,size=911,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843270133130127-0](offset=172943,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686865610764288-0](offset=1077248,size=789,type=4,seq=0), this=0x2b0e6a35a0a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ec821e6a0, capacity:1}, shared_data_block_info_arr:{cnt:6, arr:0x2b0ec821e6c0, capacity:6}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.623477] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601367, offset=9616960}, end_cursor:ObLogCursor{file_id=192, log_id=47601368, offset=9617076}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c8c050, len:448, log_data_len:116, seq:47601367, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.623496] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=21] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686865610764288-0](offset=1077248,size=789,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6a35a0a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6a35a0a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686865610764288-0](offset=1077248,size=789,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1002}, tablet_id:{id:209283}, data_tablet_id:{id:209283}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836144284549001, v:0}, start_scn:{val:1729015204449389004, v:0}, clog_checkpoint_scn:{val:1729061778311878001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729060581893153000, cur_report_version:1729060581893153000, data_checksum:2854395054, row_count:2535, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755768426000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729060581893153000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836144450006000, v:0}, create_commit_version:1728836144450006000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836144376720, space_usage:{shared_data_size:659456, data_size:0, shared_meta_size:16384, meta_size:0, occupy_bytes:659456}}, table_store_addr:{ptr:0x2b0e6a35a6b0, addr:[7667511-10399-0-2305843987392667648-0](offset=172032,size=911,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843270133130127-0](offset=172943,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a35ebb1, addr:[7663823-15172-0-4611686344845980133-0](offset=1077733,size=304,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0efb1f5220, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.623547] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=39] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1002}, tablet_meta_.tablet_id_={id:209283}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0ef9608060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f33469ab0, capacity:1}, shared_data_block_info_arr:{cnt:6, arr:0x2b0f33469ad0, capacity:6}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.623562] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] reset tablet(this=0x2b0ef9608060, ls_id={id:1002}, tablet_id={id:209283}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.623770] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1002}, tablet_id:{id:49403}}) [2024-10-16 14:57:32.624062] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1}, tablet_id:{id:49403}}) [2024-10-16 14:57:32.624429] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:4, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:4, col_desc_array:[cnt:4, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:4, 2197175160, 2197175160, 0, 0], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:5, index_type:7, row_store_type:1, schema_version:1717560328798944, column_cnt:2, store_column_cnt:2, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:2, column_array:[cnt:2, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:2, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:4, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:4, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.624548] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=39] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.624777] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:503808, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=507904, old_block=[]) [2024-10-16 14:57:32.624796] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f177fe4e0, hex: 5868A54499240600, int: 1729090560551000}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f177fe440, hex: 8A9C2E0000000000, int: 3054730}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f177fe4f0, hex: E80D715F7D2301E8, int: -1729061760554431000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f177fe4b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:4, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:573, meta_block_size:228, last_macro_size:801, macro_metas:0x2b0f177fe100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.624840] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764790845931520-0](offset=0,size=258,type=3,seq=0), buf:0x2b0ec83c3060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.624850] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), buf:0x2b0ec83c2560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764790845931520-0](offset=0,size=258,type=3,seq=0), buf:0x2b0ec83c3060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:4, data_column_checksums:[0, 0, 0, 0], row_count:3, max_merged_trans_version:1729061760554431000, contain_uncommitted_row:false, occupy_size:386, original_size:154, data_checksum:2669409747, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.624885] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0f278de060, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0f278de060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:503808, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:386, data_checksum:2669409747, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:386, original_size_:154, data_checksum_:2669409747, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f278de390, block_data:{buf:0x2b0f278de5c0, size:64, extra_buf:0x2b0f278de600, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764790845931520-0](offset=0,size=258,type=3,seq=0), orig_block_buf:0x2b0f278de490, block_data:{buf:0x2b0f278de490, size:258, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f278de5a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:503808, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.624932] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=46] finish set upper trans version(ret=0, key_={tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0f278de130, old_val=1729061760554431000, upper_trans_version=1729061760554431000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:503808, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:386, data_checksum:2669409747, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.624942] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:503808, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.625095] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f36e765c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f36e77650, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f36e765c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f36e77650, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0ef977d2f0, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015200571456000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061400920852000 1729061400920852000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061760554431000 1729061760554431000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328873906000 1717560328873906000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.625246] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846992}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f36e76060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:3, arr:0x2b0ee4bd4560, capacity:3}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ee4bd45b0, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0ee4bd45d0, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.625263] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6b2940a0, is_inited:true, wash_score:1729061852019328000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686840914378752-0](offset=753664,size=766,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846992}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:1019890677, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798928, space_usage:{shared_data_size:20480, data_size:6291456, shared_meta_size:16384, meta_size:0, occupy_bytes:1155687}}, table_store_addr:{ptr:0x2b0e6b2946b0, addr:[7663857-6566-0-2305843630911791104-0](offset=1581056,size=579,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843227184865859-0](offset=1581635,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6b297463, addr:[7663823-15172-0-4611686319075852774-0](offset=754150,size=280,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f36e76060, is_inited:true, wash_score:1729061852624357000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846992}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:1019890677, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798928, space_usage:{shared_data_size:6291456, data_size:6291456, shared_meta_size:3535, meta_size:0, occupy_bytes:1155687}}, table_store_addr:{ptr:0x2b0f36e76450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0ef977dc30, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0ee4bd44d0, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.625340] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=55] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f36e765d0, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f36e765d0, addr:[7663857-6566-0-2305844966646611968-0](offset=1572864,size=1823,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:3, nested_size:2097152, nested_offset:0, total_macro_block_count:3, total_use_old_macro_block_count:2, row_count:90516, occupy_size:1135207, data_checksum:1019890677, max_merged_trans_version:1729015200571456000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:1135207, original_size_:2896512, data_checksum_:1019890677, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:3, data_micro_block_count_:74, use_old_macro_block_count_:2, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798928, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015200571456000, max_merged_trans_version_:1729015200571456000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e768e8, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458765151623184384-0](offset=0,size=594,type=3,seq=0), orig_block_buf:0x2b0f36e76970, block_data:{buf:0x2b0f36e76908, size:104, extra_buf:0x2b0f36e76bc2, extra_size:1831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765369592774656-0](offset=0,size=797,type=3,seq=0), orig_block_buf:0x2b0f36e772e9, block_data:{buf:0x2b0f36e772e9, size:797, extra_buf:null, extra_size:0, type:1}}, data_block_count:3, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e77606, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:0, nested_size:2097152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1735, old_pos=0, pos=1735) [2024-10-16 14:57:32.625379] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef977c050, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061400920852000, timestamp:0}, this:0x2b0ef977c050, addr:[7663857-6566-0-2305843953034331935-0](offset=1574687,size=879,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:12288, nested_offset:1040384, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:385, occupy_size:10433, data_checksum:0, max_merged_trans_version:1729061400920852000, upper_trans_version:1729061400920852000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:385, occupy_size_:10433, original_size_:19258, data_checksum_:3390967937, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061400920852000, max_merged_trans_version_:1729061400920852000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef977c368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ef977c3a8, block_data:{buf:0x2b0ef977c368, size:64, extra_buf:0x2b0ef977c4a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764802657091584-0](offset=0,size=269,type=3,seq=0), orig_block_buf:0x2b0ef977c877, block_data:{buf:0x2b0ef977c877, size:269, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef977c984, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1040384, nested_size:12288}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=791, old_pos=0, pos=791) [2024-10-16 14:57:32.625416] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef977c9a0, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0ef977c9a0, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:503808, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:386, data_checksum:2669409747, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:386, original_size_:154, data_checksum_:2669409747, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef977ccb8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ef977ccf8, block_data:{buf:0x2b0ef977ccb8, size:64, extra_buf:0x2b0ef977cdf0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764790845931520-0](offset=0,size=258,type=3,seq=0), orig_block_buf:0x2b0ef977d1c7, block_data:{buf:0x2b0ef977d1c7, size:258, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef977d2c9, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:503808, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=770, old_pos=0, pos=770) [2024-10-16 14:57:32.625444] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef977d300, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0ef977d300, addr:[7663857-6566-0-2305843903642209768-0](offset=1576424,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1384448, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ef977d618, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ef977d658, block_data:{buf:0x2b0ef977d618, size:64, extra_buf:0x2b0ef977d745, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0ef977db14, block_data:{buf:0x2b0ef977db14, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef977dc10, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1384448, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=745, old_pos=0, pos=745) [2024-10-16 14:57:32.625487] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.625616] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef7ed0980, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ef7ed0980, addr:[7667511-10399-0-2305844966645215232-0](offset=176128,size=1823,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:3, nested_size:2097152, nested_offset:0, total_macro_block_count:3, total_use_old_macro_block_count:2, row_count:90516, occupy_size:1135207, data_checksum:1019890677, max_merged_trans_version:1729015200571456000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:1135207, original_size_:2896512, data_checksum_:1019890677, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:3, data_micro_block_count_:74, use_old_macro_block_count_:2, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798928, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015200571456000, max_merged_trans_version_:1729015200571456000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef7ed0c98, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458765151623184384-0](offset=0,size=594,type=3,seq=0), orig_block_buf:0x2b0ef7ed0d20, block_data:{buf:0x2b0ef7ed0cb8, size:104, extra_buf:0x2b0ef7ed0f72, extra_size:1831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765369592774656-0](offset=0,size=797,type=3,seq=0), orig_block_buf:0x2b0ef7ed1699, block_data:{buf:0x2b0ef7ed1699, size:797, extra_buf:null, extra_size:0, type:1}}, data_block_count:3, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef7ed19b6, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:0, nested_size:2097152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.625647] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=31] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f177d6050, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061400920852000, timestamp:0}, this:0x2b0f177d6050, addr:[7667511-10399-0-2305843953032935199-0](offset=177951,size=879,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:12288, nested_offset:1040384, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:385, occupy_size:10433, data_checksum:0, max_merged_trans_version:1729061400920852000, upper_trans_version:1729061400920852000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:385, occupy_size_:10433, original_size_:19258, data_checksum_:3390967937, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061400920852000, max_merged_trans_version_:1729061400920852000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177d6368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f177d63a8, block_data:{buf:0x2b0f177d6368, size:64, extra_buf:0x2b0f177d64a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764802657091584-0](offset=0,size=269,type=3,seq=0), orig_block_buf:0x2b0f177d6877, block_data:{buf:0x2b0f177d6877, size:269, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177d6984, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1040384, nested_size:12288}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.625672] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f177d69a0, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0f177d69a0, addr:[7667511-10399-0-2305843930484357774-0](offset=178830,size=858,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:503808, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:386, data_checksum:2669409747, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:386, original_size_:154, data_checksum_:2669409747, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177d6cb8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f177d6cf8, block_data:{buf:0x2b0f177d6cb8, size:64, extra_buf:0x2b0f177d6df0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764790845931520-0](offset=0,size=258,type=3,seq=0), orig_block_buf:0x2b0f177d71c7, block_data:{buf:0x2b0f177d71c7, size:258, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177d72c9, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:503808, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.625697] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f177d7300, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0f177d7300, addr:[7667511-10399-0-2305843903640813032-0](offset=179688,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1384448, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f177d7618, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f177d7658, block_data:{buf:0x2b0f177d7618, size:64, extra_buf:0x2b0f177d7745, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0f177d7b14, block_data:{buf:0x2b0f177d7b14, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177d7c10, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1384448, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.625735] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=23] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.626087] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6aa4c0a0, start_pos=13251, remain=52197) [2024-10-16 14:57:32.626158] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.626242] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846992}, table_store_addr_.addr_=[7667511-10399-0-2305843630910394368-0](offset=184320,size=579,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843227183469123-0](offset=184899,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686840914706432-0](offset=1081344,size=766,type=4,seq=0), this=0x2b0e6aa4c0a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:3, arr:0x2b0ef97862f0, capacity:3}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ef9786340, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0ef9786360, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.627574] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601368, offset=9617076}, end_cursor:ObLogCursor{file_id=192, log_id=47601369, offset=9617192}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d24050, len:332, log_data_len:116, seq:47601368, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.627595] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=22] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686840914706432-0](offset=1081344,size=766,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6aa4c0a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6aa4c0a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686840914706432-0](offset=1081344,size=766,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846992}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:1019890677, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798928, space_usage:{shared_data_size:20480, data_size:6291456, shared_meta_size:16384, meta_size:0, occupy_bytes:1155687}}, table_store_addr:{ptr:0x2b0e6aa4c6b0, addr:[7667511-10399-0-2305843630910394368-0](offset=184320,size=579,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843227183469123-0](offset=184899,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6aa4f463, addr:[7663823-15172-0-4611686319076180454-0](offset=1081830,size=280,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.627674] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=65] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846992}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f36e76060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:3, arr:0x2b0ee4bd4560, capacity:3}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ee4bd45b0, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0ee4bd45d0, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.627690] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] reset tablet(this=0x2b0f36e76060, ls_id={id:1001}, tablet_id={id:1152921504606846992}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.627980] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1001}, tablet_id:{id:49403}}) [2024-10-16 14:57:32.628162] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:6, col_desc_array:[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:6, 2197175160, 2197175160, 0, 0, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:3, index_type:0, row_store_type:1, schema_version:1729008952878920, column_cnt:4, store_column_cnt:4, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:17, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:4, column_array:[cnt:4, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"binary", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TINYINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:4, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:6, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:6, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.628335] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=46] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.628435] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully write small sstable(ret=0, block_info={nested_size:49152, nested_offset:507904, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=557056, old_block=[]) [2024-10-16 14:57:32.628454] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0f1773c4e0, hex: 00D0F97CFB408009, int: 684618592238030848}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f1773c440, hex: 60560F6700000000, int: 1729058400}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f1773c4f0, hex: 97A8F40ECD2301E8, int: -1729061418307377001}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f1773c4b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:6, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:45728, meta_block_size:228, last_macro_size:45956, macro_metas:0x2b0f1773c100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.628489] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0f334c5060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.628498] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), buf:0x2b0f334c4560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), buf:0x2b0f334c5060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:4, data_column_cnt:6, data_column_checksums:[0, 0, 0, 0, 0, 0], row_count:1218, max_merged_trans_version:1729061778311878001, contain_uncommitted_row:false, occupy_size:45375, original_size:658863, data_checksum:360298468, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.628518] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0eda24a060, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, ref_cnt:0, upper_trans_version:1729061778311878001, timestamp:0}, this:0x2b0eda24a060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:507904, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1218, occupy_size:45375, data_checksum:360298468, max_merged_trans_version:1729061778311878001, upper_trans_version:1729061778311878001, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:1218, occupy_size_:45375, original_size_:658863, data_checksum_:360298468, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:4, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061778311878001, max_merged_trans_version_:1729061778311878001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778311878001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0eda24a390, block_data:{buf:0x2b0eda24a5c0, size:64, extra_buf:0x2b0eda24a600, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0eda24a490, block_data:{buf:0x2b0eda24a490, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0eda24a5a0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:507904, nested_size:49152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.628554] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] finish set upper trans version(ret=0, key_={tablet_id:{id:209289}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, meta=0x2b0eda24a130, old_val=1729061778311878001, upper_trans_version=1729061778311878001, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:507904, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1218, occupy_size:45375, data_checksum:360298468, max_merged_trans_version:1729061778311878001, upper_trans_version:1729061778311878001, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.628563] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:49152, nested_offset:507904, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:209289}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}) [2024-10-16 14:57:32.628686] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:2, sstable_array:0x2b0f334e05c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:4, sstable_array:0x2b0f334e1b20, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:2, sstable_array:0x2b0f334e05c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:4, sstable_array:0x2b0f334e1b20, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0ea17fe9a0, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015160273410000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MAJOR 0 1729060581959460000 0 1729060581959460000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729060098249225000 1729060098249225000 1729015204449389004 1729060098249225000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729060578365064001 1729060578365064001 1729060098249225000 1729060578365064001 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR 1729061363431489000 1729061363431489000 1729060578365064001 1729061413762206004 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061778311878001 1729061778311878001 1729061413762206004 1729061778311878001 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778311878001 9223372036854775807 3 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836144450006000 1728836144450006000 1 1729036755768426000 0 false }}) [2024-10-16 14:57:32.628840] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1002}, tablet_id={id:209289}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f334e0060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ea17ffab0, capacity:1}, shared_data_block_info_arr:{cnt:7, arr:0x2b0ea17ffad0, capacity:7}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.628857] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e664b80a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686899970191360-0](offset=765952,size=821,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1002}, tablet_id:{id:209289}, data_tablet_id:{id:209289}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836144284549001, v:0}, start_scn:{val:1729015204449389004, v:0}, clog_checkpoint_scn:{val:1729061778311878001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729060581959460000, cur_report_version:1729060581959460000, data_checksum:1051322377, row_count:2415, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755768426000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729060581959460000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836144450006000, v:0}, create_commit_version:1728836144450006000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836144376720, space_usage:{shared_data_size:671744, data_size:0, shared_meta_size:16384, meta_size:0, occupy_bytes:671744}}, table_store_addr:{ptr:0x2b0e664b86b0, addr:[7663857-6566-0-2305843989541597184-0](offset=1617920,size=913,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843270134576017-0](offset=1618833,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e664bcbb1, addr:[7663823-15172-0-4611686379205407205-0](offset=766437,size=336,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0efb1f6a50, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}, this={this:0x2b0f334e0060, is_inited:true, wash_score:1729061852628067000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1002}, tablet_id:{id:209289}, data_tablet_id:{id:209289}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836144284549001, v:0}, start_scn:{val:1729015204449389004, v:0}, clog_checkpoint_scn:{val:1729061778311878001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729060581959460000, cur_report_version:1729060581959460000, data_checksum:1051322377, row_count:2415, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755768426000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729060581959460000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836144450006000, v:0}, create_commit_version:1728836144450006000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836144376720, space_usage:{shared_data_size:14680064, data_size:0, shared_meta_size:5812, meta_size:0, occupy_bytes:671744}}, table_store_addr:{ptr:0x2b0f334e0450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0ea17ff2e0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0ea17ffa20, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0efb1f6a50, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.628929] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=54] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334e05e0, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f334e05e0, addr:[7663857-6566-0-2305844270861946880-0](offset=1609728,size=1175,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:24576, nested_offset:1794048, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:322, occupy_size:20484, data_checksum:2092762268, max_merged_trans_version:1729015160273410000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:322, occupy_size_:20484, original_size_:170016, data_checksum_:2092762268, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008952878920, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015160273410000, max_merged_trans_version_:1729015160273410000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334e08f8, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f334e0990, block_data:{buf:0x2b0f334e0928, size:104, extra_buf:0x2b0f334e0af6, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764972308299776-0](offset=0,size=427,type=3,seq=0), orig_block_buf:0x2b0f334e0ead, block_data:{buf:0x2b0f334e0ead, size:427, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334e1058, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1794048, nested_size:24576}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1087, old_pos=0, pos=1087) [2024-10-16 14:57:32.628965] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f334e1080, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729060581959460000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f334e1080, addr:[7663857-6566-0-2305844281599366295-0](offset=1610903,size=1185,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:139264, nested_offset:1273856, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2415, occupy_size:134603, data_checksum:1051322377, max_merged_trans_version:1729060581959460000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:2415, occupy_size_:134603, original_size_:1275120, data_checksum_:1051322377, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:12, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008952878920, create_snapshot_version_:1729015204839740000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060581959460000, max_merged_trans_version_:1729060581959460000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f334e1398, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f334e1430, block_data:{buf:0x2b0f334e13c8, size:104, extra_buf:0x2b0f334e1596, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764977677008896-0](offset=0,size=432,type=3,seq=0), orig_block_buf:0x2b0f334e194d, block_data:{buf:0x2b0f334e194d, size:432, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f334e1afd, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1273856, nested_size:139264}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1097, old_pos=0, pos=1097) [2024-10-16 14:57:32.628994] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=27] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f177ae050, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729060098249225000, v:0}}}, ref_cnt:0, upper_trans_version:1729060098249225000, timestamp:0}, this:0x2b0f177ae050, addr:[7663857-6566-0-2305843956255594808-0](offset=1612088,size=882,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:327680, nested_offset:1146880, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:7183, occupy_size:324276, data_checksum:0, max_merged_trans_version:1729060098249225000, upper_trans_version:1729060098249225000, filled_tx_scn:{val:1729060098249225000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:7183, occupy_size_:324276, original_size_:3881174, data_checksum_:3650786842, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:25, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060098249225000, max_merged_trans_version_:1729060098249225000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060098249225000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177ae368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f177ae3a8, block_data:{buf:0x2b0f177ae368, size:64, extra_buf:0x2b0f177ae4a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764803730833408-0](offset=0,size=270,type=3,seq=0), orig_block_buf:0x2b0f177ae877, block_data:{buf:0x2b0f177ae877, size:270, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177ae985, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1146880, nested_size:327680}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=794, old_pos=0, pos=794) [2024-10-16 14:57:32.629021] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f177ae9a0, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729060098249225000, v:0}, end_scn:{val:1729060578365064001, v:0}}}, ref_cnt:0, upper_trans_version:1729060578365064001, timestamp:0}, this:0x2b0f177ae9a0, addr:[7663857-6566-0-2305843938001984682-0](offset=1612970,size=865,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:876544, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1422, occupy_size:45643, data_checksum:0, max_merged_trans_version:1729060578365064001, upper_trans_version:1729060578365064001, filled_tx_scn:{val:1729060578365064001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:1422, occupy_size_:45643, original_size_:769023, data_checksum_:2409124866, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:4, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060578365064001, max_merged_trans_version_:1729060578365064001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060578365064001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177aecb8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f177aecf8, block_data:{buf:0x2b0f177aecb8, size:64, extra_buf:0x2b0f177aedf0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f177af1c7, block_data:{buf:0x2b0f177af1c7, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177af2cd, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:876544, nested_size:49152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=777, old_pos=0, pos=777) [2024-10-16 14:57:32.629047] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f177af2f0, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729060578365064001, v:0}, end_scn:{val:1729061413762206004, v:0}}}, ref_cnt:0, upper_trans_version:1729061363431489000, timestamp:0}, this:0x2b0f177af2f0, addr:[7663857-6566-0-2305843951960629259-0](offset=1613835,size=878,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:77824, nested_offset:688128, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2343, occupy_size:73863, data_checksum:0, max_merged_trans_version:1729061363431489000, upper_trans_version:1729061363431489000, filled_tx_scn:{val:1729061413762206004, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:2343, occupy_size_:73863, original_size_:1266371, data_checksum_:111003430, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:6, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061363431489000, max_merged_trans_version_:1729061363431489000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413762206004, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177af608, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f177af648, block_data:{buf:0x2b0f177af608, size:64, extra_buf:0x2b0f177af740, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764801583349760-0](offset=0,size=268,type=3,seq=0), orig_block_buf:0x2b0f177afb17, block_data:{buf:0x2b0f177afb17, size:268, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177afc23, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:688128, nested_size:77824}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=790, old_pos=0, pos=790) [2024-10-16 14:57:32.629073] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea17fe050, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, ref_cnt:0, upper_trans_version:1729061778311878001, timestamp:0}, this:0x2b0ea17fe050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:507904, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1218, occupy_size:45375, data_checksum:360298468, max_merged_trans_version:1729061778311878001, upper_trans_version:1729061778311878001, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:1218, occupy_size_:45375, original_size_:658863, data_checksum_:360298468, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:4, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061778311878001, max_merged_trans_version_:1729061778311878001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778311878001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ea17fe368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0ea17fe3a8, block_data:{buf:0x2b0ea17fe368, size:64, extra_buf:0x2b0ea17fe4a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0ea17fe877, block_data:{buf:0x2b0ea17fe877, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea17fe97d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:507904, nested_size:49152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=777, old_pos=0, pos=777) [2024-10-16 14:57:32.629099] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ea17fe9b0, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755768426000, v:0}}}, ref_cnt:0, upper_trans_version:1728836144450006000, timestamp:0}, this:0x2b0ea17fe9b0, addr:[7663857-6566-0-2305843897199797978-0](offset=1615578,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1318912, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836144450006000, upper_trans_version:1728836144450006000, filled_tx_scn:{val:1729036755768426000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:1744735392, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836144450006000, max_merged_trans_version_:1728836144450006000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755768426000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ea17fecc8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ea17fed08, block_data:{buf:0x2b0ea17fecc8, size:64, extra_buf:0x2b0ea17fedf5, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0ea17ff1c4, block_data:{buf:0x2b0ea17ff1c4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ea17ff2ba, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1318912, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=739, old_pos=0, pos=739) [2024-10-16 14:57:32.629153] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=31] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.629377] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e72fc0, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f36e72fc0, addr:[7667511-10399-0-2305844270860525568-0](offset=188416,size=1175,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:24576, nested_offset:1794048, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:322, occupy_size:20484, data_checksum:2092762268, max_merged_trans_version:1729015160273410000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:322, occupy_size_:20484, original_size_:170016, data_checksum_:2092762268, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008952878920, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015160273410000, max_merged_trans_version_:1729015160273410000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e732d8, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f36e73370, block_data:{buf:0x2b0f36e73308, size:104, extra_buf:0x2b0f36e734d6, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764972308299776-0](offset=0,size=427,type=3,seq=0), orig_block_buf:0x2b0f36e7388d, block_data:{buf:0x2b0f36e7388d, size:427, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e73a38, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1794048, nested_size:24576}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.629412] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f33468050, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729060581959460000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f33468050, addr:[7667511-10399-0-2305844281597944983-0](offset=189591,size=1185,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:139264, nested_offset:1273856, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2415, occupy_size:134603, data_checksum:1051322377, max_merged_trans_version:1729060581959460000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:2415, occupy_size_:134603, original_size_:1275120, data_checksum_:1051322377, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:12, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1729008952878920, create_snapshot_version_:1729015204839740000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060581959460000, max_merged_trans_version_:1729060581959460000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f33468368, column_checksum_count:6, data_root_info:{addr:[0-0-0-3458764898220113920-0](offset=0,size=358,type=3,seq=0), orig_block_buf:0x2b0f33468400, block_data:{buf:0x2b0f33468398, size:104, extra_buf:0x2b0f33468566, extra_size:951, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764977677008896-0](offset=0,size=432,type=3,seq=0), orig_block_buf:0x2b0f3346891d, block_data:{buf:0x2b0f3346891d, size:432, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f33468acd, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1273856, nested_size:139264}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.629437] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f33468b20, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204449389004, v:0}, end_scn:{val:1729060098249225000, v:0}}}, ref_cnt:0, upper_trans_version:1729060098249225000, timestamp:0}, this:0x2b0f33468b20, addr:[7667511-10399-0-2305843956254173496-0](offset=190776,size=882,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:327680, nested_offset:1146880, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:7183, occupy_size:324276, data_checksum:0, max_merged_trans_version:1729060098249225000, upper_trans_version:1729060098249225000, filled_tx_scn:{val:1729060098249225000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:7183, occupy_size_:324276, original_size_:3881174, data_checksum_:3650786842, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:25, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060098249225000, max_merged_trans_version_:1729060098249225000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060098249225000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f33468e38, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f33468e78, block_data:{buf:0x2b0f33468e38, size:64, extra_buf:0x2b0f33468f70, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764803730833408-0](offset=0,size=270,type=3,seq=0), orig_block_buf:0x2b0f33469347, block_data:{buf:0x2b0f33469347, size:270, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f33469455, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1146880, nested_size:327680}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.629461] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f33469470, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729060098249225000, v:0}, end_scn:{val:1729060578365064001, v:0}}}, ref_cnt:0, upper_trans_version:1729060578365064001, timestamp:0}, this:0x2b0f33469470, addr:[7667511-10399-0-2305843938000563370-0](offset=191658,size=865,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:876544, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1422, occupy_size:45643, data_checksum:0, max_merged_trans_version:1729060578365064001, upper_trans_version:1729060578365064001, filled_tx_scn:{val:1729060578365064001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:1422, occupy_size_:45643, original_size_:769023, data_checksum_:2409124866, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:4, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060578365064001, max_merged_trans_version_:1729060578365064001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060578365064001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f33469788, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f334697c8, block_data:{buf:0x2b0f33469788, size:64, extra_buf:0x2b0f334698c0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f33469c97, block_data:{buf:0x2b0f33469c97, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f33469d9d, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:876544, nested_size:49152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.629485] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f283a0050, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729060578365064001, v:0}, end_scn:{val:1729061413762206004, v:0}}}, ref_cnt:0, upper_trans_version:1729061363431489000, timestamp:0}, this:0x2b0f283a0050, addr:[7667511-10399-0-2305843951959207947-0](offset=192523,size=878,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:77824, nested_offset:688128, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2343, occupy_size:73863, data_checksum:0, max_merged_trans_version:1729061363431489000, upper_trans_version:1729061363431489000, filled_tx_scn:{val:1729061413762206004, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:2343, occupy_size_:73863, original_size_:1266371, data_checksum_:111003430, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:6, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061363431489000, max_merged_trans_version_:1729061363431489000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061413762206004, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f283a0368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f283a03a8, block_data:{buf:0x2b0f283a0368, size:64, extra_buf:0x2b0f283a04a0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764801583349760-0](offset=0,size=268,type=3,seq=0), orig_block_buf:0x2b0f283a0877, block_data:{buf:0x2b0f283a0877, size:268, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f283a0983, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:688128, nested_size:77824}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.629509] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f283a09a0, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061413762206004, v:0}, end_scn:{val:1729061778311878001, v:0}}}, ref_cnt:0, upper_trans_version:1729061778311878001, timestamp:0}, this:0x2b0f283a09a0, addr:[7667511-10399-0-2305843938000565113-0](offset=193401,size=865,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:49152, nested_offset:507904, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1218, occupy_size:45375, data_checksum:360298468, max_merged_trans_version:1729061778311878001, upper_trans_version:1729061778311878001, filled_tx_scn:{val:1729061778311878001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:1218, occupy_size_:45375, original_size_:658863, data_checksum_:360298468, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:4, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061778311878001, max_merged_trans_version_:1729061778311878001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778311878001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f283a0cb8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764780108513280-0](offset=0,size=248,type=3,seq=0), orig_block_buf:0x2b0f283a0cf8, block_data:{buf:0x2b0f283a0cb8, size:64, extra_buf:0x2b0f283a0df0, extra_size:983, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764795140898816-0](offset=0,size=262,type=3,seq=0), orig_block_buf:0x2b0f283a11c7, block_data:{buf:0x2b0f283a11c7, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f283a12cd, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:507904, nested_size:49152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.629533] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f283a1300, key:{tablet_id:{id:209289}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755768426000, v:0}}}, ref_cnt:0, upper_trans_version:1728836144450006000, timestamp:0}, this:0x2b0f283a1300, addr:[7667511-10399-0-2305843897198376666-0](offset=194266,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1318912, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:484, data_checksum:0, max_merged_trans_version:1728836144450006000, upper_trans_version:1728836144450006000, filled_tx_scn:{val:1729036755768426000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:484, original_size_:187, data_checksum_:1744735392, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1728836144450006000, max_merged_trans_version_:1728836144450006000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755768426000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f283a1618, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f283a1658, block_data:{buf:0x2b0f283a1618, size:64, extra_buf:0x2b0f283a1745, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f283a1b14, block_data:{buf:0x2b0f283a1b14, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f283a1c0a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1318912, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.629571] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=23] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.629735] INFO [STORAGE] deserialize (ob_tablet_table_store.cpp:153) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to deserialize table store(ret=0, ls_id={id:1002}, tablet_id={id:209289}, major_tables_={cnt:2, sstable_array:0x2b0f009fc1d0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:4, sstable_array:0x2b0f009fc390, serialize_table_type:false, is_inited:true}, meta_major_tables_={cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, mds_sstables_={cnt:1, sstable_array:0x2b0f009fc700, serialize_table_type:false, is_inited:true}, memtables_={count:1}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecd518, major_tables:{cnt:2, sstable_array:0x2b0f009fc1d0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:4, sstable_array:0x2b0f009fc390, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f009fc700, serialize_table_type:false, is_inited:true}, memtables:{count:1}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015160273410000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MAJOR 0 1729060581959460000 0 1729060581959460000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729060098249225000 1729060098249225000 1729015204449389004 1729060098249225000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729060578365064001 1729060578365064001 1729060098249225000 1729060578365064001 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR 1729061363431489000 1729061363431489000 1729060578365064001 1729061413762206004 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061778311878001 1729061778311878001 1729061413762206004 1729061778311878001 0 false [22145] [ Y0-0000000000000000-0-0 ] MEM ACTIVE 9223372036854775807 18446744073709547609 1729061778311878001 9223372036854775807 4 unused [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1728836144450006000 1728836144450006000 1 1729036755768426000 0 false }}) [2024-10-16 14:57:32.630028] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=27] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6b1d80a0, start_pos=19217, remain=46231) [2024-10-16 14:57:32.630102] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.630194] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1002}, tablet_id={id:209289}, table_store_addr_.addr_=[7667511-10399-0-2305843989540175872-0](offset=196608,size=913,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843270133154705-0](offset=197521,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686899970510848-0](offset=1085440,size=821,type=4,seq=0), this=0x2b0e6b1d80a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f0bdf46a0, capacity:1}, shared_data_block_info_arr:{cnt:7, arr:0x2b0f0bdf46c0, capacity:7}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.630435] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601369, offset=9617192}, end_cursor:ObLogCursor{file_id=192, log_id=47601370, offset=9617308}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d98050, len:216, log_data_len:116, seq:47601369, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.630453] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686899970510848-0](offset=1085440,size=821,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6b1d80a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6b1d80a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686899970510848-0](offset=1085440,size=821,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1002}, tablet_id:{id:209289}, data_tablet_id:{id:209289}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:1728836144284549001, v:0}, start_scn:{val:1729015204449389004, v:0}, clog_checkpoint_scn:{val:1729061778311878001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778250505001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729060581959460000, cur_report_version:1729060581959460000, data_checksum:1051322377, row_count:2415, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755768426000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:4097, compat:1, last_compaction_type:0, wait_check_flag:1, last_medium_scn:1729060581959460000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1728836144450006000, v:0}, create_commit_version:1728836144450006000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1728836144376720, space_usage:{shared_data_size:671744, data_size:0, shared_meta_size:16384, meta_size:0, occupy_bytes:671744}}, table_store_addr:{ptr:0x2b0e6b1d86b0, addr:[7667511-10399-0-2305843989540175872-0](offset=196608,size=913,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843270133154705-0](offset=197521,size=243,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6b1dcbb1, addr:[7663823-15172-0-4611686379205726693-0](offset=1085925,size=336,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[0x2b0efb1f6a50, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:1}) [2024-10-16 14:57:32.630501] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=37] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1002}, tablet_meta_.tablet_id_={id:209289}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f334e0060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ea17ffab0, capacity:1}, shared_data_block_info_arr:{cnt:7, arr:0x2b0ea17ffad0, capacity:7}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.630517] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] reset tablet(this=0x2b0f334e0060, ls_id={id:1002}, tablet_id={id:209289}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.630689] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:20, rowkey_column_count:3, schema_rowkey_col_cnt:1, full_stored_col_cnt:20, col_desc_array:[cnt:20, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=28 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=33 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:20, 2197175160, 0, 0, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:3, index_type:0, row_store_type:1, schema_version:1717560328798944, column_cnt:18, store_column_cnt:18, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:1, rowkey_array:[cnt:1, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:18, column_array:[cnt:18, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"binary", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:18, rowkey_column_cnt:3, schema_rowkey_column_cnt:1, column_cnt:20, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:20, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=28 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=33 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0]) [2024-10-16 14:57:32.630858] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=83] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.630923] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:557056, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=561152, old_block=[]) [2024-10-16 14:57:32.630939] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:3, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0ef9786408, hex: 8A9C2E0000000000, int: 3054730}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0ef97864b0, hex: E80D715F7D2301E8, int: -1729061760554431000}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0ef9786478, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:20, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:1543, meta_block_size:218, last_macro_size:1761, macro_metas:0x2b0ef9786100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.630972] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764791919673344-0](offset=0,size=259,type=3,seq=0), buf:0x2b0e84885050, height:1, is_meta_root:true}) [2024-10-16 14:57:32.630980] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), buf:0x2b0e84884560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764791919673344-0](offset=0,size=259,type=3,seq=0), buf:0x2b0e84885050, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:20, data_column_checksums:[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], row_count:12, max_merged_trans_version:1729061761547003000, contain_uncommitted_row:false, occupy_size:1362, original_size:5305, data_checksum:1961330692, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.631005] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=16] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0ef9082060, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0ef9082060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:557056, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:1362, data_checksum:1961330692, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:12, occupy_size_:1362, original_size_:5305, data_checksum_:1961330692, index_type_:0, rowkey_column_count_:3, column_cnt_:20, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0ef9082390, block_data:{buf:0x2b0ef90825b0, size:64, extra_buf:0x2b0ef90825f0, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764791919673344-0](offset=0,size=259,type=3,seq=0), orig_block_buf:0x2b0ef9082480, block_data:{buf:0x2b0ef9082480, size:259, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9082590, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:557056, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.631040] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] finish set upper trans version(ret=0, key_={tablet_id:{id:200067}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0ef9082130, old_val=1729061761547003000, upper_trans_version=1729061761547003000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:557056, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:1362, data_checksum:1961330692, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.631050] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:557056, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:200067}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.631153] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0e931ea5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0e931eba00, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0e931ea5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0e931eba00, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0ef7fed1b0, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015201157706000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061401350833000 1729061401350833000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061761547003000 1729061761547003000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328401377000 1717560328401377000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.631287] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=18] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:200067}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0e931ea060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:4, arr:0x2b0e8487caa0, capacity:4}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8487cb10, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e8487cb30, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.631302] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6a2b80a0, is_inited:true, wash_score:1729061852526526000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686860241424384-0](offset=446464,size=784,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:200067}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:1326568258, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332152, space_usage:{shared_data_size:36864, data_size:8388608, shared_meta_size:16384, meta_size:0, occupy_bytes:3061010}}, table_store_addr:{ptr:0x2b0e6a2b868c, addr:[7663857-6566-0-2305843634132156416-0](offset=720896,size=582,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843564338938438-0](offset=721478,size=517,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a2bb6a2, addr:[7663823-15172-0-4611686344845349344-0](offset=446944,size=304,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0e931ea060, is_inited:true, wash_score:1729061852630281000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:200067}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:1326568258, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332152, space_usage:{shared_data_size:6291456, data_size:8388608, shared_meta_size:4325, meta_size:0, occupy_bytes:3061010}}, table_store_addr:{ptr:0x2b0e931ea450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0ef7fedaf0, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0e8487ca10, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.631398] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=78] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0e931ea5d0, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0e931ea5d0, addr:[7663857-6566-0-2305845828860420096-0](offset=696320,size=2626,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:4, nested_size:2097152, nested_offset:0, total_macro_block_count:4, total_use_old_macro_block_count:3, row_count:90523, occupy_size:3024146, data_checksum:1326568258, max_merged_trans_version:1729015201157706000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:99, row_count_:90523, occupy_size_:3024146, original_size_:49661578, data_checksum_:1326568258, index_type_:0, rowkey_column_count_:3, column_cnt_:20, data_macro_block_count_:4, data_micro_block_count_:225, use_old_macro_block_count_:3, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328332192, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201157706000, max_merged_trans_version_:1729015201157706000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0e931ea8e8, column_checksum_count:20, data_root_info:{addr:[0-0-0-3458765233227563008-0](offset=0,size=670,type=3,seq=0), orig_block_buf:0x2b0e931ea9e8, block_data:{buf:0x2b0e931ea988, size:96, extra_buf:0x2b0e931eac86, extra_size:1959, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458766002026708992-0](offset=0,size=1386,type=3,seq=0), orig_block_buf:0x2b0e931eb42d, block_data:{buf:0x2b0e931eb42d, size:1386, extra_buf:null, extra_size:0, type:1}}, data_block_count:4, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0e931eb997, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:0, nested_size:2097152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=2538, old_pos=0, pos=2538) [2024-10-16 14:57:32.631436] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef7fec050, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401350833000, timestamp:0}, this:0x2b0ef7fec050, addr:[7663857-6566-0-2305843945517263426-0](offset=698946,size=872,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:28672, nested_offset:90112, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:430, occupy_size:26401, data_checksum:0, max_merged_trans_version:1729061401350833000, upper_trans_version:1729061401350833000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:430, occupy_size_:26401, original_size_:220397, data_checksum_:226929565, index_type_:0, rowkey_column_count_:3, column_cnt_:20, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401350833000, max_merged_trans_version_:1729061401350833000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef7fec368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0ef7fec3a8, block_data:{buf:0x2b0ef7fec368, size:64, extra_buf:0x2b0ef7fec496, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0ef7fec7d5, block_data:{buf:0x2b0ef7fec7d5, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef7fec8e5, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:90112, nested_size:28672}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=784, old_pos=0, pos=784) [2024-10-16 14:57:32.631479] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=40] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef7fec900, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0ef7fec900, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:557056, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:1362, data_checksum:1961330692, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:12, occupy_size_:1362, original_size_:5305, data_checksum_:1961330692, index_type_:0, rowkey_column_count_:3, column_cnt_:20, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef7fecc18, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0ef7fecc58, block_data:{buf:0x2b0ef7fecc18, size:64, extra_buf:0x2b0ef7fecd46, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764791919673344-0](offset=0,size=259,type=3,seq=0), orig_block_buf:0x2b0ef7fed085, block_data:{buf:0x2b0ef7fed085, size:259, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef7fed188, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:557056, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=761, old_pos=0, pos=761) [2024-10-16 14:57:32.631507] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0ef7fed1c0, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328401377000, timestamp:0}, this:0x2b0ef7fed1c0, addr:[7663857-6566-0-2305843897198883067-0](offset=700667,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:331776, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328401377000, upper_trans_version:1717560328401377000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:965475280, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328401377000, max_merged_trans_version_:1717560328401377000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ef7fed4d8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ef7fed518, block_data:{buf:0x2b0ef7fed4d8, size:64, extra_buf:0x2b0ef7fed605, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0ef7fed9d4, block_data:{buf:0x2b0ef7fed9d4, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef7fedaca, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:331776, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=739, old_pos=0, pos=739) [2024-10-16 14:57:32.631549] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.631786] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f56f9a050, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f56f9a050, addr:[7667511-10399-0-2305845828859924480-0](offset=200704,size=2626,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:4, nested_size:2097152, nested_offset:0, total_macro_block_count:4, total_use_old_macro_block_count:3, row_count:90523, occupy_size:3024146, data_checksum:1326568258, max_merged_trans_version:1729015201157706000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:99, row_count_:90523, occupy_size_:3024146, original_size_:49661578, data_checksum_:1326568258, index_type_:0, rowkey_column_count_:3, column_cnt_:20, data_macro_block_count_:4, data_micro_block_count_:225, use_old_macro_block_count_:3, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328332192, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015201157706000, max_merged_trans_version_:1729015201157706000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f9a368, column_checksum_count:20, data_root_info:{addr:[0-0-0-3458765233227563008-0](offset=0,size=670,type=3,seq=0), orig_block_buf:0x2b0f56f9a468, block_data:{buf:0x2b0f56f9a408, size:96, extra_buf:0x2b0f56f9a706, extra_size:1959, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458766002026708992-0](offset=0,size=1386,type=3,seq=0), orig_block_buf:0x2b0f56f9aead, block_data:{buf:0x2b0f56f9aead, size:1386, extra_buf:null, extra_size:0, type:1}}, data_block_count:4, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f9b417, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:0, nested_size:2097152}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.631825] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=38] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f56f9b4a0, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061401350833000, timestamp:0}, this:0x2b0f56f9b4a0, addr:[7667511-10399-0-2305843945516767810-0](offset=203330,size=872,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:28672, nested_offset:90112, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:430, occupy_size:26401, data_checksum:0, max_merged_trans_version:1729061401350833000, upper_trans_version:1729061401350833000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:430, occupy_size_:26401, original_size_:220397, data_checksum_:226929565, index_type_:0, rowkey_column_count_:3, column_cnt_:20, data_macro_block_count_:1, data_micro_block_count_:3, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061401350833000, max_merged_trans_version_:1729061401350833000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f9b7b8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0f56f9b7f8, block_data:{buf:0x2b0f56f9b7b8, size:64, extra_buf:0x2b0f56f9b8e6, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0f56f9bc25, block_data:{buf:0x2b0f56f9bc25, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f9bd35, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:90112, nested_size:28672}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.631865] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=39] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e76050, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061761547003000, timestamp:0}, this:0x2b0f36e76050, addr:[7667511-10399-0-2305843920820706730-0](offset=204202,size=849,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:557056, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:12, occupy_size:1362, data_checksum:1961330692, max_merged_trans_version:1729061761547003000, upper_trans_version:1729061761547003000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:12, occupy_size_:1362, original_size_:5305, data_checksum_:1961330692, index_type_:0, rowkey_column_count_:3, column_cnt_:20, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061761547003000, max_merged_trans_version_:1729061761547003000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e76368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764769371095040-0](offset=0,size=238,type=3,seq=0), orig_block_buf:0x2b0f36e763a8, block_data:{buf:0x2b0f36e76368, size:64, extra_buf:0x2b0f36e76496, extra_size:831, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764791919673344-0](offset=0,size=259,type=3,seq=0), orig_block_buf:0x2b0f36e767d5, block_data:{buf:0x2b0f36e767d5, size:259, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e768d8, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:557056, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.631890] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f36e76910, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328401377000, timestamp:0}, this:0x2b0f36e76910, addr:[7667511-10399-0-2305843897198387451-0](offset=205051,size=827,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:331776, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328401377000, upper_trans_version:1717560328401377000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:965475280, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328401377000, max_merged_trans_version_:1717560328401377000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f36e76c28, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f36e76c68, block_data:{buf:0x2b0f36e76c28, size:64, extra_buf:0x2b0f36e76d55, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764777961029632-0](offset=0,size=246,type=3,seq=0), orig_block_buf:0x2b0f36e77124, block_data:{buf:0x2b0f36e77124, size:246, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e7721a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:331776, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.631930] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.632076] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.632285] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6a2040a0, start_pos=13826, remain=51622) [2024-10-16 14:57:32.632348] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.632436] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:200067}, table_store_addr_.addr_=[7667511-10399-0-2305843634131644416-0](offset=208896,size=582,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843564338426438-0](offset=209478,size=517,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686860242067456-0](offset=1089536,size=784,type=4,seq=0), this=0x2b0e6a2040a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:4, arr:0x2b0f36e77a70, capacity:4}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f36e77ae0, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0f36e77b00, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.632713] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601370, offset=9617308}, end_cursor:ObLogCursor{file_id=192, log_id=47601371, offset=9617424}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d90050, len:4196, log_data_len:116, seq:47601370, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.632729] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=22] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686860242067456-0](offset=1089536,size=784,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6a2040a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6a2040a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686860242067456-0](offset=1089536,size=784,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:200067}, data_tablet_id:{id:200067}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:1326568258, row_count:90523, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328401377000, v:0}, create_commit_version:1717560328401377000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328332152, space_usage:{shared_data_size:36864, data_size:8388608, shared_meta_size:16384, meta_size:0, occupy_bytes:3061010}}, table_store_addr:{ptr:0x2b0e6a20468c, addr:[7667511-10399-0-2305843634131644416-0](offset=208896,size=582,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843564338426438-0](offset=209478,size=517,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a2076a2, addr:[7663823-15172-0-4611686344845992416-0](offset=1090016,size=304,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.632776] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:200067}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0e931ea060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:4, arr:0x2b0e8487caa0, capacity:4}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e8487cb10, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e8487cb30, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.632791] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] reset tablet(this=0x2b0e931ea060, ls_id={id:1001}, tablet_id={id:200067}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.633375] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:4, rowkey_column_count:4, schema_rowkey_col_cnt:2, full_stored_col_cnt:4, col_desc_array:[cnt:4, column_id=17 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:4, 2197175160, 2197175160, 0, 0], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0eda24a060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:5, index_type:7, row_store_type:1, schema_version:1717560328798944, column_cnt:2, store_column_cnt:2, tablet_size:134217728, pctfree:0, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:6, encryption:"", encrypt_key:"", rowkey_cnt:2, rowkey_array:[cnt:2, {column_idx:17, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0}, {column_idx:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:2, column_array:[cnt:2, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:6, row_store_type:1, block_size:16384, schema_column_cnt:2, rowkey_column_cnt:4, schema_rowkey_column_cnt:2, column_cnt:4, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:4, column_id=17 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.633486] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=37] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.633552] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] successfully write small sstable(ret=0, block_info={nested_size:4096, nested_offset:561152, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=565248, old_block=[]) [2024-10-16 14:57:32.633567] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 37, flag: 0, null: 0, ptr: 0x2b0f12ca24e0, hex: 436F6C6C65637420616C6C2074656E616E7473206D616A6F7220636F6D70616374696F6E73, cstr: Collect all tenants major compactions}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12ca2440, hex: 8A9C2E0000000000, int: 3054730}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12ca2510, hex: E80D715F7D2301E8, int: -1729061760554431000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0f12ca24b0, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:4, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:614, meta_block_size:257, last_macro_size:871, macro_metas:0x2b0f12ca2100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.633610] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764821984444416-0](offset=0,size=287,type=3,seq=0), buf:0x2b0f36e73080, height:1, is_meta_root:true}) [2024-10-16 14:57:32.633619] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764811247026176-0](offset=0,size=277,type=3,seq=0), buf:0x2b0f36e72560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764821984444416-0](offset=0,size=287,type=3,seq=0), buf:0x2b0f36e73080, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:4, data_column_checksums:[0, 0, 0, 0], row_count:3, max_merged_trans_version:1729061760554431000, contain_uncommitted_row:false, occupy_size:398, original_size:241, data_checksum:764587105, use_old_macro_block_count:0, compressor_type:2, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.633638] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=11] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0f278da060, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0f278da060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:561152, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:398, data_checksum:764587105, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:398, original_size_:241, data_checksum_:764587105, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764811247026176-0](offset=0,size=277,type=3,seq=0), orig_block_buf:0x2b0f278da390, block_data:{buf:0x2b0f278da5f0, size:64, extra_buf:0x2b0f278da630, extra_size:1012, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764821984444416-0](offset=0,size=287,type=3,seq=0), orig_block_buf:0x2b0f278da4b0, block_data:{buf:0x2b0f278da4b0, size:287, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f278da5d0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:561152, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.633673] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] finish set upper trans version(ret=0, key_={tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, meta=0x2b0f278da130, old_val=1729061760554431000, upper_trans_version=1729061760554431000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:561152, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:398, data_checksum:764587105, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.633683] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:4096, nested_offset:561152, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}) [2024-10-16 14:57:32.633805] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f177ae5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f177af070, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f177ae5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f177af070, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f56f0a9f0, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729015200571456000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINOR 1729061400920852000 1729061400920852000 1729015204702466001 1729061414084957000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729061760554431000 1729061760554431000 1729061414084957000 1729061778253675001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560328873906000 1717560328873906000 1 1729036755691700000 0 false }}) [2024-10-16 14:57:32.633927] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846993}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f177ae060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f56f0ba60, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f56f0ba80, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.633942] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e6a1480a0, is_inited:true, wash_score:1729061852022735000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797964394496-0](offset=442368,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846993}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:2421085261, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798936, space_usage:{shared_data_size:712704, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:712704}}, table_store_addr:{ptr:0x2b0e6a1486b0, addr:[7663857-6566-0-2305843630910926848-0](offset=716800,size=579,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843227184001603-0](offset=717379,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6a14af2a, addr:[7663823-15172-0-4611686276125868518-0](offset=442854,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f177ae060, is_inited:true, wash_score:1729061852633732000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846993}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:2421085261, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798936, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:2949, meta_size:0, occupy_bytes:712704}}, table_store_addr:{ptr:0x2b0f177ae450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0f56f0b330, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0f56f0b9d0, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.634011] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=53] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f177ae5d0, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f177ae5d0, addr:[7663857-6566-0-2305844275156013056-0](offset=708608,size=1179,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90516, occupy_size:693866, data_checksum:2421085261, max_merged_trans_version:1729015200571456000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:693866, original_size_:6245580, data_checksum_:2421085261, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:44, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798936, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015200571456000, max_merged_trans_version_:1729015200571456000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177ae8e8, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764903588823040-0](offset=0,size=363,type=3,seq=0), orig_block_buf:0x2b0f177ae970, block_data:{buf:0x2b0f177ae908, size:104, extra_buf:0x2b0f177aeadb, extra_size:964, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764977677008896-0](offset=0,size=432,type=3,seq=0), orig_block_buf:0x2b0f177aee9f, block_data:{buf:0x2b0f177aee9f, size:432, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177af04f, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1091, old_pos=0, pos=1091) [2024-10-16 14:57:32.634048] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f177af090, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061400920852000, timestamp:0}, this:0x2b0f177af090, addr:[7663857-6566-0-2305844015310492827-0](offset=709787,size=937,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:163840, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:385, occupy_size:7579, data_checksum:0, max_merged_trans_version:1729061400920852000, upper_trans_version:1729061400920852000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:385, occupy_size_:7579, original_size_:30943, data_checksum_:4234878802, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061400920852000, max_merged_trans_version_:1729061400920852000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f177af3a8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764811247026176-0](offset=0,size=277,type=3,seq=0), orig_block_buf:0x2b0f177af3e8, block_data:{buf:0x2b0f177af3a8, size:64, extra_buf:0x2b0f177af4fd, extra_size:1012, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764833795604480-0](offset=0,size=298,type=3,seq=0), orig_block_buf:0x2b0f177af8f1, block_data:{buf:0x2b0f177af8f1, size:298, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f177afa1b, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:163840, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=849, old_pos=0, pos=849) [2024-10-16 14:57:32.634077] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f0a050, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0f56f0a050, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:561152, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:398, data_checksum:764587105, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:398, original_size_:241, data_checksum_:764587105, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f0a368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764811247026176-0](offset=0,size=277,type=3,seq=0), orig_block_buf:0x2b0f56f0a3a8, block_data:{buf:0x2b0f56f0a368, size:64, extra_buf:0x2b0f56f0a4bd, extra_size:1012, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764821984444416-0](offset=0,size=287,type=3,seq=0), orig_block_buf:0x2b0f56f0a8b1, block_data:{buf:0x2b0f56f0a8b1, size:287, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f0a9d0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:561152, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=828, old_pos=0, pos=828) [2024-10-16 14:57:32.634104] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f0aa00, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0f56f0aa00, addr:[7663857-6566-0-2305843903641344984-0](offset=711640,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1859584, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f56f0ad18, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f56f0ad58, block_data:{buf:0x2b0f56f0ad18, size:64, extra_buf:0x2b0f56f0ae45, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0f56f0b214, block_data:{buf:0x2b0f56f0b214, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f0b310, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1859584, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=745, old_pos=0, pos=745) [2024-10-16 14:57:32.634157] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.634261] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef9608690, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0ef9608690, addr:[7667511-10399-0-2305844275155517440-0](offset=212992,size=1179,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:696320, nested_offset:4096, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:90516, occupy_size:693866, data_checksum:2421085261, max_merged_trans_version:1729015200571456000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:97, row_count_:90516, occupy_size_:693866, original_size_:6245580, data_checksum_:2421085261, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:44, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560328798936, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729015200571456000, max_merged_trans_version_:1729015200571456000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:6, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef96089a8, column_checksum_count:4, data_root_info:{addr:[0-0-0-3458764903588823040-0](offset=0,size=363,type=3,seq=0), orig_block_buf:0x2b0ef9608a30, block_data:{buf:0x2b0ef96089c8, size:104, extra_buf:0x2b0ef9608b9b, extra_size:964, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764977677008896-0](offset=0,size=432,type=3,seq=0), orig_block_buf:0x2b0ef9608f5f, block_data:{buf:0x2b0ef9608f5f, size:432, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef960910f, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:4096, nested_size:696320}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.634291] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=29] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ef9609150, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1729015204702466001, v:0}, end_scn:{val:1729061414084957000, v:0}}}, ref_cnt:0, upper_trans_version:1729061400920852000, timestamp:0}, this:0x2b0ef9609150, addr:[7667511-10399-0-2305844015309997211-0](offset=214171,size=937,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:163840, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:385, occupy_size:7579, data_checksum:0, max_merged_trans_version:1729061400920852000, upper_trans_version:1729061400920852000, filled_tx_scn:{val:1729061414084957000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:95, row_count_:385, occupy_size_:7579, original_size_:30943, data_checksum_:4234878802, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061400920852000, max_merged_trans_version_:1729061400920852000, recycle_version:1729015204839740000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061414084957000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ef9609468, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764811247026176-0](offset=0,size=277,type=3,seq=0), orig_block_buf:0x2b0ef96094a8, block_data:{buf:0x2b0ef9609468, size:64, extra_buf:0x2b0ef96095bd, extra_size:1012, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764833795604480-0](offset=0,size=298,type=3,seq=0), orig_block_buf:0x2b0ef96099b1, block_data:{buf:0x2b0ef96099b1, size:298, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ef9609adb, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:163840, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.634316] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ec83c2050, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729061414084957000, v:0}, end_scn:{val:1729061778253675001, v:0}}}, ref_cnt:0, upper_trans_version:1729061760554431000, timestamp:0}, this:0x2b0ec83c2050, addr:[7667511-10399-0-2305843992761419844-0](offset=215108,size=916,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:561152, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:3, occupy_size:398, data_checksum:764587105, max_merged_trans_version:1729061760554431000, upper_trans_version:1729061760554431000, filled_tx_scn:{val:1729061778253675001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:3, occupy_size_:398, original_size_:241, data_checksum_:764587105, index_type_:7, rowkey_column_count_:4, column_cnt_:4, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729061760554431000, max_merged_trans_version_:1729061760554431000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729061778253675001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:2, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0ec83c2368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764811247026176-0](offset=0,size=277,type=3,seq=0), orig_block_buf:0x2b0ec83c23a8, block_data:{buf:0x2b0ec83c2368, size:64, extra_buf:0x2b0ec83c24bd, extra_size:1012, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764821984444416-0](offset=0,size=287,type=3,seq=0), orig_block_buf:0x2b0ec83c28b1, block_data:{buf:0x2b0ec83c28b1, size:287, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec83c29d0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:561152, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.634346] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=29] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0ec83c2a00, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729036755691700000, v:0}}}, ref_cnt:0, upper_trans_version:1717560328873906000, timestamp:0}, this:0x2b0ec83c2a00, addr:[7667511-10399-0-2305843903640849368-0](offset=216024,size=833,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1859584, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:1, occupy_size:481, data_checksum:0, max_merged_trans_version:1717560328873906000, upper_trans_version:1717560328873906000, filled_tx_scn:{val:1729036755691700000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:1, occupy_size_:481, original_size_:184, data_checksum_:585614547, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560328873906000, max_merged_trans_version_:1717560328873906000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729036755691700000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0ec83c2d18, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0ec83c2d58, block_data:{buf:0x2b0ec83c2d18, size:64, extra_buf:0x2b0ec83c2e45, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764784403480576-0](offset=0,size=252,type=3,seq=0), orig_block_buf:0x2b0ec83c3214, block_data:{buf:0x2b0ec83c3214, size:252, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0ec83c3310, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1859584, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.634384] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.634772] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e6656c0a0, start_pos=11914, remain=53534) [2024-10-16 14:57:32.634853] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.634983] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1001}, tablet_id={id:1152921504606846993}, table_store_addr_.addr_=[7667511-10399-0-2305843630910427136-0](offset=217088,size=579,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843227183501891-0](offset=217667,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797965045760-0](offset=1093632,size=726,type=4,seq=0), this=0x2b0e6656c0a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0ec83c37c0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0ec83c37e0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.635236] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601371, offset=9617424}, end_cursor:ObLogCursor{file_id=192, log_id=47601372, offset=9617540}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d08050, len:4080, log_data_len:116, seq:47601371, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.635260] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=16] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686797965045760-0](offset=1093632,size=726,type=4,seq=0), new_tablet_hdl={obj:0x2b0e6656c0a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e6656c0a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686797965045760-0](offset=1093632,size=726,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1001}, tablet_id:{id:1152921504606846993}, data_tablet_id:{id:200070}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729015204702466001, v:0}, clog_checkpoint_scn:{val:1729061778253675001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729061778153872001, multi_version_start:1729059970141673000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:2421085261, row_count:90516, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1729036755691700000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560328873906000, v:0}, create_commit_version:1717560328873906000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:1717560328798936, space_usage:{shared_data_size:712704, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:712704}}, table_store_addr:{ptr:0x2b0e6656c6b0, addr:[7667511-10399-0-2305843630910427136-0](offset=217088,size=579,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843227183501891-0](offset=217667,size=203,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6656ef2a, addr:[7663823-15172-0-4611686276126519782-0](offset=1094118,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.635306] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846993}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f177ae060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f56f0ba60, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f56f0ba80, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.635322] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] reset tablet(this=0x2b0f177ae060, ls_id={id:1001}, tablet_id={id:1152921504606846993}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.635728] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1003}, tablet_id:{id:49403}}) [2024-10-16 14:57:32.635979] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0BFB-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061852634865000, start_id=1729061852634865000, end_id=1729061852634865001) [2024-10-16 14:57:32.636013] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0BFF-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061852634865000, start_id=1729061852634865000, end_id=1729061852634865001) [2024-10-16 14:57:32.636147] INFO [STORAGE] init (ob_data_store_desc.cpp:295) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] success to init data desc(ret=0, this={is_row_store:true, table_cg_idx:0, row_column_count:22, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:22, col_desc_array:[cnt:22, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=33 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:22, 2197175160, 2197175160, 2197175160, 0, 0, 2197175160, 2197175160, 2388842353, 2388842353, 2388842353, 2388842353, 2388842353, 2388842353, 2388842353, 2388842353, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2197175160, 2388842353], agg_meta_array:[cnt:0]}, merge_schema={this:0x2b0ef9082060, storage_schema_version:3, version:0, is_use_bloomfilter:0, column_info_simplified:0, compat_mode:0, table_type:0, index_type:0, row_store_type:1, schema_version:1717560288896224, column_cnt:20, store_column_cnt:20, tablet_size:134217728, pctfree:10, block_size:16384, progressive_merge_round:1, master_key_id:18446744073709551615, compressor_type:1, encryption:"", encrypt_key:"", rowkey_cnt:3, rowkey_array:[cnt:3, {column_idx:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}, {column_idx:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0}], column_cnt:20, column_array:[cnt:20, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:1, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"NULL":"NULL"}}, {meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, is_column_stored_in_sstable:1, is_rowkey_column:0, is_generated_column:0, orig_default_value:{"BIGINT":0}}], skip_index_cnt:0, skip_idx_attr_array:[cnt:0], column_group_cnt:1, column_group_array:[cnt:1, {version:1, type:1, compressor_type:1, row_store_type:1, block_size:16384, schema_column_cnt:20, rowkey_column_cnt:5, schema_rowkey_column_cnt:3, column_cnt:22, column_idxs:null}], has_all_column_group:true}, table_cg_idx=0, is_oracle_mode=false, col_desc_array_=[cnt:22, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=33 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0]) [2024-10-16 14:57:32.638615] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=127] Async write macro block(macro_id_=[12379](ver=0,mode=0,seq=7667512)) [2024-10-16 14:57:32.638723] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:226) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8] successfully write small sstable(ret=0, block_info={nested_size:20480, nested_offset:565248, macro_id:[12379](ver=0,mode=0,seq=7667512)}, offset_=585728, old_block=[]) [2024-10-16 14:57:32.638759] INFO [STORAGE] close (ob_index_block_builder.cpp:2857) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to close rebuilder(index_tree_root_ctx={allocator_:0x2b0f12c72050, last_key:{datum_cnt:5, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea170e408, hex: 0000000000000000, int: 0}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea170e440, hex: 8FA1070000000000, int: 500111}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea170e478, hex: 1700000000000000, int: 23}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea170e520, hex: E04FEF7E952401E8, int: -1729060557435220000}, idx=4:{len: 8, flag: 0, null: 0, ptr: 0x2b0ea170e4e8, hex: 0000000000000000, int: 0}], store_rowkey:}, task_idx:-1, data_column_cnt:22, data_blocks_cnt:1, use_old_macro_block_count:0, meta_block_offset:19430, meta_block_size:272, last_macro_size:19702, macro_metas:0x2b0ea170e100, absolute_offsets:null, use_absolute_offset:false, is_inited:true}, need_sort_=true) [2024-10-16 14:57:32.638799] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:2586) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=14] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), buf:0x2b0f36e05060, height:1, is_meta_root:true}) [2024-10-16 14:57:32.638807] INFO [STORAGE] close (ob_index_block_builder.cpp:1041) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to close sstable index builder(res={root_desc:{addr:[0-0-0-3458764774739804160-0](offset=0,size=243,type=3,seq=0), buf:0x2b0f36e04560, height:2, is_meta_root:false}, data_root_desc:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), buf:0x2b0f36e05060, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:2, data_column_cnt:22, data_column_checksums:[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], row_count:196, max_merged_trans_version:1729060557435220000, contain_uncommitted_row:false, occupy_size:19014, original_size:18517, data_checksum:1708441798, use_old_macro_block_count:0, compressor_type:1, root_row_store_type:0, nested_offset:0, nested_size:2097152, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2024-10-16 14:57:32.638831] INFO [STORAGE] init (ob_sstable.cpp:304) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=13] succeeded to init sstable(ret=0, this={ObITable:{this:0x2b0eda24a060, key:{tablet_id:{id:323}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729060578209260001, v:0}}}, ref_cnt:0, upper_trans_version:1729060557435220000, timestamp:0}, this:0x2b0eda24a060, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:565248, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:196, occupy_size:19014, data_checksum:1708441798, max_merged_trans_version:1729060557435220000, upper_trans_version:1729060557435220000, filled_tx_scn:{val:1729060578209260001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:196, occupy_size_:19014, original_size_:18517, data_checksum_:1708441798, index_type_:0, rowkey_column_count_:5, column_cnt_:22, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060557435220000, max_merged_trans_version_:1729060557435220000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060578209260001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:null, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764774739804160-0](offset=0,size=243,type=3,seq=0), orig_block_buf:0x2b0eda24a390, block_data:{buf:0x2b0eda24a5d0, size:64, extra_buf:0x2b0eda24a610, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0eda24a490, block_data:{buf:0x2b0eda24a490, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0eda24a5b0, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:565248, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2024-10-16 14:57:32.638867] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1043) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=36] finish set upper trans version(ret=0, key_={tablet_id:{id:323}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729060578209260001, v:0}}}, meta=0x2b0eda24a130, old_val=1729060557435220000, upper_trans_version=1729060557435220000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:565248, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:196, occupy_size:19014, data_checksum:1708441798, max_merged_trans_version:1729060557435220000, upper_trans_version:1729060557435220000, filled_tx_scn:{val:1729060578209260001, v:0}, contain_uncommitted_row:false}) [2024-10-16 14:57:32.638877] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:730) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9] successfully rebuild one sstable(ret=0, block_info={nested_size:20480, nested_offset:565248, macro_id:[12379](ver=0,mode=0,seq=7667512)}, new_sstable.get_key()={tablet_id:{id:323}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729060578209260001, v:0}}}) [2024-10-16 14:57:32.639003] INFO [STORAGE] init (ob_tablet_table_store.cpp:346) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to batch replace table store(major_tables_={cnt:1, sstable_array:0x2b0f36e2c5c0, serialize_table_type:false, is_inited:true}, minor_tables_={cnt:2, sstable_array:0x2b0f36e2d220, serialize_table_type:false, is_inited:true}, memtables_={count:0}, PRINT_TS(*this)={ObTabletTableStore_Pretty:this:0x2b0e24ecf8c0, major_tables:{cnt:1, sstable_array:0x2b0f36e2c5c0, serialize_table_type:false, is_inited:true}, minor_tables:{cnt:2, sstable_array:0x2b0f36e2d220, serialize_table_type:false, is_inited:true}, mds_sstables:{cnt:1, sstable_array:0x2b0f56f9aa20, serialize_table_type:false, is_inited:true}, memtables:{count:0}, is_ready_for_read:true, table_array:{ [22145] [ Y0-0000000000000000-0-0 ] table_arr table_type upper_trans_ver max_merge_ver start_scn end_scn ref uncommit_row [22145] [ Y0-0000000000000000-0-0 ] MAJOR MAJOR 0 1729000842358944000 0 1729015204839740000 0 false [22145] [ Y0-0000000000000000-0-0 ] MINOR MINI 1729060557435220000 1729060557435220000 1729001176236986000 1729060578209260001 0 false [22145] [ Y0-0000000000000000-0-0 ] MINI 1729060901022301000 1729060901022301000 1729060578209260001 1729060940167357001 0 false [22145] [ Y0-0000000000000000-0-0 ] MDS MDS_MINOR 1717560287003454000 1717560287003454000 1 1728978212337222000 0 false }}) [2024-10-16 14:57:32.639150] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=19] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:323}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f36e2c060, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f36e761b0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f36e761d0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120a8102 0x120b027d 0x13ace00a 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.639166] INFO [STORAGE] init_for_defragment (ob_tablet.cpp:762) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to init tablet for sstable defragmentation(ret=0, old_tablet={this:0x2b0e665d80a0, is_inited:true, wash_score:1729061852154936000, hold_ref_cnt:true, ref_cnt:4, version:4, length:0, tablet_addr:[7663823-15172-0-4611686803333455872-0](offset=794624,size=731,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:323}, data_tablet_id:{id:323}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729001176236986000, v:0}, clog_checkpoint_scn:{val:1729060940167357001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729060939970781000, multi_version_start:1729059129663351000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:1126791159, row_count:387, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:49152, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:49152}}, table_store_addr:{ptr:0x2b0e665d86d4, addr:[7663857-6566-0-2305843631985647616-0](offset=1695744,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7663857-6566-0-2305843629838164548-0](offset=1696324,size=578,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e665db160, addr:[7663823-15172-0-4611686276126220779-0](offset=795115,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}, this={this:0x2b0f36e2c060, is_inited:true, wash_score:1729061852638545000, hold_ref_cnt:true, ref_cnt:1, version:4, length:0, tablet_addr:[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), allocator:0x2b0e24ed0000, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:323}, data_tablet_id:{id:323}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729001176236986000, v:0}, clog_checkpoint_scn:{val:1729060940167357001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729060939970781000, multi_version_start:1729059129663351000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:1126791159, row_count:387, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:8388608, data_size:0, shared_meta_size:3122, meta_size:0, occupy_bytes:49152}}, table_store_addr:{ptr:0x2b0f36e2c450, addr:[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0)}, storage_schema_addr:{ptr:0x2b0f56f9b360, addr:[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0)}, macro_info_addr:{ptr:0x2b0f36e76120, addr:[0-0-0-3458764651259494400-0](offset=0,size=128,type=3,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.639238] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=54] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f36e2c5d0, key:{tablet_id:{id:323}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f36e2c5d0, addr:[7663857-6566-0-2305844551108644864-0](offset=1691648,size=1436,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:2019328, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:387, occupy_size:5737, data_checksum:1126791159, max_merged_trans_version:1729000842358944000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:94, row_count_:387, occupy_size_:5737, original_size_:68112, data_checksum_:1126791159, index_type_:0, rowkey_column_count_:5, column_cnt_:22, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560288896224, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729000842358944000, max_merged_trans_version_:1729000842358944000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e2c8e8, column_checksum_count:22, data_root_info:{addr:[0-0-0-3458764939022303232-0](offset=0,size=396,type=3,seq=0), orig_block_buf:0x2b0f36e2ca08, block_data:{buf:0x2b0f36e2c998, size:112, extra_buf:0x2b0f36e2cb94, extra_size:1095, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765109747253248-0](offset=0,size=555,type=3,seq=0), orig_block_buf:0x2b0f36e2cfdb, block_data:{buf:0x2b0f36e2cfdb, size:555, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e2d206, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2019328, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=1348, old_pos=0, pos=1348) [2024-10-16 14:57:32.639275] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=34] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f36e2d240, key:{tablet_id:{id:323}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729060578209260001, v:0}}}, ref_cnt:0, upper_trans_version:1729060557435220000, timestamp:0}, this:0x2b0f36e2d240, addr:[0-0-0-3458764719978971136-0](offset=0,size=192,type=3,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:565248, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:196, occupy_size:19014, data_checksum:1708441798, max_merged_trans_version:1729060557435220000, upper_trans_version:1729060557435220000, filled_tx_scn:{val:1729060578209260001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:196, occupy_size_:19014, original_size_:18517, data_checksum_:1708441798, index_type_:0, rowkey_column_count_:5, column_cnt_:22, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060557435220000, max_merged_trans_version_:1729060557435220000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060578209260001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f36e2d558, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764774739804160-0](offset=0,size=243,type=3,seq=0), orig_block_buf:0x2b0f36e2d598, block_data:{buf:0x2b0f36e2d558, size:64, extra_buf:0x2b0f36e2d68b, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0f36e2daea, block_data:{buf:0x2b0f36e2daea, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f36e2dbfa, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:565248, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=782, old_pos=0, pos=782) [2024-10-16 14:57:32.639304] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=26] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f9a050, key:{tablet_id:{id:323}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729060578209260001, v:0}, end_scn:{val:1729060940167357001, v:0}}}, ref_cnt:0, upper_trans_version:1729060901022301000, timestamp:0}, this:0x2b0f56f9a050, addr:[7663857-6566-0-2305843932633356546-0](offset=1693954,size=860,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:16384, nested_offset:1667072, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:119, occupy_size:11823, data_checksum:0, max_merged_trans_version:1729060901022301000, upper_trans_version:1729060901022301000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:119, occupy_size_:11823, original_size_:11390, data_checksum_:1154704995, index_type_:0, rowkey_column_count_:5, column_cnt_:22, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060901022301000, max_merged_trans_version_:1729060901022301000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060940167357001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f56f9a368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764774739804160-0](offset=0,size=243,type=3,seq=0), orig_block_buf:0x2b0f56f9a3a8, block_data:{buf:0x2b0f56f9a368, size:64, extra_buf:0x2b0f56f9a49b, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), orig_block_buf:0x2b0f56f9a8fa, block_data:{buf:0x2b0f56f9a8fa, size:265, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f9aa03, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1667072, nested_size:16384}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=772, old_pos=0, pos=772) [2024-10-16 14:57:32.639330] INFO [STORAGE] serialize_full_table (ob_sstable.cpp:1191) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize full sstable(status.pack_=21762, this={ObITable:{this:0x2b0f56f9aa30, key:{tablet_id:{id:323}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0f56f9aa30, addr:[7663857-6566-0-2305843896126135390-0](offset=1694814,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1617920, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:626, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:626, original_size_:329, data_checksum_:1596289367, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f56f9ad48, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f56f9ad88, block_data:{buf:0x2b0f56f9ad48, size:64, extra_buf:0x2b0f56f9ae75, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0f56f9b244, block_data:{buf:0x2b0f56f9b244, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f56f9b339, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1617920, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, buf_len=738, old_pos=0, pos=738) [2024-10-16 14:57:32.639373] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=25] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.639625] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0bdf4660, key:{tablet_id:{id:323}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1729015204839740000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x2b0f0bdf4660, addr:[7667511-10399-0-2305844551107174400-0](offset=221184,size=1436,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:2019328, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:387, occupy_size:5737, data_checksum:1126791159, max_merged_trans_version:1729000842358944000, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:94, row_count_:387, occupy_size_:5737, original_size_:68112, data_checksum_:1126791159, index_type_:0, rowkey_column_count_:5, column_cnt_:22, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1717560288896224, create_snapshot_version_:1728928802547408000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729000842358944000, max_merged_trans_version_:1729000842358944000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f0bdf4978, column_checksum_count:22, data_root_info:{addr:[0-0-0-3458764939022303232-0](offset=0,size=396,type=3,seq=0), orig_block_buf:0x2b0f0bdf4a98, block_data:{buf:0x2b0f0bdf4a28, size:112, extra_buf:0x2b0f0bdf4c24, extra_size:1095, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458765109747253248-0](offset=0,size=555,type=3,seq=0), orig_block_buf:0x2b0f0bdf506b, block_data:{buf:0x2b0f0bdf506b, size:555, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0bdf5296, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:2019328, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.639664] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=38] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f0bdf52d0, key:{tablet_id:{id:323}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729001176236986000, v:0}, end_scn:{val:1729060578209260001, v:0}}}, ref_cnt:0, upper_trans_version:1729060557435220000, timestamp:0}, this:0x2b0f0bdf52d0, addr:[7667511-10399-0-2305843943369303452-0](offset=222620,size=870,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:20480, nested_offset:565248, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:196, occupy_size:19014, data_checksum:1708441798, max_merged_trans_version:1729060557435220000, upper_trans_version:1729060557435220000, filled_tx_scn:{val:1729060578209260001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:88, row_count_:196, occupy_size_:19014, original_size_:18517, data_checksum_:1708441798, index_type_:0, rowkey_column_count_:5, column_cnt_:22, data_macro_block_count_:1, data_micro_block_count_:2, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060557435220000, max_merged_trans_version_:1729060557435220000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060578209260001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f0bdf55e8, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764774739804160-0](offset=0,size=243,type=3,seq=0), orig_block_buf:0x2b0f0bdf5628, block_data:{buf:0x2b0f0bdf55e8, size:64, extra_buf:0x2b0f0bdf571b, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764805878317056-0](offset=0,size=272,type=3,seq=0), orig_block_buf:0x2b0f0bdf5b7a, block_data:{buf:0x2b0f0bdf5b7a, size:272, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f0bdf5c8a, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:565248, nested_size:20480}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.639690] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f33468050, key:{tablet_id:{id:323}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1729060578209260001, v:0}, end_scn:{val:1729060940167357001, v:0}}}, ref_cnt:0, upper_trans_version:1729060901022301000, timestamp:0}, this:0x2b0f33468050, addr:[7667511-10399-0-2305843932631886082-0](offset=223490,size=860,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:16384, nested_offset:1667072, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:119, occupy_size:11823, data_checksum:0, max_merged_trans_version:1729060901022301000, upper_trans_version:1729060901022301000, filled_tx_scn:{val:1729060940167357001, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:119, occupy_size_:11823, original_size_:11390, data_checksum_:1154704995, index_type_:0, rowkey_column_count_:5, column_cnt_:22, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1729060901022301000, max_merged_trans_version_:1729060901022301000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1729060940167357001, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1}, column_checksums:0x2b0f33468368, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764774739804160-0](offset=0,size=243,type=3,seq=0), orig_block_buf:0x2b0f334683a8, block_data:{buf:0x2b0f33468368, size:64, extra_buf:0x2b0f3346849b, extra_size:1119, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764798362124288-0](offset=0,size=265,type=3,seq=0), orig_block_buf:0x2b0f334688fa, block_data:{buf:0x2b0f334688fa, size:265, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f33468a03, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1667072, nested_size:16384}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.639714] INFO [STORAGE] serialize (ob_sstable.cpp:1256) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] succeed to serialize sstable(status.pack_=21761, this={ObITable:{this:0x2b0f33468a30, key:{tablet_id:{id:323}, column_group_idx:0, table_type:"MDS_MINOR", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728978212337222000, v:0}}}, ref_cnt:0, upper_trans_version:1717560287003454000, timestamp:0}, this:0x2b0f33468a30, addr:[7667511-10399-0-2305843896124664926-0](offset=224350,size=826,type=2,seq=0), meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1617920, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:626, data_checksum:0, max_merged_trans_version:1717560287003454000, upper_trans_version:1717560287003454000, filled_tx_scn:{val:1728978212337222000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:85, row_count_:2, occupy_size_:626, original_size_:329, data_checksum_:1596289367, index_type_:0, rowkey_column_count_:4, column_cnt_:6, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:0, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1717560287003454000, max_merged_trans_version_:1717560287003454000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1728978212337222000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:0}, column_checksums:0x2b0f33468d48, column_checksum_count:0, data_root_info:{addr:[0-0-0-3458764768297353216-0](offset=0,size=237,type=3,seq=0), orig_block_buf:0x2b0f33468d88, block_data:{buf:0x2b0f33468d48, size:64, extra_buf:0x2b0f33468e75, extra_size:975, type:1}}, macro_info:{macro_meta_info:{addr:[0-0-0-3458764776887287808-0](offset=0,size=245,type=3,seq=0), orig_block_buf:0x2b0f33469244, block_data:{buf:0x2b0f33469244, size:245, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x2b0f33469339, other_block_ids:null, linked_block_ids:null, entry_id:[-1](ver=0,mode=0,seq=0), is_meta_root:true, nested_offset:1617920, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2024-10-16 14:57:32.639756] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=24] Async write macro block(macro_id_=[10399](ver=0,mode=0,seq=7667511)) [2024-10-16 14:57:32.640201] INFO [STORAGE] transform (ob_tablet_persister.cpp:914) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] TINY TABLET: tablet + rowkey_read_info + tablet store + auto_inc_seq(buf=0x2b0e699480a0, start_pos=12480, remain=52968) [2024-10-16 14:57:32.640267] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:185) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7] Async write macro block(macro_id_=[15172](ver=0,mode=0,seq=7663823)) [2024-10-16 14:57:32.640346] INFO [STORAGE] inc_ref_with_macro_iter (ob_tablet.cpp:2786) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that inner increases ref cnt is(ret=0, hold_ref_cnt_=false, ls_id={id:1}, tablet_id={id:323}, table_store_addr_.addr_=[7667511-10399-0-2305843631984177152-0](offset=225280,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7667511-10399-0-2305843629836694084-0](offset=225860,size=578,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686803333758976-0](offset=1097728,size=731,type=4,seq=0), this=0x2b0e699480a0, macro_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f33469c40, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f33469c60, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120df064 0x120937e2 0x120841f0 0x13ace09f 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.640585] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601372, offset=9617540}, end_cursor:ObLogCursor{file_id=192, log_id=47601373, offset=9617656}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bd8050, len:3964, log_data_len:116, seq:47601372, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:32.640612] INFO [STORAGE] update_tablet_table_store (ob_ls_tablet_service.cpp:1075) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=15] succeeded to build new tablet(ret=0, disk_addr=[7663823-15172-0-4611686803333758976-0](offset=1097728,size=731,type=4,seq=0), new_tablet_hdl={obj:0x2b0e699480a0, obj_pool:0x2b0e0bf4faf0, allocator:null, wash_priority:0, allow_copy_and_assign:true}, new_tablet_hdl.get_obj()={this:0x2b0e699480a0, is_inited:true, wash_score:-9223372036854775808, hold_ref_cnt:true, ref_cnt:2, version:4, length:0, tablet_addr:[7663823-15172-0-4611686803333758976-0](offset=1097728,size=731,type=4,seq=0), allocator:null, tablet_meta:{version:1, ls_id:{id:1}, tablet_id:{id:323}, data_tablet_id:{id:323}, ref_tablet_id:{id:0}, has_next_tablet:false, create_scn:{val:18446744073709551615, v:3}, start_scn:{val:1729001176236986000, v:0}, clog_checkpoint_scn:{val:1729060940167357001, v:0}, ddl_checkpoint_scn:{val:1, v:0}, snapshot_version:1729060939970781000, multi_version_start:1729059129663351000, compat_mode:0, ha_status:{restore_status:0, data_status:0, expected_status:0, reserved:0}, report_status:{merge_snapshot_version:1729015204839740000, cur_report_version:1729015204839740000, data_checksum:1126791159, row_count:387, found_cg_checksum_error:false}, table_store_flag:{with_major_sstable:1}, ddl_start_scn:{val:0, v:0}, ddl_snapshot_version:0, max_sync_storage_schema_version:0, max_serialized_medium_scn:0, ddl_execution_id:-1, ddl_data_format_version:0, ddl_commit_scn:{val:0, v:0}, mds_checkpoint_scn:{val:1728978212337222000, v:0}, transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:0, v:0}, transfer_seq:0, has_transfer_table:false}, extra_medium_info:{info:257, compat:1, last_compaction_type:1, wait_check_flag:0, last_medium_scn:1729015204839740000}, last_persisted_committed_tablet_status:{tablet_status:{val:1, str:"NORMAL"}, transfer_scn:{val:18446744073709551615, v:3}, transfer_ls_id:{id:-1}, data_type:1, create_commit_scn:{val:1717560287003454000, v:0}, create_commit_version:1717560287003454000, delete_commit_scn:{val:18446744073709551615, v:3}, delete_commit_version:-1, transfer_out_commit_version:-1}, create_schema_version:0, space_usage:{shared_data_size:49152, data_size:0, shared_meta_size:12288, meta_size:0, occupy_bytes:49152}}, table_store_addr:{ptr:0x2b0e699486d4, addr:[7667511-10399-0-2305843631984177152-0](offset=225280,size=580,type=2,seq=0)}, storage_schema_addr:{ptr:null, addr:[7667511-10399-0-2305843629836694084-0](offset=225860,size=578,type=2,seq=0)}, macro_info_addr:{ptr:0x2b0e6994b160, addr:[7663823-15172-0-4611686276126523883-0](offset=1098219,size=240,type=4,seq=0)}, mds_data:null, ddl_kvs:null, ddl_kv_count:0, memtables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], memtable_count:0}) [2024-10-16 14:57:32.640658] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=35] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:323}, table_store_addr_.addr_=[0-0-0-3458764891777662976-0](offset=0,size=352,type=3,seq=0), storage_schema_addr_.addr_=[0-0-0-3458764908957532160-0](offset=0,size=368,type=3,seq=0), tablet_addr_=[0-0-0-3458765587562364928-0](offset=0,size=1000,type=3,seq=0), this=0x2b0f36e2c060, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0f36e761b0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0f36e761d0, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x6611e5c 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.640673] INFO [STORAGE] reset (ob_tablet.cpp:273) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12] reset tablet(this=0x2b0f36e2c060, ls_id={id:1}, tablet_id={id:323}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x6611e65 0x13ace138 0x12c3576b 0x12c3b75e 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.640983] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1001}, tablet_id:{id:49401}}) [2024-10-16 14:57:32.641065] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully defragment data blocks(ret=0, rewrite_cnt=22, block_used_size_.count()=110) [2024-10-16 14:57:32.641080] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=4] successfully defragment data blocks(rewrite_cnt=22) [2024-10-16 14:57:32.641088] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] tablet skip filter destructed(total_cnt=2150, skip_cnt=8, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105368325, total_skip_cnt=310297, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:57:32.643398] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1002}, tablet_meta_.tablet_id_={id:209283}, table_store_addr_.addr_=[7663857-6566-0-2305843987393167360-0](offset=671744,size=911,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843270133629839-0](offset=672655,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686899969847296-0](offset=421888,size=821,type=4,seq=0), this=0x2b0e664940a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e66498c31, capacity:1}, shared_data_block_info_arr:{cnt:7, arr:0x2b0e66498c49, capacity:7}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643422] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=19] reset tablet(this=0x2b0e664940a0, ls_id={id:1002}, tablet_id={id:209283}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643437] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to gc tablet(ret=0, tablet=0x2b0e664940a0) [2024-10-16 14:57:32.643447] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=4] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846992}, table_store_addr_.addr_=[7663857-6566-0-2305843630911791104-0](offset=1581056,size=579,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843227184865859-0](offset=1581635,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686840914378752-0](offset=753664,size=766,type=4,seq=0), this=0x2b0e6b2940a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:3, arr:0x2b0e6b2974e3, capacity:3}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6b29752b, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e6b297543, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643458] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=9] reset tablet(this=0x2b0e6b2940a0, ls_id={id:1001}, tablet_id={id:1152921504606846992}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643471] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e6b2940a0) [2024-10-16 14:57:32.643479] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1002}, tablet_meta_.tablet_id_={id:209289}, table_store_addr_.addr_=[7663857-6566-0-2305843989541597184-0](offset=1617920,size=913,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843270134576017-0](offset=1618833,size=243,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686899970191360-0](offset=765952,size=821,type=4,seq=0), this=0x2b0e664b80a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e664bcc31, capacity:1}, shared_data_block_info_arr:{cnt:7, arr:0x2b0e664bcc49, capacity:7}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643490] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=9] reset tablet(this=0x2b0e664b80a0, ls_id={id:1002}, tablet_id={id:209289}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643499] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e664b80a0) [2024-10-16 14:57:32.643507] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:200067}, table_store_addr_.addr_=[7663857-6566-0-2305843634132156416-0](offset=720896,size=582,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843564338938438-0](offset=721478,size=517,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686860241424384-0](offset=446464,size=784,type=4,seq=0), this=0x2b0e6a2b80a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:4, arr:0x2b0e6a2bb722, capacity:4}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6a2bb782, capacity:1}, shared_data_block_info_arr:{cnt:3, arr:0x2b0e6a2bb79a, capacity:3}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643517] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=8] reset tablet(this=0x2b0e6a2b80a0, ls_id={id:1001}, tablet_id={id:200067}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643528] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e6a2b80a0) [2024-10-16 14:57:32.643535] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1001}, tablet_meta_.tablet_id_={id:1152921504606846993}, table_store_addr_.addr_=[7663857-6566-0-2305843630910926848-0](offset=716800,size=579,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843227184001603-0](offset=717379,size=203,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686797964394496-0](offset=442368,size=726,type=4,seq=0), this=0x2b0e6a1480a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e6a14afaa, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e6a14afc2, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643546] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=9] reset tablet(this=0x2b0e6a1480a0, ls_id={id:1001}, tablet_id={id:1152921504606846993}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643555] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e6a1480a0) [2024-10-16 14:57:32.643561] INFO [STORAGE] dec_ref_with_aggregated_info (ob_tablet.cpp:2964) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] the tablet that decreases ref cnt is(is_inited_=true, tablet_meta_.ls_id_={id:1}, tablet_meta_.tablet_id_={id:323}, table_store_addr_.addr_=[7663857-6566-0-2305843631985647616-0](offset=1695744,size=580,type=2,seq=0), storage_schema_addr_.addr_=[7663857-6566-0-2305843629838164548-0](offset=1696324,size=578,type=2,seq=0), tablet_addr_=[7663823-15172-0-4611686803333455872-0](offset=794624,size=731,type=4,seq=0), this=0x2b0e665d80a0, info_iter={macro_info:{entry_block:[-1](ver=0,mode=0,seq=0), meta_block_info_arr:{cnt:0, arr:null, capacity:0}, data_block_info_arr:{cnt:0, arr:null, capacity:0}, shared_meta_block_info_arr:{cnt:1, arr:0x2b0e665db1e0, capacity:1}, shared_data_block_info_arr:{cnt:4, arr:0x2b0e665db1f8, capacity:4}, is_inited:true}, cur_type:4, target_type:6, is_linked:false}, lbt()="0x1a5796b1 0x120e114a 0x120e0906 0x120c6e42 0x1315344c 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643572] INFO [STORAGE] reset (ob_tablet.cpp:273) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=9] reset tablet(this=0x2b0e665d80a0, ls_id={id:1}, tablet_id={id:323}, lbt()="0x1a5796b1 0x120a45d3 0x120a3fc3 0x120a3653 0x131ad108 0x1313fac4 0x13153503 0x6a663a2 0x131404d8 0x6809ea1 0x1a61b361 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:32.643582] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:822) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to gc tablet(ret=0, tablet=0x2b0e665d80a0) [2024-10-16 14:57:32.643585] INFO [STORAGE] gc_tablets_in_queue (ob_tenant_meta_mem_mgr.cpp:827) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=2] gc tablets in queue(gc_tablets_cnt=6, err_tablets_cnt=0, tablet_gc_queue_.count()=0) [2024-10-16 14:57:32.645881] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=16] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:57:32.645942] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:32.646636] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0BFC-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061852646141000, start_id=1729061852646141000, end_id=1729061852646141001) [2024-10-16 14:57:32.646704] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0BFE-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061852646141000, start_id=1729061852646141000, end_id=1729061852646141001) [2024-10-16 14:57:32.655534] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:57:32.655549] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:32.655555] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:32.655560] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:32.657824] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=111, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:57:32.657970] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=143] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:32.665829] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:32.668487] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:57:32.669416] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061852666453000, start_id=1729061852666453000, end_id=1729061852666453001) [2024-10-16 14:57:32.674932] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061852673615000, start_id=1729061852673615000, end_id=1729061852673615001) [2024-10-16 14:57:32.681463] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EEF-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:32.691871] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=23, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:57:32.691888] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=16] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=1973, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:57:32.691894] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=6] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:57:32.691900] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB EXECUTE TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=-611, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:57:32.691919] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=19] [APPLY STAT CB TOTAL TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=1385, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:57:32.691933] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:32.696967] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061852694789000, start_id=1729061852694789000, end_id=1729061852694789001) [2024-10-16 14:57:32.697046] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:32.697062] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:32.698092] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:32.699785] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.702285] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=15] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:32.719331] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:32.720376] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FC7-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=4606, avg_log_submit_to_slide_time=4925) [2024-10-16 14:57:32.723542] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:32.736086] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.739547] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D1E-0-0] [lt=17][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173059, log_task={header:{begin_lsn:{lsn:14352363401}, end_lsn:{lsn:14352363523}, log_id:115173059, min_scn:{val:1729061852636039000, v:0}, max_scn:{val:1729061852636039000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352363279}, prev_proposal_id:17, committed_end_lsn:{lsn:14352363401}, data_checksum:940009244, accum_checksum:3761956714, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061852639490, freeze_ts:1729061852639490, submit_ts:1729061852639496, flushed_ts:1729061852642864, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:3368}, fs_cb_begin_ts=1729061852739540, log_life_time=100050) [2024-10-16 14:57:32.741700] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D64-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:32.741720] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D64-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061852134673005, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061852134673005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:32.745305] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D64-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061852745057000, start_id=1729061852745057000, end_id=1729061852745057001) [2024-10-16 14:57:32.746010] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1, min_version={val:1729061852544670000, v:0}, base_version={val:1729061852132552000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061852544670000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061852745998, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061852544670000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061852704562, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061852544670000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061852652267, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:32.746799] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061852745057000, start_id=1729061852745057000, end_id=1729061852745057001) [2024-10-16 14:57:32.754359] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F7F-0-0] [lt=9] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846984}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328401377000, delete_commit_version:-1}, snapshot_version=1729061852562846000) [2024-10-16 14:57:32.754416] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F7F-0-0] [lt=15] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846984}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:32.755598] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:32.755631] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=19] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:32.755636] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=5] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:32.760761] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:32.760776] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:32.762739] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD107-0-0] [lt=24] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=2632, avg_log_submit_to_slide_time=2923) [2024-10-16 14:57:32.767611] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.767999] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B9F-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:32.768010] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B9F-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=65929, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:32.768015] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2B9F-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=1879, avg_log_submit_to_slide_time=65923) [2024-10-16 14:57:32.768084] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:32.768122] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=53, stat_interval=1000000, avg (count/cost)=329, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:32.768127] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=5] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=53, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:32.768245] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:32.770751] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:32.773908] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:207589536617}, scn={val:1729061852636039002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061852681895, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589536617}, next_to_submit_scn_:{val:1729061852636039002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589536495}, end_lsn:{lsn:207589536617}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061852636039001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061852636039001, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:73626199, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061852636039001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598774, miss_cnt:9, cache_read_size:149777901058, hit_ratio:"9.999999345924447391e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:57:32.779714] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:32.779729] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:32.779737] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=5] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=49) [2024-10-16 14:57:32.781151] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=13] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:32.781214] INFO [ARCHIVE] handle (ob_ls_meta_recorder.cpp:190) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=11] ls meta not refresh, no need record(share::ObArchiveLSMetaType::Type::SCHEMA_META=1, record_context={last_record_scn:{val:1729053451333382000, v:0}, last_record_round:3, last_record_piece:74, last_record_file:1729053451333382000}, scn={val:1729053451333382000, v:0}) [2024-10-16 14:57:32.789902] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.789919] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.789935] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.789944] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.789953] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=3] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.789961] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.789973] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.789984] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.790476] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EF1-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:32.790494] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EF1-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:32.790622] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.790648] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.790662] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:32.791642] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:32.791655] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:32.791839] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15D8-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:32.792895] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2411, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:32.792907] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=304, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:32.792932] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:32.798223] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061852796548000, start_id=1729061852796548000, end_id=1729061852796548001) [2024-10-16 14:57:32.799854] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=41][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.818708] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:32.819184] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061848803887, last_first_register_time_us=1722494157071428, register_time_us=1729061848803887, curr_time_us=1729061852819182) [2024-10-16 14:57:32.819222] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=13] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061852819198, last_submit_register_req_time_us=1729061852819198, register_parent_reason=3) [2024-10-16 14:57:32.819230] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:32.820085] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061852818324000, start_id=1729061852818324000, end_id=1729061852818324001) [2024-10-16 14:57:32.824503] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1001}, {id:1}]) [2024-10-16 14:57:32.830326] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=49] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:32.830924] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:32.830966] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061844396711000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:32.831011] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=44] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061844396711000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1, freeze_info:{latest_snapshot_gc_scn:{val:1729061844396711000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}}) [2024-10-16 14:57:32.836151] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.852085] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=7] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3913, mem_used=8318976) [2024-10-16 14:57:32.856818] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76C3-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1817, avg_group_log_size=181) [2024-10-16 14:57:32.875747] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=39] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:32.876366] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:32.876425] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061851830492000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:32.876478] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=51] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061851830492000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1004, freeze_info:{latest_snapshot_gc_scn:{val:1729061851830492000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}}) [2024-10-16 14:57:32.877395] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=51] get number(ret=0, service_type_=0, range=1, base_id=1729061852872027000, start_id=1729061852872027000, end_id=1729061852872027001) [2024-10-16 14:57:32.877458] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=21, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:32.877471] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10632, avg_group_log_size=625) [2024-10-16 14:57:32.877479] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10162) [2024-10-16 14:57:32.877486] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:32.878204] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=730, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:32.878219] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10040) [2024-10-16 14:57:32.878231] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=191, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:32.878252] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10040) [2024-10-16 14:57:32.878261] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:32.879214] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F943C-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:32.879224] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F943C-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=3512, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:32.879228] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F943C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=1550, avg_log_submit_to_slide_time=3484) [2024-10-16 14:57:32.892004] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:32.893391] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41E2-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=13, avg_log_submit_to_flush_time=893, avg_log_submit_to_slide_time=1661) [2024-10-16 14:57:32.896698] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061852895606000, start_id=1729061852895606000, end_id=1729061852895606001) [2024-10-16 14:57:32.899163] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] [COMPACTION DAG STATUS] (start_time=1729061842897982, end_time=1729061852899162, dag_status={}) [2024-10-16 14:57:32.899176] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:32.899184] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:32.899188] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:32.899192] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:32.899195] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:32.899198] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:32.899202] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:32.899205] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:32.899208] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:32.899212] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899218] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899222] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899226] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899230] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899234] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899238] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899242] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899248] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899254] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899263] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899269] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899275] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899280] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899285] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899290] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899296] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899300] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899306] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899312] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899316] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899320] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899323] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899327] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899335] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899339] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899342] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899346] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899350] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899354] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899357] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899361] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899365] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899369] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899373] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899377] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899381] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899385] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899388] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899392] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899404] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899407] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899411] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899415] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899419] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899422] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899426] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899429] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899433] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899437] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899441] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899445] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899449] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899452] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899456] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899460] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:32.899464] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:32.899468] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:32.899472] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:32.899474] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:32.899476] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:32.899479] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:32.899481] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:32.899484] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:32.899486] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:32.899488] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:32.899491] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:32.899906] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.900766] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:32.913302] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD7-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061852880974, keepalive_ts:-1}) [2024-10-16 14:57:32.913323] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD7-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061852880974, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:32.918117] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3545, avg_group_log_size=354) [2024-10-16 14:57:32.919027] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2642, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:32.919043] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1301, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:32.921373] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=9] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:32.921752] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061848892970, last_first_register_time_us=1722494150998907, register_time_us=1729061848892970, curr_time_us=1729061852921751) [2024-10-16 14:57:32.921779] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=11] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061852921764, last_submit_register_req_time_us=1729061852921764, register_parent_reason=3) [2024-10-16 14:57:32.921787] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:32.926631] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106820][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2013-0-0] [lt=13] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:32.926649] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106820][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2013-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:32.928295] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=28] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:32.928341] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=9] ====== tenant freeze timer task ====== [2024-10-16 14:57:32.936286] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:32.938474] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:57:32.938497] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=21] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:32.938504] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:32.938509] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:32.940844] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:57:32.941186] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=340] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:32.941230] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=2531, mem_used=6239232) [2024-10-16 14:57:32.946907] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061852945921000, start_id=1729061852945921000, end_id=1729061852945921001) [2024-10-16 14:57:32.947133] INFO [COMMON] print_io_status (ob_io_manager.cpp:1583) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS](tenant_id=1002, io_status_log=group_id: 0, group_name: OTHER_GROUPS, mode: write, size: 22.55, iops: 26356.87, rt: 720.13; ) [2024-10-16 14:57:32.947157] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4259.84, iops: 24.51, rt: 178.28; sys_group_name: SHARED_BLOCK_ReaderWriter_IO, mode: read, size: 4232.53, iops: 147.06, rt: 536.45; sys_group_name: SHARED_BLOCK_ReaderWriter_IO, mode: write, size: 4421.08, iops: 61.76, rt: 517.27; sys_group_name: SHARED_MACRO_BLOCK_MGR_IO, mode: read, size: 27368.73, iops: 21.57, rt: 2083.09; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 5266.29, iops: 6.86, rt: 3988.00; ) [2024-10-16 14:57:32.947170] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:32.947205] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.92, rt: 829.75; ) [2024-10-16 14:57:32.947210] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:32.947218] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:32.948336] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=3] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:32.953350] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=21] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:32.955740] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76C5-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:32.955752] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76C5-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=41459, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:32.955775] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1817, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:32.957354] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:32.957368] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:32.957376] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=157) [2024-10-16 14:57:32.961021] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:32.961050] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2257, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:32.961070] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=251, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:32.969653] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=18] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:32.969888] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:32.972710] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Cache wash clean map node details(ret=0, clean_node_count=39, clean_time=0, clean_start_pos=754968, clean_num=62914) [2024-10-16 14:57:32.973137] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BA3-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:32.973379] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=590, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:32.973392] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1049, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:32.973421] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:32.980257] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=20] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:32.981243] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061852980439000, start_id=1729061852980439000, end_id=1729061852980439001) [2024-10-16 14:57:32.981531] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061852981530, bytes: 4446810209964, bw: 0.483971 MB/s, add_ts: 1003435, add_bytes: 509224 [2024-10-16 14:57:32.982318] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:32.983102] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:32.983127] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:32.983147] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:32.983206] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061869029444000, latest_log_ts:{val:1729061852392712001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:32.983249] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=39] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:32.983267] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:57:32.983274] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061852392712001, v:0}) [2024-10-16 14:57:32.983278] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:32.983282] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:32.983288] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:32.983298] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:57:32.983306] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:32.983311] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:32.986421] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EF5-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497328, log_task={header:{begin_lsn:{lsn:207589536739}, end_lsn:{lsn:207589536861}, log_id:218497328, min_scn:{val:1729061852636039003, v:0}, max_scn:{val:1729061852636039003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589536617}, prev_proposal_id:81, committed_end_lsn:{lsn:207589536739}, data_checksum:940009244, accum_checksum:3376733553, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061852881556, freeze_ts:1729061852881556, submit_ts:1729061852881560, flushed_ts:1729061852882148, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:588}, fs_cb_begin_ts=1729061852986415, log_life_time=104859) [2024-10-16 14:57:32.986476] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:32.986490] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598776, miss_cnt:9, cache_read_size:149777901302, hit_ratio:"9.999999345924457383e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:32.992109] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:32.992124] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:32.994167] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=696, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:32.994180] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=155, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:32.994210] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:32.996856] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=14] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:32.997175] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=19] PNIO [ratelimit] time: 1729061852997173, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007941, add_bytes: 0 [2024-10-16 14:57:32.998841] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061852997221000, start_id=1729061852997221000, end_id=1729061852997221001) [2024-10-16 14:57:32.998934] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:32.998943] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=6] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27208900, tenant_id=1004) [2024-10-16 14:57:32.998951] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:33.000008] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.003572] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=16] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:33.010126] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2055) [21603][T1_MediumLoop][T1][YB4A0A000407-00061E99739F3875-0-0] [lt=10] start schedule all tablet merge(merge_version=1729015200372135000, medium_ls_tablet_iter_={ls_idx:-1, ls_ids:[{id:1}], tablet_idx:0, tablet_ids_.count():0, schedule_tablet_cnt:0, max_batch_tablet_cnt:50000}) [2024-10-16 14:57:33.010323] INFO [STORAGE] get_adaptive_merge_reason (ob_partition_merge_policy.cpp:1751) [21603][T1_MediumLoop][T1][YB4A0A000407-00061E99739F3875-0-0] [lt=16] Check tablet adaptive merge reason(ret=0, ls_id={id:1}, tablet_id={id:60476}, reason=0, tablet_analyzer={tablet_stat:{ls_id:1, tablet_id:60476, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, total_tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_small_tenant:false, boost_factor:"1.000000000000000000e+00"}, crazy_medium_flag=false) [2024-10-16 14:57:33.023578] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:57:33.024518] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061853020633000, start_id=1729061853020633000, end_id=1729061853020633001) [2024-10-16 14:57:33.037657] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA877-0-0] [lt=7] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:33.037682] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA877-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061852636039004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061852636039004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:33.039876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061853036726000, start_id=1729061853036726000, end_id=1729061853036726001) [2024-10-16 14:57:33.039896] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=677, get_gts_with_stc_cnt=29, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=28, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:33.039911] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=542, get_gts_with_stc_cnt=11, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:33.039919] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=3] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061853039915], gts:1729061853036726000, latest_srr:[mts=1729061853039915]}) [2024-10-16 14:57:33.041833] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.041868] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:33.041886] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:33.041898] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291891603, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:33.041967] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [22811][T1004_Compactio][T1004][YB4A0A000407-00061E99743D0D6E-0-0] [lt=12] InfoPoolResizeTask(cost_ts=8) [2024-10-16 14:57:33.049616] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=3][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:114618429392}], scn_array=[cnt:1, {val:1729061853020633000, v:0}], curr_size=592, accum_size=1184, time_cost=24762) [2024-10-16 14:57:33.050131] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=24] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257718, offset=39020612}, end_cursor:ObLogCursor{file_id=197, log_id=49257719, offset=39020895}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b20050, len:1980, log_data_len:283, seq:49257718, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:33.050207] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601373, offset=9617656}, end_cursor:ObLogCursor{file_id=192, log_id=47601374, offset=9617940}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d10050, len:3848, log_data_len:284, seq:47601373, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:33.051277] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2809, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:33.051296] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1261, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:33.051328] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:33.052659] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA877-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061853041952000, start_id=1729061853041952000, end_id=1729061853041952001) [2024-10-16 14:57:33.052831] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD10E-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:33.052842] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD10E-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=45261, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:33.052846] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD10E-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=8, avg_log_submit_to_flush_time=3759, avg_log_submit_to_slide_time=45253) [2024-10-16 14:57:33.052844] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:33.052860] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656865, miss_cnt:17, cache_read_size:8050154232, hit_ratio:"9.999997286810409225e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:33.055313] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561190, miss_cnt:1, cache_read_size:152590666599, hit_ratio:"9.999999928346842148e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:33.057563] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA49B-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061853052697000, start_id=1729061853052697000, end_id=1729061853052697001) [2024-10-16 14:57:33.067918] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=21][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:114618429984}], scn_array=[cnt:1, {val:1729061853052697000, v:0}], curr_size=1142, accum_size=2326, time_cost=10213) [2024-10-16 14:57:33.069094] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:33.069111] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:33.071517] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB366-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061853068336000, start_id=1729061853068336000, end_id=1729061853068336001) [2024-10-16 14:57:33.072556] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=8] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601374, offset=9617940}, end_cursor:ObLogCursor{file_id=192, log_id=47601375, offset=9618214}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cb4050, len:3564, log_data_len:274, seq:47601374, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:33.072617] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:33.072640] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:33.073147] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650128, miss_cnt:32, cache_read_size:8050406851, hit_ratio:"9.999994892271624192e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:33.078131] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [21603][T1_MediumLoop][T1][YB4A0A000407-00061E99739F3875-0-0] [lt=29] batch clear tablet stat in ls(ret=0, ls_id={id:1}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:57:33.078148] INFO [STORAGE.COMPACTION] after_schedule_tenant_medium (ob_tenant_tablet_scheduler.cpp:2161) [21603][T1_MediumLoop][T1][YB4A0A000407-00061E99739F3875-0-0] [lt=12] finish schedule all tablet merge(merge_version=1729015200372135000, schedule_stats_={schedule_dag_cnt:0, submit_clog_cnt:0, finish_cnt:1693, wait_rs_validate_cnt:0}, tenant_merge_finish=true, merged_version_=1729015200372135000, is_scan_all_tablet_finish=true, time_guard=) [2024-10-16 14:57:33.078159] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:103) [21603][T1_MediumLoop][T1][YB4A0A000407-00061E99739F3875-0-0] [lt=10] MediumLoopTask(cost_ts=68057) [2024-10-16 14:57:33.078171] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [21603][T1_MediumLoop][T1][YB4A0A000407-00061E99739F3876-0-0] [lt=4] MediumCheckTask(cost_ts=6) [2024-10-16 14:57:33.081099] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F81-0-0] [lt=13] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:200067}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328401377000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:33.081233] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F81-0-0] [lt=13] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:200067}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:33.083060] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:33.083071] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151784220, tenant_id=1) [2024-10-16 14:57:33.083084] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:33.084437] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257719, offset=39020895}, end_cursor:ObLogCursor{file_id=197, log_id=49257720, offset=39021177}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b94050, len:1697, log_data_len:282, seq:49257719, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:33.086272] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF229A-0-0] [lt=11] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846989}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328873906000, delete_commit_version:-1}, snapshot_version=1729061853036726000) [2024-10-16 14:57:33.086322] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF229A-0-0] [lt=14] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846989}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:33.086448] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA877-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061853083105000, start_id=1729061853083105001, end_id=1729061853083105002) [2024-10-16 14:57:33.090808] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=27] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601375, offset=9618214}, end_cursor:ObLogCursor{file_id=192, log_id=47601376, offset=9618488}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c0c050, len:3290, log_data_len:274, seq:47601375, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:33.090942] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F85-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061853083105000, start_id=1729061853083105000, end_id=1729061853083105001) [2024-10-16 14:57:33.091522] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601376, offset=9618488}, end_cursor:ObLogCursor{file_id=192, log_id=47601377, offset=9618762}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c84050, len:3016, log_data_len:274, seq:47601376, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:33.093539] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2013248, replace_num=31457) [2024-10-16 14:57:33.093552] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:33.094069] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:33.094095] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:33.094828] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:33.097482] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061853094738000, start_id=1729061853094738000, end_id=1729061853094738001) [2024-10-16 14:57:33.097905] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA1-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061853064799, keepalive_ts:-1}) [2024-10-16 14:57:33.097918] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA1-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061853064799, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:33.097950] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA2-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061853064855, keepalive_ts:-1}) [2024-10-16 14:57:33.097954] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107054][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA2-0-0] [lt=4] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061853064855, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:33.100077] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.100393] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1729061852557368003, v:0}, new_max_version={val:1729061852557368003, v:0}, last_min_version={val:1729061852134673005, v:0}, last_max_version={val:1729061852134673005, v:0}, total_time_us=3755, delta_us=543024) [2024-10-16 14:57:33.100409] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1729061852557368003, v:0}, min_version={val:1729061852557368003, v:0}, max_version={val:1729061852557368003, v:0}, in_service=true, leader_epoch=17) [2024-10-16 14:57:33.100539] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F85-0-0] [lt=7] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:200070}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328873906000, delete_commit_version:-1}, snapshot_version=1729061853083105000) [2024-10-16 14:57:33.100595] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F85-0-0] [lt=13] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:200070}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:33.100649] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F85-0-0] [lt=14] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846983}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328401377000, delete_commit_version:-1}, snapshot_version=1729061853083105000) [2024-10-16 14:57:33.100670] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F85-0-0] [lt=5] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846983}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:33.104469] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21575][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:33.104523] WDIAG read_multi_row (ob_table_access_helper.h:238) [21575][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:33.105230] INFO delete_row (ob_table_access_helper.h:312) [21575][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061853104530) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061853104530) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:33.107903] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257720, offset=39021177}, end_cursor:ObLogCursor{file_id=197, log_id=49257721, offset=39021459}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bac050, len:1415, log_data_len:282, seq:49257720, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:33.108043] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257721, offset=39021459}, end_cursor:ObLogCursor{file_id=197, log_id=49257722, offset=39021741}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d20050, len:1133, log_data_len:282, seq:49257721, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:33.112036] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:33.112048] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:33.122474] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=14] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97827692544) [2024-10-16 14:57:33.125706] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=11] current monitor number(seq_=-1) [2024-10-16 14:57:33.127289] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:33.129262] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1536, mem_used=4159488) [2024-10-16 14:57:33.136075] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0C00-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061853135708000, start_id=1729061853135708000, end_id=1729061853135708001) [2024-10-16 14:57:33.136202] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0C02-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061853135708000, start_id=1729061853135708000, end_id=1729061853135708001) [2024-10-16 14:57:33.136262] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0C04-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061853135708000, start_id=1729061853135708000, end_id=1729061853135708001) [2024-10-16 14:57:33.136426] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B978-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061853135708000, start_id=1729061853135708000, end_id=1729061853135708001) [2024-10-16 14:57:33.140325] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:33.141887] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.150093] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061853148402000, start_id=1729061853148402000, end_id=1729061853148402001) [2024-10-16 14:57:33.150174] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:33.150187] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:33.155276] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76C9-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:33.171852] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D63-0-0] [lt=2][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510321040246, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410584, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:33.173025] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:33.178932] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5023-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:33.178946] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5023-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:33.181541] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061853180063000, start_id=1729061853180063000, end_id=1729061853180063001) [2024-10-16 14:57:33.183738] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EF9-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:33.183750] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84EF9-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=65090, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:33.183791] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=26] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=118, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:33.183797] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:33.193935] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:33.193949] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:33.199077] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061853198279000, start_id=1729061853198279000, end_id=1729061853198279001) [2024-10-16 14:57:33.200137] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.224679] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061853224313000, start_id=1729061853224313000, end_id=1729061853224313001) [2024-10-16 14:57:33.241949] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.255927] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] succ to submit log task to replay service(to_submit_lsn={lsn:213058226917}, to_submit_scn={val:1729061853135708001, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061853255897, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058226983}, next_to_submit_scn_:{val:1729061853135708002, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058226861}, end_lsn:{lsn:213058226983}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061853135708001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061853135708001, v:0}, data_checksum:3348837161, flag:1}}, init_mode_version:0, accumulate_checksum:167253827, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061853135708001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561193, miss_cnt:1, cache_read_size:152590666965, hit_ratio:"9.999999928346844369e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:33.268428] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BA9-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:33.269219] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:33.269245] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:33.269274] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:33.269280] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:33.269520] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:33.273845] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1032-0-0] [lt=8] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:33.293621] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:33.297545] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061853296641000, start_id=1729061853296641000, end_id=1729061853296641001) [2024-10-16 14:57:33.297619] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=11836, avg_group_log_size=622) [2024-10-16 14:57:33.297630] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11365) [2024-10-16 14:57:33.297879] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=25] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=1468, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:33.297892] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11836) [2024-10-16 14:57:33.297902] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=464, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:33.297926] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11365) [2024-10-16 14:57:33.297934] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:33.299010] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797BBC-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:33.299019] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797BBC-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4563, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:33.299023] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797BBC-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=353, avg_log_gen_to_submit_time=379, avg_log_submit_to_flush_time=2523, avg_log_submit_to_slide_time=4184) [2024-10-16 14:57:33.300635] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.314047] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6416, mem_used=12478464) [2024-10-16 14:57:33.321897] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=14] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:33.323679] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3423) [2024-10-16 14:57:33.324007] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3423) [2024-10-16 14:57:33.324033] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3423) [2024-10-16 14:57:33.324190] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FCC-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:33.324197] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FCC-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5378, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:33.342044] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.350153] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061853349559000, start_id=1729061853349559000, end_id=1729061853349559001) [2024-10-16 14:57:33.353389] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA9-0-0] [lt=12] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:33.353406] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AA9-0-0] [lt=14] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=18) [2024-10-16 14:57:33.355537] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1524, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:33.355564] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=513, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:33.355579] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:33.358367] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=10] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:33.358484] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=10] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:33.360464] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107054][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=5] worker thread exit(tenant_->id()=1004, workers_.get_size()=4) [2024-10-16 14:57:33.360479] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107054][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=11] tenant thread end_run(id_=1004, ret=0, thread_count_=193) [2024-10-16 14:57:33.369396] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1053, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:33.369408] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:33.369423] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:33.369645] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD77-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:33.369651] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD77-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1717, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:33.371230] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:33.371405] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD78-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:33.371412] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD78-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3054, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:33.371416] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD78-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=12, avg_log_submit_to_flush_time=1235, avg_log_submit_to_slide_time=3042) [2024-10-16 14:57:33.373300] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:33.380006] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=17] blacklist_loop exec finished(cost_time=14, is_enabled=true, send_cnt=0) [2024-10-16 14:57:33.381669] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061853380769000, start_id=1729061853380769000, end_id=1729061853380769001) [2024-10-16 14:57:33.382867] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:33.393997] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:33.394972] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=6] query dest_info_ succ(dest_info=[], archive_dest_changed=false) [2024-10-16 14:57:33.394989] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=15] finish to resize log external storage handler(current_ts=1729061853394987, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:33.395065] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41F1-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=4, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=830, avg_log_submit_to_slide_time=1530) [2024-10-16 14:57:33.396149] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=10] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:33.399221] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061853398426000, start_id=1729061853398426000, end_id=1729061853398426001) [2024-10-16 14:57:33.400696] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.408855] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3825-0-0] [lt=16] MediumCheckTask(cost_ts=9) [2024-10-16 14:57:33.424898] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061853424886000, start_id=1729061853424886000, end_id=1729061853424886001) [2024-10-16 14:57:33.442117] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.455464] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76CF-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1939) [2024-10-16 14:57:33.460024] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=40] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:33.460597] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=20] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:33.460689] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=55] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061848719447000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:33.461407] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A77-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:33.461423] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A77-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:33.468928] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BAD-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=12, avg_log_batch_cnt=0, total_group_log_size=1454, avg_group_log_size=121) [2024-10-16 14:57:33.469535] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:33.486074] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:33.486083] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:33.486093] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:33.486099] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:33.486102] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:33.486105] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:33.486108] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:33.486110] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:33.486112] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:33.486115] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:33.486103] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED0-0-0] [lt=7] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:33.486118] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:33.486122] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED0-0-0] [lt=15] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:33.489586] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:33.489614] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=23] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:33.489632] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] cannot_recycle_log_size statistics(cannot_recycle_log_size=70596151, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:33.489708] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:468) [21814][T1001_SSTableDe][T1001][Y0-0000000000000000-0-0] [lt=24] skip defragment task(ret=0, need_defragment=false) [2024-10-16 14:57:33.489723] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [21814][T1001_SSTableDe][T1001][Y0-0000000000000000-0-0] [lt=14] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=10) [2024-10-16 14:57:33.489728] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [21814][T1001_SSTableDe][T1001][Y0-0000000000000000-0-0] [lt=5] successfully defragment data blocks(rewrite_cnt=0) [2024-10-16 14:57:33.489734] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [21814][T1001_SSTableDe][T1001][Y0-0000000000000000-0-0] [lt=3] tablet skip filter destructed(total_cnt=0, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105368325, total_skip_cnt=310297, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:57:33.493703] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:33.494150] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:33.494170] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:33.494637] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=541, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:33.494651] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=97, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:33.494663] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:33.494691] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:33.495501] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4012-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:33.495515] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4012-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1340, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:33.497737] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061853494778000, start_id=1729061853494778000, end_id=1729061853494778001) [2024-10-16 14:57:33.498695] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=12] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:33.498715] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info(lsn={lsn:14538478376}, scn={val:1729061853135708004, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061853469226, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538478376}, next_to_submit_scn_:{val:1729061853135708004, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538478254}, end_lsn:{lsn:14538478376}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061853135708003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061853135708003, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:3653225388, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061853135708003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650133, miss_cnt:32, cache_read_size:8050407461, hit_ratio:"9.999994892272031644e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:33.498761] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=41] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:33.498784] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] Garbage Collector is running(seq_=656494, gc_interval=10000000) [2024-10-16 14:57:33.498801] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:33.498807] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=4] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:33.499141] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:33.499991] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=7] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:33.500011] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=12] gc_check_member_list_ cost time(ret=0, time_us=1222) [2024-10-16 14:57:33.500019] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:33.500689] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:33.500766] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.501377] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=62] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:33.502011] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=13] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:33.502570] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=12] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:33.502588] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=16] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:33.502609] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=20] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:33.502614] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:33.502622] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:33.502628] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=40) [2024-10-16 14:57:33.502634] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=4] ObSafeDestroyHandler start process [2024-10-16 14:57:33.504913] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:33.505858] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=14] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:33.505896] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=37] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:33.505924] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=23] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:33.506017] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=16] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061865590307000, latest_log_ts:{val:1729061848964072001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:33.506087] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=66] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:33.506114] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:33.506126] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=10] get rec log scn(service_type_=0, rec_log_ts={val:1729061848964072001, v:0}) [2024-10-16 14:57:33.506131] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:33.506137] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:33.506146] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:33.506159] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:33.506175] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:33.506186] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=9] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:33.511020] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BB-0-0] [lt=21] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:33.511047] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BB-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061853135708004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061853135708004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:33.512115] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9BB-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061853511613000, start_id=1729061853511613000, end_id=1729061853511613001) [2024-10-16 14:57:33.513090] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9A279FA9BB-0-0] [lt=19] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.054953432083129883e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=302, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.073582535704922066e+03) [2024-10-16 14:57:33.516205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9BB-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061853515855000, start_id=1729061853515855000, end_id=1729061853515855001) [2024-10-16 14:57:33.516881] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:33.539870] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=904810496, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=168931328, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:33.539939] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=32] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:33.539955] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=14] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:33.539998] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=967602176, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=106139648, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:33.540037] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D2E-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:33.540043] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=13] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:33.540051] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:33.540055] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D2E-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1333, avg_group_log_size=121) [2024-10-16 14:57:33.540064] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D2E-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1333) [2024-10-16 14:57:33.540094] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=940462080, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=133279744, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:33.540140] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=12] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:33.540153] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=13] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011859, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:33.540194] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=963133440, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=110608384, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:33.540236] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061853539570000, start_id=1729061853539570000, end_id=1729061853539570001) [2024-10-16 14:57:33.540243] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=14] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:33.540251] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:33.540256] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:33.540269] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061853540263], gts:1729061853539570000, latest_srr:[mts=1729061853540263]}) [2024-10-16 14:57:33.540329] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=14] submit log success(service_type_=0, last_id=1729061870098662000, limited_id=1729061870098662000) [2024-10-16 14:57:33.540334] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=4] submit log with lock success(service_type_=0, last_id=1729061870098662000, limited_id=1729061870098662000) [2024-10-16 14:57:33.541338] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] update id meta(ret=0, service_type=0, limited_id=1729061870098662000, latest_scn={val:1729061853539570001, v:0}, *this={tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850523518004, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870098662000, latest_log_ts:{val:1729061853539570001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:33.541393] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=54] handle submit callback(service_type_=0, success=true, used_ts=1063, log_ts={val:1729061853539570001, v:0}, limited_id=1729061870098662000) [2024-10-16 14:57:33.541428] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=34] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:33.541438] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=9] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:33.542136] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.550259] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061853550171000, start_id=1729061853550171000, end_id=1729061853550171001) [2024-10-16 14:57:33.555659] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1939) [2024-10-16 14:57:33.563547] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C48-0-0] [lt=5] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:33.563584] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C48-0-0] [lt=23] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061853135708004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061853135708004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:33.564686] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C48-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061853563898000, start_id=1729061853563898000, end_id=1729061853563898001) [2024-10-16 14:57:33.567420] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21935][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C48-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061853567352000, start_id=1729061853567352000, end_id=1729061853567352001) [2024-10-16 14:57:33.568514] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.568534] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.568550] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.569071] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.573591] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:33.578029] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D3-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061853569204, keepalive_ts:-1}) [2024-10-16 14:57:33.578047] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D3-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061853569204, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:33.581756] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061853581621000, start_id=1729061853581621000, end_id=1729061853581621001) [2024-10-16 14:57:33.581823] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26148][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB19CC6EB-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:33.584095] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:33.584163] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1001}, {id:1003}]) [2024-10-16 14:57:33.586871] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:33.586889] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:33.595077] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D4-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061853562857, keepalive_ts:-1}) [2024-10-16 14:57:33.595095] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D4-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061853562857, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:33.599368] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061853598621000, start_id=1729061853598621000, end_id=1729061853598621001) [2024-10-16 14:57:33.600822] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.608241] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=4817, mem_used=10398720) [2024-10-16 14:57:33.624780] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061853623915000, start_id=1729061853623915000, end_id=1729061853623915001) [2024-10-16 14:57:33.636755] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B97A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061853636618000, start_id=1729061853636618000, end_id=1729061853636618001) [2024-10-16 14:57:33.637014] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0C08-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061853636618000, start_id=1729061853636618000, end_id=1729061853636618001) [2024-10-16 14:57:33.637046] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B97E-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061853636618000, start_id=1729061853636618000, end_id=1729061853636618001) [2024-10-16 14:57:33.639561] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0C07-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061853638892000, start_id=1729061853638892000, end_id=1729061853638892001) [2024-10-16 14:57:33.642066] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:33216819320}, base_lsn={lsn:33149755392}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:33.642132] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=65] get_replay_process success(id={id:1}, replayed_log_size=67063928, unreplayed_log_size=0) [2024-10-16 14:57:33.642150] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=63, last_replayed_log_size(MB)=63, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:33.642199] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:33.642212] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:33.642217] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.655983] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:33.656010] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=25] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:33.656017] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:33.656022] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:33.658173] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] Dumping All Cache Objs(alloc_obj_list.count()=104, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}]) [2024-10-16 14:57:33.658322] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=146] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:33.658425] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:468) [22707][T1004_SSTableDe][T1004][Y0-0000000000000000-0-0] [lt=10] skip defragment task(ret=0, need_defragment=false) [2024-10-16 14:57:33.658438] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [22707][T1004_SSTableDe][T1004][Y0-0000000000000000-0-0] [lt=12] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=18) [2024-10-16 14:57:33.658443] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [22707][T1004_SSTableDe][T1004][Y0-0000000000000000-0-0] [lt=5] successfully defragment data blocks(rewrite_cnt=0) [2024-10-16 14:57:33.658449] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [22707][T1004_SSTableDe][T1004][Y0-0000000000000000-0-0] [lt=5] tablet skip filter destructed(total_cnt=0, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105368325, total_skip_cnt=310297, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:57:33.665928] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:33.669404] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:33.669415] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:57:33.669419] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:33.669433] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=2] get_max_applied_scn(scn={val:1729061853539570000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493455863}, palf_committed_end_scn_:{val:1729061853539570000, v:0}, last_check_scn_:{val:1729061853539570000, v:0}, max_applied_cb_scn_:{val:1729061853539570000, v:0}}) [2024-10-16 14:57:33.669450] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:33.670097] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=43] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850545633002, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061860721265000, latest_log_ts:{val:1729061844127564001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:33.670199] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=49] get_max_applied_scn(scn={val:1729061853539570000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645409382}, palf_committed_end_scn_:{val:1729061853539570000, v:0}, last_check_scn_:{val:1729061853539570000, v:0}, max_applied_cb_scn_:{val:1729061853539570000, v:0}}) [2024-10-16 14:57:33.670209] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:33.670842] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850545633002, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:33.671484] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850545633002, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:33.671514] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=25] get_min_unreplayed_log_info(lsn={lsn:14352364734}, scn={val:1729061853636618001, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061853641375, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352364734}, next_to_submit_scn_:{val:1729061853636618001, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352364612}, end_lsn:{lsn:14352364734}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061853636618000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061853636618000, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:345031534, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061853636618000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656871, miss_cnt:17, cache_read_size:8050154955, hit_ratio:"9.999997286810669017e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:33.672130] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850545633002, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:33.676280] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:33.677025] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061853676825000, start_id=1729061853676825000, end_id=1729061853676825001) [2024-10-16 14:57:33.682269] INFO [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:262) [22249][T1002_ArcSender][T1002][YB4A0A000407-00061E996F9960C1-0-0] [lt=10] ObArchiveSender is running(thread_index=1) [2024-10-16 14:57:33.684060] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F03-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:33.693780] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:33.694523] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1348) [2024-10-16 14:57:33.694941] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41FA-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:33.694951] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C41FA-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1408, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:33.697896] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061853696848000, start_id=1729061853696848000, end_id=1729061853696848001) [2024-10-16 14:57:33.697959] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:33.697992] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:33.698164] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:33.698745] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=4] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:33.700876] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.701429] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=17] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:33.723652] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:33.724273] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=13, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:33.724577] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FD0-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=852, avg_log_submit_to_slide_time=1089) [2024-10-16 14:57:33.739952] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=15] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:33.742233] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.747686] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D65-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:33.747717] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D65-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061853135708004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061853135708004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:33.748756] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D65-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061853748423000, start_id=1729061853748423000, end_id=1729061853748423001) [2024-10-16 14:57:33.750457] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061853748423000, start_id=1729061853748423000, end_id=1729061853748423001) [2024-10-16 14:57:33.752527] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D65-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061853751796000, start_id=1729061853751796000, end_id=1729061853751796001) [2024-10-16 14:57:33.755663] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:33.755686] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:33.755693] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:33.768754] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BB3-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:33.768771] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BB3-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21425, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:33.768776] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BB3-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=662, avg_log_submit_to_slide_time=21418) [2024-10-16 14:57:33.768833] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=36] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:33.768864] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:33.768869] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=5] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:33.769089] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:33.769537] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:33.769559] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:33.770770] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:33.772581] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:33.773879] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:33.781221] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=8] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:33.782121] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061853781669000, start_id=1729061853781669000, end_id=1729061853781669001) [2024-10-16 14:57:33.788622] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.788645] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.788661] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.788671] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.788681] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.788688] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.788704] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.788718] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.789493] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.789518] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.789535] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:33.794314] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1226, avg_group_log_size=122) [2024-10-16 14:57:33.794332] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1226) [2024-10-16 14:57:33.795009] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=11] [APPLY STAT CB APPEND COST TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=43, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:57:33.795021] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=1030, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:57:33.795025] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:57:33.795028] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=-604, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:57:33.795031] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=469, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:57:33.799521] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061853798597000, start_id=1729061853798597000, end_id=1729061853798597001) [2024-10-16 14:57:33.800917] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.804314] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=17] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:57:33.819742] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:33.824094] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] no need to update min/max version(new_version={val:1729061853424886000, v:0}, min_version={val:1729061853424886000, v:0}, max_version={val:1729061853424886000, v:0}) [2024-10-16 14:57:33.824092] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=13] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061849804671, last_first_register_time_us=1722494157071428, register_time_us=1729061849804671, curr_time_us=1729061853824090) [2024-10-16 14:57:33.824147] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=21] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061853824118, last_submit_register_req_time_us=1729061853824118, register_parent_reason=3) [2024-10-16 14:57:33.824159] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:33.824604] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1001}, {id:1003}]) [2024-10-16 14:57:33.827860] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:33.827876] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:33.827888] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52332162, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:33.842288] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.849259] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=15][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:33.852148] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3976, mem_used=8318976) [2024-10-16 14:57:33.870428] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=22, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:57:33.870443] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=15] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=2405, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:57:33.870447] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:57:33.870451] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=-656, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:57:33.870454] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=1771, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:57:33.870498] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD128-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=543, avg_log_submit_to_slide_time=982) [2024-10-16 14:57:33.884162] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F07-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:33.884190] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F07-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:33.884434] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=330, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:33.884457] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=33, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:33.884496] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:33.891184] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:14352365034}, scn={val:1729061853636618003, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:33.891186] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:19, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061853891121, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352365090}, next_to_submit_scn_:{val:1729061853636618004, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352364978}, end_lsn:{lsn:14352365090}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061853636618003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061853636618003, v:0}, data_checksum:3700780717, flag:0}}, init_mode_version:0, accumulate_checksum:826692281, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061853636618003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656874, miss_cnt:17, cache_read_size:8050155311, hit_ratio:"9.999997286810800023e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14352365034}, scn_:{val:1729061853636618003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061853858321, first_handle_ts_:1729061853891185, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0ef5fff280, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:33.891339] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060051830492000) [2024-10-16 14:57:33.891351] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14352365034}, scn_:{val:1729061853636618003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061853858321, first_handle_ts_:1729061853891328, replay_cost_:21, retry_cost_:-1, read_log_buf_:0x2b0ef5fff290, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061853891121, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352365090}, next_to_submit_scn_:{val:1729061853636618004, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352364978}, end_lsn:{lsn:14352365090}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061853636618003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061853636618003, v:0}, data_checksum:3700780717, flag:0}}, init_mode_version:0, accumulate_checksum:826692281, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061853636618003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656874, miss_cnt:17, cache_read_size:8050155311, hit_ratio:"9.999997286810800023e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}, replay_queue_idx=17, ret=0) [2024-10-16 14:57:33.893857] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:33.895016] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F401E-0-0] [lt=15] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=504, avg_log_submit_to_slide_time=911) [2024-10-16 14:57:33.898062] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061853897294000, start_id=1729061853897294000, end_id=1729061853897294001) [2024-10-16 14:57:33.900841] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=10] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:33.900963] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.921472] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:33.924004] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=3423, avg_group_log_size=380) [2024-10-16 14:57:33.925103] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061853923831000, start_id=1729061853923831000, end_id=1729061853923831001) [2024-10-16 14:57:33.925163] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:33.925185] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=9572, avg_group_log_size=598) [2024-10-16 14:57:33.925193] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9572) [2024-10-16 14:57:33.925200] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:33.925380] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=22] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=2421, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:33.925392] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9572) [2024-10-16 14:57:33.925403] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=134, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:33.925425] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9572) [2024-10-16 14:57:33.925433] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:33.925505] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=561, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:33.925527] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=339, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:33.925840] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F944C-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=6, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:33.925855] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F944C-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=3776, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:33.925860] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F944C-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=2816, avg_log_submit_to_slide_time=3743) [2024-10-16 14:57:33.941300] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=8] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2561, mem_used=6239232) [2024-10-16 14:57:33.942349] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:33.950403] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061853949462000, start_id=1729061853949462000, end_id=1729061853949462001) [2024-10-16 14:57:33.950865] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15D2-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:33.950933] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15D4-0-0] [lt=11] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:33.951061] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 204.25; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 4096.00, iops: 5.98, rt: 94.67; ) [2024-10-16 14:57:33.951082] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:33.951121] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 376.00; ) [2024-10-16 14:57:33.951126] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:33.951138] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:33.955848] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76D9-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:33.960448] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107240][][T1][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1, ret=0, thread_count_=241) [2024-10-16 14:57:33.960466] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107240][][T1][Y0-0000000000000000-0-0] [lt=17] Init thread local success [2024-10-16 14:57:33.960488] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107240][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FD3-0-0] [lt=5] init point thread id with(&point=0x55e3435c68c0, idx_=745, point=[thread id=107240, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107240) [2024-10-16 14:57:33.960667] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] worker thread created(tenant_->id()=1, group_id_=1, token=1) [2024-10-16 14:57:33.970150] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1636) [2024-10-16 14:57:33.970151] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=465, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:33.970185] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=67, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:33.974182] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:33.982332] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061853981755000, start_id=1729061853981755000, end_id=1729061853981755001) [2024-10-16 14:57:33.982581] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=26] PNIO [ratelimit] time: 1729061853982580, bytes: 4446810589127, bw: 0.361219 MB/s, add_ts: 1001050, add_bytes: 379163 [2024-10-16 14:57:33.992689] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=8] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:33.992713] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=210) [2024-10-16 14:57:33.994440] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:33.994456] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:33.995101] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=293, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:33.995116] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=51, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:33.995129] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:33.999705] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061853999615000, start_id=1729061853999615000, end_id=1729061853999615001) [2024-10-16 14:57:34.001020] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.004431] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=9] PNIO [ratelimit] time: 1729061854004430, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007257, add_bytes: 0 [2024-10-16 14:57:34.017627] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:34.018349] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.018378] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.018410] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:34.018545] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850523518004, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870098662000, latest_log_ts:{val:1729061853539570001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:34.018598] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=50] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850523518004, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870098662000, latest_log_ts:{val:1729061853539570001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:34.018639] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=36] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:34.018649] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=0, rec_log_ts={val:1729061853539570001, v:0}) [2024-10-16 14:57:34.018656] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:34.018665] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:34.018681] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:34.018691] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:34.019399] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.019414] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.019425] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:34.019462] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850523518004, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:34.019490] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850523518004, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:34.019514] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:34.019519] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:34.019525] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:34.019530] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:34.019639] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=8] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:34.020215] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.020227] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.020237] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:34.020271] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850523518004, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:34.020298] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850523518004, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:34.020320] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:34.020325] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:34.020330] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:34.020335] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:34.021054] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.021066] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.021076] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:34.021108] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850523518004, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:34.021135] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061850523518004, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:34.021157] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:34.021162] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:34.021168] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:34.021173] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:34.021176] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:34.040821] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061854040663000, start_id=1729061854040663000, end_id=1729061854040663001) [2024-10-16 14:57:34.040854] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=17] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061854040851], gts:1729061854040663000, latest_srr:[mts=1729061854040851]}) [2024-10-16 14:57:34.042409] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.055775] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76DB-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:34.055791] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76DB-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19229, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:34.055819] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=66] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:34.055832] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561200, miss_cnt:1, cache_read_size:152590667819, hit_ratio:"9.999999928346847700e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:34.065020] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F91-0-0] [lt=28] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846990}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328873906000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:57:34.065127] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F91-0-0] [lt=20] refresh ddl data cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846990}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:34.066677] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=16] Garbage Collector is running(seq_=656491, gc_interval=10000000) [2024-10-16 14:57:34.066708] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=12] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:34.066715] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:34.068504] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:34.068808] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=12] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:34.068821] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] gc_check_member_list_ cost time(ret=0, time_us=2131) [2024-10-16 14:57:34.068830] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:34.069579] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=2] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:34.070178] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BB9-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:34.070263] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=11] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:34.070375] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=337, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:34.070389] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=282, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:34.070415] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:34.070991] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=39] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:34.071588] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=14] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:34.071606] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:34.071615] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:34.071621] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:34.071626] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:34.071631] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=25) [2024-10-16 14:57:34.071635] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:34.071639] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:133) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-10-16 14:57:34.071653] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:140) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=4] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-10-16 14:57:34.071841] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:34.071855] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:34.072234] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD12E-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1636) [2024-10-16 14:57:34.072284] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD12F-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:34.074157] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B7E3C9C8E-0-0] [lt=19] refresh tablet status cache(ret=0, ls_id={id:1}, tablet_id={id:394}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560287003454000, delete_commit_version:-1}, snapshot_version=1729061854040663000) [2024-10-16 14:57:34.086721] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1464, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:34.086743] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598788, miss_cnt:9, cache_read_size:149777902766, hit_ratio:"9.999999345924514005e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:34.086728] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [22325][T1003_COMMONLSS][T1003][YB4A0A000407-00061E9B3A19CB0C-0-0] [lt=23][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:34.093784] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601377, offset=9618762}, end_cursor:ObLogCursor{file_id=192, log_id=47601378, offset=9619046}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cbc050, len:2742, log_data_len:284, seq:47601377, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:34.093933] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:34.094802] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=24601196, this={dio_aligned_buf:{buf_write_offset:742, buf_padding_size:3354, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:93866933, truncate_used_ts:39944914}, log_block_size:67108864, dir_fd:173, io_fd:196}, aligned_buf_len=4096, aligned_block_offset=24600576, buf_len=122, total_write_size_=70420, total_write_size_after_dio_=573440, ob_pwrite_used_ts=54414, count=122, each_pwrite_cost=446) [2024-10-16 14:57:34.094821] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:34.094847] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:34.095498] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA49C-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061854094680000, start_id=1729061854094680000, end_id=1729061854094680001) [2024-10-16 14:57:34.096403] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601378, offset=9619046}, end_cursor:ObLogCursor{file_id=192, log_id=47601379, offset=9619320}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ce8050, len:2458, log_data_len:274, seq:47601378, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:34.098390] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA878-0-0] [lt=21] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:34.098412] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061854095741000, start_id=1729061854095741000, end_id=1729061854095741001) [2024-10-16 14:57:34.098416] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [29808][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA878-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061853638892005, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061853638892005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:34.099141] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=27] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601379, offset=9619320}, end_cursor:ObLogCursor{file_id=192, log_id=47601380, offset=9619594}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bf0050, len:2184, log_data_len:274, seq:47601379, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:34.099539] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1729061853539570001, v:0}, new_max_version={val:1729061853539570001, v:0}, last_min_version={val:1729061853135708004, v:0}, last_max_version={val:1729061853135708004, v:0}, total_time_us=1786, delta_us=559968) [2024-10-16 14:57:34.099554] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1729061853539570001, v:0}, min_version={val:1729061853539570001, v:0}, max_version={val:1729061853539570001, v:0}, in_service=true, leader_epoch=17) [2024-10-16 14:57:34.099541] WDIAG [MDS] ~ObOccamTimeGuard (ob_occam_time_guard.h:269) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B7E3C9C8E-0-0] [lt=14][errcode=0] cost too much time:ob_i_tablet_mds_interface.ipp:int oceanbase::storage::ObITabletMdsInterface::get_mds_data_from_tablet(const mds::DummyKey &, const share::SCN &, const int64_t, const common::ObFunction &) c..., (*this=|threshold=10.00ms|start at 14:57:34.741|137=0us|total=25.39ms) [2024-10-16 14:57:34.099565] WDIAG [MDS] ~ObOccamTimeGuard (ob_occam_time_guard.h:269) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B7E3C9C8E-0-0] [lt=22][errcode=0] cost too much time:ob_i_tablet_mds_interface.ipp:int oceanbase::storage::ObITabletMdsInterface::get_latest(OP &&, bool &) const [T = oceanbase::storage::ObTabletBindingMdsUserData, OP = oceanbase::storage::ReadBindingInfoOp], (*this=|threshold=10.00ms|start at 14:57:34.741|430=0us|442=1us|444=0us|451=2us|461=0us|472=25.39ms|total=25.40ms) [2024-10-16 14:57:34.099574] WDIAG [MDS] ~ObOccamTimeGuard (ob_occam_time_guard.h:269) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B7E3C9C8E-0-0] [lt=7][errcode=0] cost too much time:ob_i_tablet_mds_interface.cpp:int oceanbase::storage::ObITabletMdsInterface::get_latest_ddl_data(oceanbase::storage::ObTabletBindingMdsUserData &, bool &) const, (*this=|threshold=10.00ms|start at 14:57:34.741|123=0us|total=25.41ms) [2024-10-16 14:57:34.099579] INFO [STORAGE] check_snapshot_readable_with_cache (ob_tablet.cpp:7246) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B7E3C9C8E-0-0] [lt=4] refresh ddl data cache(ret=0, ls_id={id:1}, tablet_id={id:394}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:57:34.099864] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601380, offset=9619594}, end_cursor:ObLogCursor{file_id=192, log_id=47601381, offset=9619868}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c9c050, len:1910, log_data_len:274, seq:47601380, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:34.100107] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA878-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061854099689000, start_id=1729061854099689000, end_id=1729061854099689001) [2024-10-16 14:57:34.101581] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.102954] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA878-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061854102223000, start_id=1729061854102223000, end_id=1729061854102223001) [2024-10-16 14:57:34.103804] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:34.105472] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0F9F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061854104410000, start_id=1729061854104410000, end_id=1729061854104410001) [2024-10-16 14:57:34.108970] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257722, offset=39021741}, end_cursor:ObLogCursor{file_id=197, log_id=49257723, offset=39022024}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cd8050, len:851, log_data_len:283, seq:49257722, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:34.110681] WDIAG [STORAGE] refresh_queuing_mode (ob_tenant_tablet_stat_mgr.cpp:1125) [22621][T1004_TabletSta][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=0] failed to fetch table id from inner table, may be recycled or never exists, skip it(tablet_id={id:200408}) [2024-10-16 14:57:34.110715] INFO [STORAGE] refresh_queuing_mode (ob_tenant_tablet_stat_mgr.cpp:1160) [22621][T1004_TabletSta][T1004][Y0-0000000000000000-0-0] [lt=10] refresh queuing mode(ret=0, tenant_id=1004, stream_cnt=24, update_schema_cnt=23, cost_time=63) [2024-10-16 14:57:34.110721] INFO [STORAGE] runTimerTask (ob_tenant_tablet_stat_mgr.cpp:1210) [22621][T1004_TabletSta][T1004][Y0-0000000000000000-0-0] [lt=5] TenantTabletStatMgr refresh all tablet stream(MTL_ID()=1004, interval_step=1, global_iter_pool={is_inited:false, is_washing:false, is_disabled:false, get_cnt:0, tenant_id:1004, bucket_cnt:0, tenant_mem_user_limit:1073741824, tenant_mem_user_hold:384110592, cached_node_array:0x2b0e405b6360}) [2024-10-16 14:57:34.111144] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB367-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061854110931000, start_id=1729061854110931000, end_id=1729061854110931001) [2024-10-16 14:57:34.112696] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:34.112708] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:34.113251] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257723, offset=39022024}, end_cursor:ObLogCursor{file_id=197, log_id=49257724, offset=39022306}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ccc050, len:568, log_data_len:282, seq:49257723, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:34.115333] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=28] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257724, offset=39022306}, end_cursor:ObLogCursor{file_id=197, log_id=49257725, offset=39022588}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bb0050, len:4382, log_data_len:282, seq:49257724, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:34.115481] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257725, offset=39022588}, end_cursor:ObLogCursor{file_id=197, log_id=49257726, offset=39022870}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ce4050, len:4100, log_data_len:282, seq:49257725, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:34.115522] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650138, miss_cnt:32, cache_read_size:8050408071, hit_ratio:"9.999994892272439095e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:34.115550] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=32] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656876, miss_cnt:17, cache_read_size:8050155555, hit_ratio:"9.999997286810886621e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:34.125024] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061854124625000, start_id=1729061854124625000, end_id=1729061854124625001) [2024-10-16 14:57:34.125795] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=11] current monitor number(seq_=-1) [2024-10-16 14:57:34.127347] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:34.129355] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1574, mem_used=4159488) [2024-10-16 14:57:34.137189] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B97F-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061854136200000, start_id=1729061854136200000, end_id=1729061854136200001) [2024-10-16 14:57:34.137467] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B981-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061854137257000, start_id=1729061854137257000, end_id=1729061854137257001) [2024-10-16 14:57:34.137476] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B983-0-0] [lt=8] submit log success(service_type_=0, last_id=1729061870721265000, limited_id=1729061870721265000) [2024-10-16 14:57:34.137483] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B983-0-0] [lt=7] submit log with lock success(service_type_=0, last_id=1729061870721265000, limited_id=1729061870721265000) [2024-10-16 14:57:34.137487] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B983-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061854137257000, start_id=1729061854137257000, end_id=1729061854137257001) [2024-10-16 14:57:34.137609] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B982-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061854137257000, start_id=1729061854137257000, end_id=1729061854137257001) [2024-10-16 14:57:34.137844] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:109) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0C0E-0-0] [lt=4] is logging [2024-10-16 14:57:34.137861] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:57) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0C0E-0-0] [lt=16] submit log with lock(ret=-4023, service_type_=0, last_id=1729061870721265000, limited_id=1729061870721265000) [2024-10-16 14:57:34.138100] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061870721265000, latest_scn={val:1729061854137257001, v:0}, *this={tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061851834415001, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870721265000, latest_log_ts:{val:1729061854137257001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:34.138149] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=48] handle submit callback(service_type_=0, success=true, used_ts=673, log_ts={val:1729061854137257001, v:0}, limited_id=1729061870721265000) [2024-10-16 14:57:34.138158] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=9] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:34.138166] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=7] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:34.140518] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=353, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:34.140532] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1323) [2024-10-16 14:57:34.140543] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=66, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:34.140573] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:34.140752] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD131-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:34.140762] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD131-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1039, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:34.140763] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=59] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1445, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:34.140766] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD131-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=770, avg_log_submit_to_slide_time=1033) [2024-10-16 14:57:34.142423] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.150786] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061854150632000, start_id=1729061854150632000, end_id=1729061854150632001) [2024-10-16 14:57:34.150852] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=26, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:34.150867] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:34.155844] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76DD-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:34.174467] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:34.174703] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=35] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:34.174766] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD005-0-0] [lt=17] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061854165992, keepalive_ts:-1}) [2024-10-16 14:57:34.174785] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD005-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061854165992, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:34.177808] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D65-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510322046201, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410585, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:34.179780] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5024-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:34.179796] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5024-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:34.182484] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061854182263000, start_id=1729061854182263000, end_id=1729061854182263001) [2024-10-16 14:57:34.184785] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5312-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:34.184797] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5312-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=518, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:34.184819] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=25] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:34.184826] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:34.187853] INFO [SHARE.LOCATION] renew_all_ls_locations_by_rpc (ob_ls_location_service.cpp:801) [21248][AutoLSLocRpc][T0][YB4A0A000407-00061E9B390A2250-0-0] [lt=73] [LS_LOCATION] Get ls leaders by RPC(ret=0, ret="OB_SUCCESS", dests=["10.0.4.7:2890", "10.0.4.8:2890", "10.0.4.9:2890"], leaders=[{key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}}, {key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}}, {key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}}, {key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}}, {key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}}, {key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}}, {key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}}, {key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}}, {key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}]) [2024-10-16 14:57:34.199949] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061854199106000, start_id=1729061854199106000, end_id=1729061854199106001) [2024-10-16 14:57:34.201614] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.210624] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5877-0-0] [lt=19] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:34.210723] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5877-0-0] [lt=14][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:34.232902] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] ====== checkpoint timer task ====== [2024-10-16 14:57:34.233629] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.233656] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.233679] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:34.233809] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061851834415001, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870721265000, latest_log_ts:{val:1729061854137257001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:34.233863] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=50] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:34.233883] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:34.233893] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061854137257001, v:0}) [2024-10-16 14:57:34.233900] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.233908] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:34.233923] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:34.233932] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:34.234661] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.234675] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.234686] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:34.234758] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061851834415001, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:34.234789] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061851834415001, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:34.234819] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:34.234825] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:34.234832] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:34.234837] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:34.235540] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.235552] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.235562] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:34.235634] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061851834415001, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:34.235663] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061851834415001, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:34.235686] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:34.235691] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:34.235698] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:34.235703] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:34.236393] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.236405] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:34.236414] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:34.236488] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061851834415001, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:34.236515] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061851834415001, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:34.236538] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:34.236544] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:34.236550] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:34.236555] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:34.236559] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:34.242483] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.242705] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E0A-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061854242197000, start_id=1729061854242197000, end_id=1729061854242197001) [2024-10-16 14:57:34.246499] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD08-0-0] [lt=15][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:34.250880] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21659][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE880-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510322119273, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410586, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:34.264606] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:34.269503] INFO [LIB] stat (utility.h:1165) [107146][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BBD-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:34.271653] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1DFF-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061854270754000, start_id=1729061854270754000, end_id=1729061854270754001) [2024-10-16 14:57:34.271930] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1764, avg_group_log_size=160) [2024-10-16 14:57:34.271943] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1764) [2024-10-16 14:57:34.272024] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:34.272036] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:34.272195] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:34.277484] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB96-0-0] [lt=8] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12287, add_timestamp:1729061854277471}) [2024-10-16 14:57:34.277501] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E30-0-0] [lt=8] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12287, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:34.277640] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E30-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12287, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061854277628, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061854277628, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061854277628, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:34.277660] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E30-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:12287, add_timestamp:1729061854277471}, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061854277628, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061854277628, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061854277628, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:34.282723] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB96-0-0] [lt=18][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:34.289011] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=14] ls blacklist refresh finish(cost_time=12285) [2024-10-16 14:57:34.294007] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:34.294615] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:34.294628] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:34.298667] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061854298493000, start_id=1729061854298493000, end_id=1729061854298493001) [2024-10-16 14:57:34.298736] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9331, avg_group_log_size=666) [2024-10-16 14:57:34.298746] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9331) [2024-10-16 14:57:34.298797] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183801393, max_lsn:114264652154, max_scn:{val:1729061854298493000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45336899169}, reuse_lsn:{lsn:114264651561}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114222709114}}, last_submit_lsn:{lsn:114264651561}, last_submit_end_lsn:{lsn:114264652154}, last_submit_log_id:183801393, last_submit_log_pid:33576, max_flushed_lsn:{lsn:114264650968}, max_flushed_end_lsn:{lsn:114264651561}, max_flushed_log_pid:33576, committed_end_lsn:{lsn:114264651561}, last_slide_log_id:183801392, last_slide_scn:{val:1729061854199106000, v:0}, last_slide_lsn:{lsn:114264650968}, last_slide_end_lsn:{lsn:114264651561}, last_slide_log_pid:33576, last_slide_log_accum_checksum:2546141157, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e2efe2530}, [StateMgr]={this:0x2b0e2efe6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, leader:"10.0.4.7:2890", leader_epoch:2, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183801393, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261840845, palf_role_change_cb:0x2b0e2efed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, reconfig_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, persistent_config_version:{proposal_id:33576, config_seq:9}, ms_ack_list:0, resend_config_version:{proposal_id:33576, config_seq:9}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494259837171, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494259837171, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e2efe3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262041271, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729031774558563002, v:0}, min_block_id:1697, min_block_min_scn:{val:1729025370959302001, v:0}, base_lsn_for_block_gc:{lsn:114212315136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, log_config_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, log_snapshot_meta:{version:1, base_lsn:{lsn:114212315136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:232079360}, readable_log_tail:{lsn:232079360}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114279419904}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/meta", dir_fd:187, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:36339712, block_header_serialize_buf_:0x2b0e2efe89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114264651561}, readable_log_tail:{lsn:114264651561}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114212315136}, min_scn:{val:1729057087482360001, v:0}, curr_block_id:1702, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/log", dir_fd:190, min_block_id:1697, max_block_id:1703, curr_writable_block_id:1702}, logical_block_size_:67104768, curr_block_writable_size_:14768343, block_header_serialize_buf_:0x2b0e2efea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261839896, this:0x2b0e2efe6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33576, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33575, majority_max_lsn:{lsn:45338084276}, saved_end_lsn:{lsn:45338084276}, last_submit_prepare_req_time_us:1722494261840968, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e2efe69b0}) [2024-10-16 14:57:34.298990] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=189] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061854200499, last_flushed_end_lsn:{lsn:114264651561}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061854201073, last_flushed_end_lsn:{lsn:114264651561}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061854200617, last_flushed_end_lsn:{lsn:114264651561}}]) [2024-10-16 14:57:34.299001] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183801393, log_task={header:{begin_lsn:{lsn:114264651561}, end_lsn:{lsn:114264652154}, log_id:183801393, min_scn:{val:1729061854298493000, v:0}, max_scn:{val:1729061854298493000, v:0}, data_len:537, proposal_id:33576, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:114264651561}, data_checksum:2929312632, accum_checksum:2888485807, is_padding_log:false, is_raw_write:false}, state_map:{val:30}, ref_cnt:0, gen_ts:1729061854298698, freeze_ts:1729061854298700, submit_ts:1729061854298733, flushed_ts:-1, gen_to_freeze cost time:2, gen_to_submit cost time:35, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061854200499, last_flushed_end_lsn:{lsn:114264651561}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061854201073, last_flushed_end_lsn:{lsn:114264651561}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061854200617, last_flushed_end_lsn:{lsn:114264651561}}]) [2024-10-16 14:57:34.299208] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=328, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:34.299225] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9331) [2024-10-16 14:57:34.299238] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=56, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:34.299259] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9331) [2024-10-16 14:57:34.299269] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:34.299975] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD0B6-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:34.299985] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD0B6-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=655, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:34.299989] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD0B6-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=423, avg_log_submit_to_slide_time=623) [2024-10-16 14:57:34.301372] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=8] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:34.301673] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.302079] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985CF-0-0] [lt=14] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=187) [2024-10-16 14:57:34.302189] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061854300617000, start_id=1729061854300617000, end_id=1729061854300617001) [2024-10-16 14:57:34.311440] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:57:34.311459] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:34.311479] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373573666, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:34.314186] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6475, mem_used=12478464) [2024-10-16 14:57:34.317513] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B21-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061854316836000, start_id=1729061854316836000, end_id=1729061854316836001) [2024-10-16 14:57:34.321723] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=22] BlockGCTimerTask success(ret=0, cost_time_us=8, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):113, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:34.322313] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=14] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:34.325291] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061854324184000, start_id=1729061854324184000, end_id=1729061854324184001) [2024-10-16 14:57:34.342531] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.351781] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061854351298000, start_id=1729061854351298000, end_id=1729061854351298001) [2024-10-16 14:57:34.351883] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=28] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3969) [2024-10-16 14:57:34.352159] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4434) [2024-10-16 14:57:34.352195] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=27] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3969) [2024-10-16 14:57:34.352589] INFO [LIB] stat (utility.h:1165) [107240][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FD6-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:34.352610] INFO [LIB] stat (utility.h:1165) [107240][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FD6-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1884, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:34.355931] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76E1-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938194, log_task={header:{begin_lsn:{lsn:213058228081}, end_lsn:{lsn:213058228203}, log_id:220938194, min_scn:{val:1729061854137257002, v:0}, max_scn:{val:1729061854137257002, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058227959}, prev_proposal_id:88, committed_end_lsn:{lsn:213058228081}, data_checksum:1715074492, accum_checksum:1345133069, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061854255835, freeze_ts:1729061854255835, submit_ts:1729061854255840, flushed_ts:1729061854258491, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:2651}, fs_cb_begin_ts=1729061854355924, log_life_time=100089) [2024-10-16 14:57:34.356198] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=220, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:34.356218] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=538, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:34.356252] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:34.362061] INFO [SHARE] adaptive_update_limit (ob_share_throttle_define.cpp:112) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789DA-0-0] [lt=13] adaptive update(Tenant ID=1004, Config Specify Resource Limit(MB)=511, TxShare Current Memory Limit(MB)=154, Holding Memory(MB)=46, Tenant Remain Memory(MB)=179, Usable Remain Memory(MB)=107, Last Update Limit Timestamp=1729061854360823, Is Updated=true) [2024-10-16 14:57:34.362129] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=10] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:34.362159] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789DA-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061854360823000, start_id=1729061854360823000, end_id=1729061854360823001) [2024-10-16 14:57:34.362480] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=46221677, this={dio_aligned_buf:{buf_write_offset:2960, buf_padding_size:1136, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33201899, truncate_used_ts:11502108}, log_block_size:67108864, dir_fd:195, io_fd:131}, aligned_buf_len=4096, aligned_block_offset=46219264, buf_len=547, total_write_size_=15421, total_write_size_after_dio_=425984, ob_pwrite_used_ts=68248, count=100, each_pwrite_cost=682) [2024-10-16 14:57:34.362533] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107146][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=15] worker thread exit(tenant_->id()=1004, workers_.get_size()=3) [2024-10-16 14:57:34.362546] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107146][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=8] tenant thread end_run(id_=1004, ret=0, thread_count_=192) [2024-10-16 14:57:34.371861] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:34.371880] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=14] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:34.371898] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:34.372516] INFO [LIB] stat (utility.h:1165) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD139-0-0] [lt=26] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:34.372531] INFO [LIB] stat (utility.h:1165) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD139-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=653, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:34.372537] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD139-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=22, avg_log_submit_to_flush_time=318, avg_log_submit_to_slide_time=631) [2024-10-16 14:57:34.374811] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:34.375176] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:34.375192] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:34.375210] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=14] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:34.382708] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061854382498000, start_id=1729061854382498000, end_id=1729061854382498001) [2024-10-16 14:57:34.387584] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F11-0-0] [lt=27][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497342, log_task={header:{begin_lsn:{lsn:207589538447}, end_lsn:{lsn:207589538569}, log_id:218497342, min_scn:{val:1729061854137257001, v:0}, max_scn:{val:1729061854137257001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589538325}, prev_proposal_id:81, committed_end_lsn:{lsn:207589538447}, data_checksum:940009244, accum_checksum:1425516440, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061854285936, freeze_ts:1729061854285936, submit_ts:1729061854285941, flushed_ts:1729061854286179, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:238}, fs_cb_begin_ts=1729061854387578, log_life_time=101642) [2024-10-16 14:57:34.394055] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=22] gc stale ls task succ [2024-10-16 14:57:34.400711] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] Garbage Collector is running(seq_=656748, gc_interval=10000000) [2024-10-16 14:57:34.400742] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=13] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:34.400754] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=29) [2024-10-16 14:57:34.400760] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:34.400765] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:34.400772] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:34.400777] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=6) [2024-10-16 14:57:34.400781] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=2] ObSafeDestroyHandler start process [2024-10-16 14:57:34.400907] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061854399226000, start_id=1729061854399226000, end_id=1729061854399226001) [2024-10-16 14:57:34.401579] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061854401201000, start_id=1729061854401201000, end_id=1729061854401201001) [2024-10-16 14:57:34.401784] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.437857] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488B-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061854405579, keepalive_ts:-1}) [2024-10-16 14:57:34.437890] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488B-0-0] [lt=31] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061854405579, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:34.452684] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.453001] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] no need to update min/max version(new_version={val:1729061854150632000, v:0}, min_version={val:1729061854150632000, v:0}, max_version={val:1729061854150632000, v:0}) [2024-10-16 14:57:34.455977] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76E3-0-0] [lt=34] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:34.462148] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A78-0-0] [lt=9] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:34.462169] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A78-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:34.472218] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:34.475122] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [21604][T1_SSTableGC][T1][YB4A0A000407-00061E99776CB729-0-0] [lt=11] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60476}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015268507128011}, old_min_reserved_snapshot=1729060044396711000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060044396711000) [2024-10-16 14:57:34.483538] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=309, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:34.483552] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=928, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:34.483568] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:34.486327] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD9A-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:34.486343] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FD9A-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1819, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:34.486891] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD006-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061854466149, keepalive_ts:-1}) [2024-10-16 14:57:34.486905] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD006-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061854466149, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:34.494104] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:34.494888] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C84-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061854494132000, start_id=1729061854494132000, end_id=1729061854494132001) [2024-10-16 14:57:34.495015] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:34.495028] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:34.495615] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C420F-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1956) [2024-10-16 14:57:34.495764] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F402D-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:34.495771] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F402D-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=748, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:34.495797] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [21604][T1_SSTableGC][T1][YB4A0A000407-00061E99776CB729-0-0] [lt=16] SSTableGCTask(cost_ts=20859) [2024-10-16 14:57:34.497462] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:34.497648] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=476, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:34.497664] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=300, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:34.497701] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=766, avg_log_submit_to_slide_time=977) [2024-10-16 14:57:34.497718] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:34.500879] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B5A0C6795-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061854500469000, start_id=1729061854500469000, end_id=1729061854500469001) [2024-10-16 14:57:34.501846] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.507410] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26170][T1_L0_G5][T1][YB4A0A000408-00061E9B5A0C6795-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061854506763000, start_id=1729061854506763000, end_id=1729061854506763001) [2024-10-16 14:57:34.509165] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B5A0C6795-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061854508223000, start_id=1729061854508223000, end_id=1729061854508223001) [2024-10-16 14:57:34.518250] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B5A0C6795-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061854517829000, start_id=1729061854517829000, end_id=1729061854517829001) [2024-10-16 14:57:34.518620] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BC-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:34.518646] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BC-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061854137257002, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061854137257002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:34.519684] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9BC-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061854518868000, start_id=1729061854518868000, end_id=1729061854518868001) [2024-10-16 14:57:34.520583] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9A279FA9BC-0-0] [lt=22] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.054953432083129883e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=302, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.073582535704922066e+03) [2024-10-16 14:57:34.522040] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9BC-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061854520996000, start_id=1729061854520996000, end_id=1729061854520996001) [2024-10-16 14:57:34.522677] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:34.525581] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061854522060000, start_id=1729061854522060000, end_id=1729061854522060001) [2024-10-16 14:57:34.527299] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [21736][T1001_COMMONLSS][T1001][YB4A0A000407-00061E9B3A29CAF3-0-0] [lt=16][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:34.532514] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:34.532529] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:34.532544] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444487218, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:34.540575] INFO [LIB] stat (utility.h:1165) [107148][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D42-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:34.540597] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107148][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D42-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:34.540612] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107148][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D42-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:34.540626] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107148][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D42-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173078, log_task={header:{begin_lsn:{lsn:14352365700}, end_lsn:{lsn:14352365822}, log_id:115173078, min_scn:{val:1729061854137257003, v:0}, max_scn:{val:1729061854137257003, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352365578}, prev_proposal_id:17, committed_end_lsn:{lsn:14352365700}, data_checksum:940009244, accum_checksum:880761669, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061854440585, freeze_ts:1729061854440586, submit_ts:1729061854440592, flushed_ts:1729061854442568, gen_to_freeze cost time:1, gen_to_submit cost time:7, submit_to_flush cost time:1976}, fs_cb_begin_ts=1729061854540619, log_life_time=100034) [2024-10-16 14:57:34.541244] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061854540978000, start_id=1729061854540978000, end_id=1729061854540978001) [2024-10-16 14:57:34.541262] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:34.541273] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=516, get_gts_with_stc_cnt=73, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=73, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:34.541423] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=518, get_gts_with_stc_cnt=8, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:34.541429] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061854541418], gts:1729061854540978000, latest_srr:[mts=1729061854541418]}) [2024-10-16 14:57:34.542403] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B5A0C6795-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061854542035000, start_id=1729061854542035000, end_id=1729061854542035001) [2024-10-16 14:57:34.552746] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.553836] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061854553681000, start_id=1729061854553681000, end_id=1729061854553681001) [2024-10-16 14:57:34.554496] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D36-0-0] [lt=12] [APPLY STAT CB APPEND COST TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=49, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:34.554508] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D36-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=1413, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:34.554512] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D36-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:34.554516] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D36-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=-815, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:34.554519] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D36-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=647, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:34.556198] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:34.569043] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.569395] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.569396] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.569421] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.569792] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BC3-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:34.569963] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C49-0-0] [lt=25] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:34.569991] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C49-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061854530265000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061854530265000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:34.575225] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:34.575279] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C49-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061854574722000, start_id=1729061854574722000, end_id=1729061854574722001) [2024-10-16 14:57:34.583676] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:34.584146] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061854583170000, start_id=1729061854583170000, end_id=1729061854583170001) [2024-10-16 14:57:34.584295] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1001}, {id:1}]) [2024-10-16 14:57:34.584879] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:34.586856] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=2][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:34.586871] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:34.587567] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D3B-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061854578810, keepalive_ts:-1}) [2024-10-16 14:57:34.587578] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D3B-0-0] [lt=10] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061854578810, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:34.587616] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D3C-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061854578875, keepalive_ts:-1}) [2024-10-16 14:57:34.587635] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D3C-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061854578875, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:34.592086] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B2B-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061854591661000, start_id=1729061854591661000, end_id=1729061854591661001) [2024-10-16 14:57:34.592358] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23483686, this={dio_aligned_buf:{buf_write_offset:1864, buf_padding_size:2232, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:47003979, truncate_used_ts:19010259}, log_block_size:67108864, dir_fd:134, io_fd:183}, aligned_buf_len=4096, aligned_block_offset=23482368, buf_len=546, total_write_size_=39721, total_write_size_after_dio_=475136, ob_pwrite_used_ts=66500, count=106, each_pwrite_cost=627) [2024-10-16 14:57:34.594746] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1DFE-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061854593729000, start_id=1729061854593729000, end_id=1729061854593729001) [2024-10-16 14:57:34.594980] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=52343055, this={dio_aligned_buf:{buf_write_offset:820, buf_padding_size:3276, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80310884, truncate_used_ts:29009458}, log_block_size:67108864, dir_fd:190, io_fd:179}, aligned_buf_len=4096, aligned_block_offset=52342784, buf_len=549, total_write_size_=102365, total_write_size_after_dio_=790528, ob_pwrite_used_ts=73838, count=168, each_pwrite_cost=439) [2024-10-16 14:57:34.600569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061854598998000, start_id=1729061854598998000, end_id=1729061854598998001) [2024-10-16 14:57:34.601627] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061854598998000, start_id=1729061854598998001, end_id=1729061854598998002) [2024-10-16 14:57:34.601909] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.608354] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=23] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=4839, mem_used=10398720) [2024-10-16 14:57:34.625960] INFO [ARCHIVE] do_thread_task_ (ob_archive_fetcher.cpp:310) [21598][T1_ArcFetcher][T1][YB4A0A000407-00061E996C4960C1-0-0] [lt=13] ObArchiveFetcher is running(thread_index=0) [2024-10-16 14:57:34.637557] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B985-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061854637437000, start_id=1729061854637437000, end_id=1729061854637437001) [2024-10-16 14:57:34.637678] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B986-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061854637437000, start_id=1729061854637437000, end_id=1729061854637437001) [2024-10-16 14:57:34.638077] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0C12-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061854637437000, start_id=1729061854637437001, end_id=1729061854637437002) [2024-10-16 14:57:34.638154] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B984-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061854637437000, start_id=1729061854637437000, end_id=1729061854637437001) [2024-10-16 14:57:34.652816] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.662952] INFO [STORAGE.COMPACTION] schedule_all_tablets_minor (ob_tenant_tablet_scheduler.cpp:553) [22467][T1003_MergeLoop][T1003][YB4A0A000407-00061E99804A3684-0-0] [lt=12] start schedule all tablet minor merge(minor_ls_tablet_iter_={ls_idx:-1, ls_ids:[{id:1}], tablet_idx:0, tablet_ids_.count():0, schedule_tablet_cnt:0, max_batch_tablet_cnt:50000}) [2024-10-16 14:57:34.666627] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:34.676135] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D67-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510322544528, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410587, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:34.676344] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:85) [22467][T1003_MergeLoop][T1003][YB4A0A000407-00061E99804A3684-0-0] [lt=20] MergeLoopTask(cost_ts=13414) [2024-10-16 14:57:34.677131] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D68-0-0] [lt=14][errcode=-4201] hashtable not init [2024-10-16 14:57:34.677165] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D68-0-0] [lt=33][errcode=-4016] get cursor info failed(cursor_id=1112, get_sessid()=3221565766) [2024-10-16 14:57:34.677238] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D69-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061854676580000, start_id=1729061854676580000, end_id=1729061854676580001) [2024-10-16 14:57:34.684705] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F17-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:34.694171] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:34.695053] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22398][T1003_TntShared][T1003][Y0-0000000000000000-0-0] [lt=13] clear dtl interm result cost(us)(clear_cost=15, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:34.695368] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1950) [2024-10-16 14:57:34.695673] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4215-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:34.695688] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4215-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=965, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:34.698078] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=8] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:34.698222] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:34.698897] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061854698728000, start_id=1729061854698728000, end_id=1729061854698728001) [2024-10-16 14:57:34.701970] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.707332] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31896][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB1FCC6DA-0-0] [lt=25] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:34.712205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A11778B1E-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061854711458000, start_id=1729061854711458000, end_id=1729061854711458001) [2024-10-16 14:57:34.712520] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=42717427, this={dio_aligned_buf:{buf_write_offset:791, buf_padding_size:3305, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35133193, truncate_used_ts:12254318}, log_block_size:67108864, dir_fd:169, io_fd:144}, aligned_buf_len=4096, aligned_block_offset=42717184, buf_len=548, total_write_size_=15061, total_write_size_after_dio_=413696, ob_pwrite_used_ts=36262, count=97, each_pwrite_cost=373) [2024-10-16 14:57:34.715758] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:34.715782] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=14] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=211) [2024-10-16 14:57:34.723762] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:34.725776] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061854725308000, start_id=1729061854725308000, end_id=1729061854725308001) [2024-10-16 14:57:34.725847] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=9] LogLoopThread round_cost_time(us)(round_cost_time=19) [2024-10-16 14:57:34.726823] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=80] success to update min reserved snapshot(reserved_snapshot=1729060051830492000, duration=1800, snapshot_gc_ts=1729061851830492000) [2024-10-16 14:57:34.728303] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23146785, this={dio_aligned_buf:{buf_write_offset:401, buf_padding_size:3695, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33384912, truncate_used_ts:14052847}, log_block_size:67108864, dir_fd:200, io_fd:174}, aligned_buf_len=4096, aligned_block_offset=23146496, buf_len=112, total_write_size_=12050, total_write_size_after_dio_=417792, ob_pwrite_used_ts=49720, count=99, each_pwrite_cost=502) [2024-10-16 14:57:34.728610] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=15] update reserved snapshot finished(cost_ts=1789, reserved_snapshot=1729060051830492000) [2024-10-16 14:57:34.752864] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.753170] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061854553681000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199485, in_cluster_service=true, cluster_version={val:1729061854351298000, v:0}, min_cluster_version={val:1729061854351298000, v:0}, max_cluster_version={val:1729061854351298000, v:0}, get_cluster_version_err=0, cluster_version_delta=401868, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65248830, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=352025881, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:34.753964] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061854753270000, start_id=1729061854753270000, end_id=1729061854753270001) [2024-10-16 14:57:34.754376] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:34.754584] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107240][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FDD-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=13, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=1371, avg_log_submit_to_slide_time=1589) [2024-10-16 14:57:34.755200] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D66-0-0] [lt=23] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:34.755223] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D66-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061854137257004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061854137257004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:34.755667] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:34.755688] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=12] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:34.755697] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=8] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:34.756232] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D66-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061854755426000, start_id=1729061854755426000, end_id=1729061854755426001) [2024-10-16 14:57:34.759028] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D66-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061854758631000, start_id=1729061854758631000, end_id=1729061854758631001) [2024-10-16 14:57:34.759089] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D66-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:34.759101] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D66-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:34.769767] INFO [LIB] stat (utility.h:1165) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BC5-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:34.769780] INFO [LIB] stat (utility.h:1165) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BC5-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=46465, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:34.769784] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BC5-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=283, avg_log_submit_to_slide_time=46457) [2024-10-16 14:57:34.769784] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:34.769792] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BC5-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178386, log_task={header:{begin_lsn:{lsn:14538479596}, end_lsn:{lsn:14538479718}, log_id:115178386, min_scn:{val:1729061854548383000, v:0}, max_scn:{val:1729061854548383000, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538479474}, prev_proposal_id:23, committed_end_lsn:{lsn:14538479596}, data_checksum:1715074492, accum_checksum:3947629297, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061854569781, freeze_ts:1729061854569781, submit_ts:1729061854569785, flushed_ts:1729061854570158, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:373}, fs_cb_begin_ts=1729061854769760, log_life_time=199979) [2024-10-16 14:57:34.769870] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=51, stat_interval=1000000, avg (count/cost)=41, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:34.769876] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=51, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:34.770002] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:34.770834] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:34.772157] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:34.772170] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:34.773377] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=18] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:34.773800] INFO [PALF] read (log_cache.cpp:103) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=144376, hit_cnt=1, read_cnt=1, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:34.773819] INFO [PALF] next (palf_iterator.h:170) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=16] [PALF STAT ITERATOR INFO](io_ctx_={user:"ARCHIVE", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:1, miss_cnt:0, cache_read_size:144376, hit_ratio:"1.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:13645266451}}}) [2024-10-16 14:57:34.774138] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15D8-0-0] [lt=13] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:34.774258] INFO [ARCHIVE] statistic (ob_archive_fetcher.cpp:1067) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=12] archive_fetcher statistic in 10s(total_read_log_size=328960, total_read_cost_ts=1022, total_read_task_count=2, avg_task_lsn_size=164480, avg_task_cost_ts=511) [2024-10-16 14:57:34.774265] INFO [ARCHIVE] build_send_task_ (ob_archive_fetcher.cpp:868) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=6] build send task succ(id={id:1001}, station={key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}) [2024-10-16 14:57:34.774273] INFO [ARCHIVE] back_fill (ob_archive_task.cpp:137) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=8] print back fill task succ(cur_piece={interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset={lsn:13645266451}, end_offset={lsn:13645410827}, max_scn={val:1729061854598998000, v:0}, this={tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729042537979987002, v:0}, start_offset:{lsn:13622267904}, end_offset:{lsn:13689372672}, cur_offset:{lsn:13645410827}, unfinished_data_size:43961845, max_scn:{val:1729061854598998000, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:13645266451}, end_offset:{lsn:13645410827}, max_scn:{val:1729061854598998000, v:0}, file_id:0, file_offset:-1, data:G, data_len:144376, this:0x2b0f382ee030}, this:0x2b0e86660a00}) [2024-10-16 14:57:34.774297] INFO [ARCHIVE] update_log_fetch_task_ (ob_archive_fetcher.cpp:812) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=23] back fill log fetch task succ(fetch_task={tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729042537979987002, v:0}, start_offset:{lsn:13622267904}, end_offset:{lsn:13689372672}, cur_offset:{lsn:13645410827}, unfinished_data_size:43961845, max_scn:{val:1729061854598998000, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:13645266451}, end_offset:{lsn:13645410827}, max_scn:{val:1729061854598998000, v:0}, file_id:0, file_offset:-1, data:G, data_len:144376, this:0x2b0f382ee030}, this:0x2b0e86660a00}) [2024-10-16 14:57:34.774312] INFO [ARCHIVE] push_fetch_log (ob_ls_task.cpp:749) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=14] print push_fetch_log(task={tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729042537979987002, v:0}, start_offset:{lsn:13622267904}, end_offset:{lsn:13689372672}, cur_offset:{lsn:13645410827}, unfinished_data_size:43961845, max_scn:{val:1729061854598998000, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:13645266451}, end_offset:{lsn:13645410827}, max_scn:{val:1729061854598998000, v:0}, file_id:0, file_offset:-1, data:G, data_len:144376, this:0x2b0f382ee030}, this:0x2b0e86660a00}, this={has_encount_error:false, is_worm:false, max_no_limit_lsn:{lsn:13531789325}, max_archived_info:{offset:{lsn:13645266451}, scn:{val:1729061734410085002, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, max_seq_log_offset:{lsn:13689372672}, max_fetch_info:{offset:{lsn:13645266451}, scn:{val:1729061734410085002, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, piece_min_lsn:{lsn:13635346983}, archive_file_id:204, archive_file_offset:9919468, piece_dir_exist:true, wait_send_task_count:1, tasks:{tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729042537979987002, v:0}, start_offset:{lsn:13622267904}, end_offset:{lsn:13689372672}, cur_offset:{lsn:13645410827}, unfinished_data_size:43961845, max_scn:{val:1729061854598998000, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:13645266451}, end_offset:{lsn:13645410827}, max_scn:{val:1729061854598998000, v:0}, file_id:0, file_offset:-1, data:G, data_len:144376, this:0x2b0f382ee030}, this:0x2b0e86660a00}, {tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729042537979987002, v:0}, start_offset:{lsn:13622267904}, end_offset:{lsn:13689372672}, cur_offset:{lsn:13645410827}, unfinished_data_size:43961845, max_scn:{val:1729061854598998000, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:13645266451}, end_offset:{lsn:13645410827}, max_scn:{val:1729061854598998000, v:0}, file_id:0, file_offset:-1, data:G, data_len:144376, this:0x2b0f382ee030}, this:0x2b0e86660a00}}) [2024-10-16 14:57:34.774356] INFO [ARCHIVE] submit_fetch_log_ (ob_archive_fetcher.cpp:844) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=44] push fetch log succ(id={id:1001}, &task=0x2b0e86660a00) [2024-10-16 14:57:34.774360] INFO [ARCHIVE] handle_log_fetch_task_ (ob_archive_fetcher.cpp:403) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=4] handle log fetch task succ(id={id:1001}) [2024-10-16 14:57:34.774374] INFO [ARCHIVE] get_sorted_fetch_log (ob_ls_task.cpp:192) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=12] the same file(tuple={offset:{lsn:13645266451}, scn:{val:1729061734410085002, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, tmp_task={tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729042537979987002, v:0}, start_offset:{lsn:13622267904}, end_offset:{lsn:13689372672}, cur_offset:{lsn:13645410827}, unfinished_data_size:43961845, max_scn:{val:1729061854598998000, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:13645266451}, end_offset:{lsn:13645410827}, max_scn:{val:1729061854598998000, v:0}, file_id:0, file_offset:-1, data:G, data_len:144376, this:0x2b0f382ee030}, this:0x2b0e86660a00}) [2024-10-16 14:57:34.774390] INFO [ARCHIVE] get_sorted_fetch_log_ (ob_archive_fetcher.cpp:945) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=16] get sorted fetch log succ(task={tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729042537979987002, v:0}, start_offset:{lsn:13622267904}, end_offset:{lsn:13689372672}, cur_offset:{lsn:13645410827}, unfinished_data_size:43961845, max_scn:{val:1729061854598998000, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1001}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:13645266451}, end_offset:{lsn:13645410827}, max_scn:{val:1729061854598998000, v:0}, file_id:0, file_offset:-1, data:G, data_len:144376, this:0x2b0f382ee030}, this:0x2b0e86660a00}) [2024-10-16 14:57:34.774407] INFO [ARCHIVE] push_task_status (ob_archive_sender.cpp:188) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=13] push succ(task_status=0x2b0eb3a7c500) [2024-10-16 14:57:34.774411] INFO [ARCHIVE] submit_send_task_ (ob_archive_fetcher.cpp:1002) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=3] submit send task succ(send_task=0x2b0f382ee030) [2024-10-16 14:57:34.774492] INFO [PALF] read (log_cache.cpp:103) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=8] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=185977, hit_cnt=1, read_cnt=1, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:34.774498] INFO [PALF] next (palf_iterator.h:170) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=5] [PALF STAT ITERATOR INFO](io_ctx_={user:"ARCHIVE", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:1, miss_cnt:0, cache_read_size:185977, hit_ratio:"1.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:17493272856}}}) [2024-10-16 14:57:34.774513] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=13][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.774550] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=15][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.774937] INFO [ARCHIVE] build_send_task_ (ob_archive_fetcher.cpp:868) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=8] build send task succ(id={id:1}, station={key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}) [2024-10-16 14:57:34.774942] INFO [ARCHIVE] back_fill (ob_archive_task.cpp:137) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=5] print back fill task succ(cur_piece={interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset={lsn:17493272856}, end_offset={lsn:17493458833}, max_scn={val:1729061854598998002, v:0}, this={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729032503563174001, v:0}, start_offset:{lsn:17447239680}, end_offset:{lsn:17514344448}, cur_offset:{lsn:17493458833}, unfinished_data_size:20885615, max_scn:{val:1729061854598998002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:17493272856}, end_offset:{lsn:17493458833}, max_scn:{val:1729061854598998002, v:0}, file_id:0, file_offset:-1, data:G, data_len:185977, this:0x2b0f38314030}, this:0x2b0e866607e0}) [2024-10-16 14:57:34.774958] INFO [ARCHIVE] update_log_fetch_task_ (ob_archive_fetcher.cpp:812) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=16] back fill log fetch task succ(fetch_task={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729032503563174001, v:0}, start_offset:{lsn:17447239680}, end_offset:{lsn:17514344448}, cur_offset:{lsn:17493458833}, unfinished_data_size:20885615, max_scn:{val:1729061854598998002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:17493272856}, end_offset:{lsn:17493458833}, max_scn:{val:1729061854598998002, v:0}, file_id:0, file_offset:-1, data:G, data_len:185977, this:0x2b0f38314030}, this:0x2b0e866607e0}) [2024-10-16 14:57:34.774971] INFO [ARCHIVE] push_fetch_log (ob_ls_task.cpp:749) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=13] print push_fetch_log(task={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729032503563174001, v:0}, start_offset:{lsn:17447239680}, end_offset:{lsn:17514344448}, cur_offset:{lsn:17493458833}, unfinished_data_size:20885615, max_scn:{val:1729061854598998002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:17493272856}, end_offset:{lsn:17493458833}, max_scn:{val:1729061854598998002, v:0}, file_id:0, file_offset:-1, data:G, data_len:185977, this:0x2b0f38314030}, this:0x2b0e866607e0}, this={has_encount_error:false, is_worm:false, max_no_limit_lsn:{lsn:17343057821}, max_archived_info:{offset:{lsn:17493272856}, scn:{val:1729061734410085003, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, max_seq_log_offset:{lsn:17514344448}, max_fetch_info:{offset:{lsn:17493272856}, scn:{val:1729061734410085003, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, piece_min_lsn:{lsn:17479894340}, archive_file_id:261, archive_file_offset:13378516, piece_dir_exist:true, wait_send_task_count:1, tasks:{tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729032503563174001, v:0}, start_offset:{lsn:17447239680}, end_offset:{lsn:17514344448}, cur_offset:{lsn:17493458833}, unfinished_data_size:20885615, max_scn:{val:1729061854598998002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:17493272856}, end_offset:{lsn:17493458833}, max_scn:{val:1729061854598998002, v:0}, file_id:0, file_offset:-1, data:G, data_len:185977, this:0x2b0f38314030}, this:0x2b0e866607e0}, {tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729032503563174001, v:0}, start_offset:{lsn:17447239680}, end_offset:{lsn:17514344448}, cur_offset:{lsn:17493458833}, unfinished_data_size:20885615, max_scn:{val:1729061854598998002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:17493272856}, end_offset:{lsn:17493458833}, max_scn:{val:1729061854598998002, v:0}, file_id:0, file_offset:-1, data:G, data_len:185977, this:0x2b0f38314030}, this:0x2b0e866607e0}}) [2024-10-16 14:57:34.775011] INFO [ARCHIVE] submit_fetch_log_ (ob_archive_fetcher.cpp:844) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=40] push fetch log succ(id={id:1}, &task=0x2b0e866607e0) [2024-10-16 14:57:34.775014] INFO [ARCHIVE] handle_log_fetch_task_ (ob_archive_fetcher.cpp:403) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=2] handle log fetch task succ(id={id:1}) [2024-10-16 14:57:34.775018] INFO [ARCHIVE] get_sorted_fetch_log (ob_ls_task.cpp:192) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=2] the same file(tuple={offset:{lsn:17493272856}, scn:{val:1729061734410085003, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, tmp_task={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729032503563174001, v:0}, start_offset:{lsn:17447239680}, end_offset:{lsn:17514344448}, cur_offset:{lsn:17493458833}, unfinished_data_size:20885615, max_scn:{val:1729061854598998002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:17493272856}, end_offset:{lsn:17493458833}, max_scn:{val:1729061854598998002, v:0}, file_id:0, file_offset:-1, data:G, data_len:185977, this:0x2b0f38314030}, this:0x2b0e866607e0}) [2024-10-16 14:57:34.775032] INFO [ARCHIVE] get_sorted_fetch_log_ (ob_archive_fetcher.cpp:945) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=15] get sorted fetch log succ(task={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1729032503563174001, v:0}, start_offset:{lsn:17447239680}, end_offset:{lsn:17514344448}, cur_offset:{lsn:17493458833}, unfinished_data_size:20885615, max_scn:{val:1729061854598998002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}, start_offset:{lsn:17493272856}, end_offset:{lsn:17493458833}, max_scn:{val:1729061854598998002, v:0}, file_id:0, file_offset:-1, data:G, data_len:185977, this:0x2b0f38314030}, this:0x2b0e866607e0}) [2024-10-16 14:57:34.775045] INFO [ARCHIVE] push_task_status (ob_archive_sender.cpp:188) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=12] push succ(task_status=0x2b0ed5972500) [2024-10-16 14:57:34.775050] INFO [ARCHIVE] submit_send_task_ (ob_archive_fetcher.cpp:1002) [22803][T1004_ArcFetche][T1004][YB4A0A000407-00061E99727960C1-0-0] [lt=2] submit send task succ(send_task=0x2b0f38314030) [2024-10-16 14:57:34.775072] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=6][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.775530] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1181712384, global_cache_size=1581383680, tenant_max_wash_size=4291788, tenant_min_wash_size=4291788, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:34.778006] INFO [ARCHIVE] push_log_ (ob_archive_sender.cpp:743) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=11] push log succ(id={id:1001}) [2024-10-16 14:57:34.778019] INFO [ARCHIVE] archive_log_ (ob_archive_sender.cpp:629) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=12] finish task succ(id={id:1001}) [2024-10-16 14:57:34.778023] INFO [ARCHIVE] statistic (ob_archive_sender.cpp:912) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=3] archive_sender statistic in 10s(total_send_log_size=328827, total_send_buf_size=328827, total_send_task_count=2, total_send_cost_ts=7143, avg_task_lsn_size=164413, avg_task_buf_size=164413, avg_task_cost_ts=3571) [2024-10-16 14:57:34.778029] INFO [ARCHIVE] handle (ob_archive_sender.cpp:465) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=6] archive log succ(id={id:1001}) [2024-10-16 14:57:34.781348] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=24] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:34.781497] INFO [ARCHIVE] push_log_ (ob_archive_sender.cpp:743) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=3] push log succ(id={id:1}) [2024-10-16 14:57:34.781511] INFO [ARCHIVE] archive_log_ (ob_archive_sender.cpp:629) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=13] finish task succ(id={id:1}) [2024-10-16 14:57:34.781515] INFO [ARCHIVE] handle (ob_archive_sender.cpp:465) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=3] archive log succ(id={id:1}) [2024-10-16 14:57:34.781520] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=3][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.784265] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061854784036000, start_id=1729061854784036000, end_id=1729061854784036001) [2024-10-16 14:57:34.785097] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=13][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2024-10-16 14:57:34.788538] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.788541] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.788560] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.788574] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.788585] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=3] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.788595] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.788616] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.788732] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.789451] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.789453] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.789464] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:34.791593] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=7][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.795198] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=12][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2024-10-16 14:57:34.800706] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061854799910000, start_id=1729061854799910000, end_id=1729061854799910001) [2024-10-16 14:57:34.801792] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061854800968000, start_id=1729061854800968000, end_id=1729061854800968001) [2024-10-16 14:57:34.801972] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=17][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.802033] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.802785] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:34.805272] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=12][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2024-10-16 14:57:34.812043] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=11][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.815342] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=11][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2024-10-16 14:57:34.821444] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=16] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:34.822116] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=14][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.824701] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1001}, {id:1003}]) [2024-10-16 14:57:34.825411] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=11][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2024-10-16 14:57:34.825811] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=39] success to update min reserved snapshot(reserved_snapshot=1729060047238487000, duration=1800, snapshot_gc_ts=1729061847238487000) [2024-10-16 14:57:34.826506] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=11] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060047238487000) [2024-10-16 14:57:34.826521] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=14] update reserved snapshot finished(cost_ts=712, reserved_snapshot=1729060047238487000) [2024-10-16 14:57:34.832189] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=13][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.835487] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=14][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2024-10-16 14:57:34.841002] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=59054700, this={dio_aligned_buf:{buf_write_offset:2790, buf_padding_size:1306, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:38334773, truncate_used_ts:14110971}, log_block_size:67108864, dir_fd:207, io_fd:202}, aligned_buf_len=4096, aligned_block_offset=59052032, buf_len=122, total_write_size_=12294, total_write_size_after_dio_=425984, ob_pwrite_used_ts=66645, count=101, each_pwrite_cost=659) [2024-10-16 14:57:34.842263] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=15][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.845567] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=15][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2024-10-16 14:57:34.852267] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4044, mem_used=8318976) [2024-10-16 14:57:34.852339] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=15][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.852931] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.855647] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=16][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2024-10-16 14:57:34.862413] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22813][T1004_ArcSender][T1004][YB4A0A000407-00061E9972C960C1-0-0] [lt=12][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1001}}) [2024-10-16 14:57:34.865725] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:334) [22810][T1004_ArcSender][T1004][YB4A0A000407-00061E9972B960C1-0-0] [lt=12][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2024-10-16 14:57:34.874610] INFO [ARCHIVE] update_archive_progress (ob_ls_task.cpp:380) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=17] update archive progress succ(ret=0, id_={id:1001}, station={key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, tuple={offset:{lsn:13645410827}, scn:{val:1729061854598998000, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, file_id=204, file_offset=10063844) [2024-10-16 14:57:34.874637] INFO [ARCHIVE] update_archive_progress (ob_ls_task.cpp:380) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=23] update archive progress succ(ret=0, id_={id:1}, station={key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, tuple={offset:{lsn:17493458833}, scn:{val:1729061854598998002, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, file_id=261, file_offset=13564493) [2024-10-16 14:57:34.881162] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=4] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:34.881782] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:34.881829] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061847238487000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:34.881870] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=39] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061847238487000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1003, freeze_info:{latest_snapshot_gc_scn:{val:1729061847238487000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}}) [2024-10-16 14:57:34.885283] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FDA4-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=1329, avg_log_submit_to_slide_time=2872) [2024-10-16 14:57:34.885682] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F1B-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:34.885885] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=28] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=189, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:34.885902] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:34.885929] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:34.887712] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061854886103000, start_id=1729061854886103000, end_id=1729061854886103001) [2024-10-16 14:57:34.888208] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C421A-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1333) [2024-10-16 14:57:34.894241] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:34.895485] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2498, avg_group_log_size=249) [2024-10-16 14:57:34.895498] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2498) [2024-10-16 14:57:34.896792] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4038-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=8, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=21, avg_log_submit_to_flush_time=306, avg_log_submit_to_slide_time=756) [2024-10-16 14:57:34.900929] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:34.902095] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.916233] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2349-0-0] [lt=37] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:57:34.916916] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2349-0-0] [lt=19] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1004) [2024-10-16 14:57:34.916934] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2349-0-0] [lt=16] no transfer partition job [2024-10-16 14:57:34.916939] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2349-0-0] [lt=3] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:57:34.918198] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2349-0-0] [lt=6] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1004, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:57:34.918219] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2349-0-0] [lt=14] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:57:34.919435] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C2349-0-0] [lt=13] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1002, status:0}], ls_array_=[{tenant_id:1004, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729059563578310, last_statistic_bg_stat_time=1725269669481368, last_statistic_schema_version=1725266205820024, last_statistic_max_transfer_task_id={id:5}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:57:34.919722] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD9-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061854882483, keepalive_ts:-1}) [2024-10-16 14:57:34.919736] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBD9-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061854882483, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:34.921570] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:34.924613] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=6580, avg_group_log_size=548) [2024-10-16 14:57:34.925858] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061854925633000, start_id=1729061854925633000, end_id=1729061854925633001) [2024-10-16 14:57:34.925918] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:34.925930] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=10467, avg_group_log_size=654) [2024-10-16 14:57:34.925947] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10468) [2024-10-16 14:57:34.925953] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:34.926150] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=226, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:34.926161] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10467) [2024-10-16 14:57:34.926171] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=240, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:34.926184] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10468) [2024-10-16 14:57:34.926191] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:34.926373] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F945C-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:34.926379] INFO [LIB] stat (utility.h:1165) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F945C-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=871, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:34.926383] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106892][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F945C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=584, avg_log_submit_to_slide_time=841) [2024-10-16 14:57:34.927051] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=9] ====== tenant freeze timer task ====== [2024-10-16 14:57:34.927076] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=9] TxData Memory Statistic(logstream info): (ls_id={id:1}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=5) [2024-10-16 14:57:34.927085] INFO [STORAGE] check_and_freeze_tx_data_ (ob_tenant_freezer.cpp:566) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=6] TxData Memory Statistic : (Tenant Total Memory(MB)=1024, Tenant Frozen TxData Memory(MB)=0, Tenant Active TxData Memory(MB)=5, Freeze TxData Trigger Memory(MB)=20, Total TxDataTable Hold Memory(MB)=28, Total TxDataTable Memory Limit(MB)=204) [2024-10-16 14:57:34.941393] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=2610, mem_used=6239232) [2024-10-16 14:57:34.953114] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 232.00; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 4096.00, iops: 6.99, rt: 3619.71; ) [2024-10-16 14:57:34.953139] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:34.953177] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 6144.00, iops: 3.99, rt: 176.00; ) [2024-10-16 14:57:34.953182] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:34.953190] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:34.953228] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:34.954855] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061854954585000, start_id=1729061854954585000, end_id=1729061854954585001) [2024-10-16 14:57:34.956121] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=427, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:34.956139] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=54, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:34.956709] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76ED-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:34.965189] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=11] worker thread began to shrink(tenant_->id()=1, group_id_=1, token=0) [2024-10-16 14:57:34.966850] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107240][T1_L0_G1][T1][Y0-0000000000000000-0-0] [lt=11] worker thread exit(tenant_->id()=1, workers_.get_size()=2) [2024-10-16 14:57:34.966861] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107240][T1_L0_G1][T0][Y0-0000000000000000-0-0] [lt=7] tenant thread end_run(id_=1, ret=0, thread_count_=240) [2024-10-16 14:57:34.972497] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2310) [2024-10-16 14:57:34.972497] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=414, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:34.972513] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1047, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:34.975823] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:34.975889] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2013-0-0] [lt=15] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:34.975902] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2013-0-0] [lt=11] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:34.978743] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=41] Cache wash clean map node details(ret=0, clean_node_count=38, clean_time=0, clean_start_pos=817882, clean_num=62914) [2024-10-16 14:57:34.986362] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:34.988133] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061854988132, bytes: 4446811087586, bw: 0.472743 MB/s, add_ts: 1005552, add_bytes: 498459 [2024-10-16 14:57:34.995035] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:34.995087] WDIAG read_multi_row (ob_table_access_helper.h:238) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:34.995794] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=16][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:34.995814] INFO delete_row (ob_table_access_helper.h:312) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1004, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061854995094) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061854995094) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:34.995847] WDIAG read_multi_row (ob_table_access_helper.h:238) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=16][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:34.995869] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=11] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:34.995912] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=5] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:34.995954] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:34.996531] INFO delete_row (ob_table_access_helper.h:312) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=5] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1001, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061854995853) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061854995853) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:34.996631] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:34.996642] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:34.996912] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=203, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:34.996921] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:34.996968] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:34.999642] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:34.999656] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27208900, tenant_id=1004) [2024-10-16 14:57:34.999670] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=11] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:35.001911] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061855000413000, start_id=1729061855000413000, end_id=1729061855000413001) [2024-10-16 14:57:35.002131] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061855000413000, start_id=1729061855000413001, end_id=1729061855000413002) [2024-10-16 14:57:35.002235] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:35.002253] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:35.002259] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.012070] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061855012068, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007638, add_bytes: 0 [2024-10-16 14:57:35.015978] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=7] ====== traversal_flush timer task ====== [2024-10-16 14:57:35.016085] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAA-0-0] [lt=13] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061854925633000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061865590307000, latest_log_ts:{val:1729061848964072001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:35.016170] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAA-0-0] [lt=80] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:35.041613] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061855040596000, start_id=1729061855040596000, end_id=1729061855040596001) [2024-10-16 14:57:35.041639] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=17] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061855041605], gts:1729061855040596000, latest_srr:[mts=1729061855041605]}) [2024-10-16 14:57:35.041660] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=557, get_gts_with_stc_cnt=79, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=76, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:35.041971] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:35.041981] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=7] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:35.041995] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=291901605, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:35.053256] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.069275] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=58] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:35.069917] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:35.069978] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061850192549000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:35.070392] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=302, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:35.070408] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=30, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:35.070440] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:35.070699] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107148][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FDAA-0-0] [lt=18] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:35.072322] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:35.072339] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:35.073122] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107148][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FDAB-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1210) [2024-10-16 14:57:35.073151] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD14E-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2310) [2024-10-16 14:57:35.082712] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:35.082732] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=14] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151993940, tenant_id=1) [2024-10-16 14:57:35.082745] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:35.085567] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:35.085589] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598798, miss_cnt:9, cache_read_size:149777903977, hit_ratio:"9.999999345924561744e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:35.086985] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] ====== checkpoint timer task ====== [2024-10-16 14:57:35.087750] INFO [STORAGE.TRANS] create_tx_ctx_ (ob_trans_ctx_mgr_v4.cpp:418) [22235][T1002_TenantWea][T1001][YB4A0A000407-00061E9B3B0B76A3-0-0] [lt=13] transaction statistics(ls_id={id:1}, total_count=1) [2024-10-16 14:57:35.087773] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:35.087799] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:35.087818] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:35.087887] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061855086682000, start_id=1729061855086682000, end_id=1729061855086682001) [2024-10-16 14:57:35.087918] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061867008855000, latest_log_ts:{val:1729061850421032001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:35.087962] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=42] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:35.087973] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:57:35.087980] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061850421032001, v:0}) [2024-10-16 14:57:35.087983] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:35.087987] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:35.087995] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:35.088010] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:57:35.088018] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:35.088023] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:35.094288] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:35.097182] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=976) [2024-10-16 14:57:35.097202] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=976) [2024-10-16 14:57:35.102296] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=56][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.102402] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601381, offset=9619868}, end_cursor:ObLogCursor{file_id=192, log_id=47601382, offset=9620152}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2da0050, len:1636, log_data_len:284, seq:47601381, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:35.104598] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA49D-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061855103945000, start_id=1729061855103945000, end_id=1729061855103945001) [2024-10-16 14:57:35.105968] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601382, offset=9620152}, end_cursor:ObLogCursor{file_id=192, log_id=47601383, offset=9620426}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bdc050, len:1352, log_data_len:274, seq:47601382, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:35.106574] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA879-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:35.106613] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA879-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061854637437004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061854637437004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:35.107737] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA879-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061855107113000, start_id=1729061855107113001, end_id=1729061855107113002) [2024-10-16 14:57:35.108681] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601383, offset=9620426}, end_cursor:ObLogCursor{file_id=192, log_id=47601384, offset=9620700}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cc8050, len:1078, log_data_len:274, seq:47601383, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:35.109087] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601384, offset=9620700}, end_cursor:ObLogCursor{file_id=192, log_id=47601385, offset=9620974}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d60050, len:804, log_data_len:274, seq:47601384, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:35.109142] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561210, miss_cnt:1, cache_read_size:152590669039, hit_ratio:"9.999999928346853251e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:35.112306] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA879-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061855110278000, start_id=1729061855110278001, end_id=1729061855110278002) [2024-10-16 14:57:35.113471] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:35.113488] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:35.113971] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:35.116584] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0FB1-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061855115767000, start_id=1729061855115767000, end_id=1729061855115767001) [2024-10-16 14:57:35.118009] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=10][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:35.118518] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257726, offset=39022870}, end_cursor:ObLogCursor{file_id=197, log_id=49257727, offset=39023153}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2b68050, len:3818, log_data_len:283, seq:49257726, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:35.121800] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22628][T1004_TeRec][T1004][YB4A0A000407-00061E9B221FB368-0-0] [lt=22] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=0, MAX_GAP=900000000000, new_sync_scn={val:1729061854040663000, v:0}, new_replayable_scn={val:1729061854040663000, v:0}, new_readable_scn={val:1729061854040663000, v:0}, old_tenant_info={tenant_id:1004, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061853036726000, v:0}, replayable_scn:{val:1729061853036726000, v:0}, standby_scn:{val:1729061853036726000, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"ARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:57:35.122008] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB368-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061855121036000, start_id=1729061855121036000, end_id=1729061855121036001) [2024-10-16 14:57:35.125851] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061855125766000, start_id=1729061855125766000, end_id=1729061855125766001) [2024-10-16 14:57:35.126034] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=11] current monitor number(seq_=-1) [2024-10-16 14:57:35.126556] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257727, offset=39023153}, end_cursor:ObLogCursor{file_id=197, log_id=49257728, offset=39023435}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dc8050, len:3535, log_data_len:282, seq:49257727, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:35.127120] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1001, new_min_version={val:1729061854886103000, v:0}, new_max_version={val:1729061854886103000, v:0}, last_min_version={val:1729061854583170000, v:0}, last_max_version={val:1729061854583170000, v:0}, total_time_us=2004, delta_us=241016) [2024-10-16 14:57:35.127136] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1001, current_version={val:1729061854886103000, v:0}, min_version={val:1729061854886103000, v:0}, max_version={val:1729061854886103000, v:0}, in_service=true, leader_epoch=33705) [2024-10-16 14:57:35.127176] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=27] get_max_applied_scn(scn={val:1729061854925633000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618452798}, palf_committed_end_scn_:{val:1729061855125766000, v:0}, last_check_scn_:{val:1729061855125766000, v:0}, max_applied_cb_scn_:{val:1729061854925633000, v:0}}) [2024-10-16 14:57:35.127189] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:35.127408] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:35.129453] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=1669, mem_used=4159488) [2024-10-16 14:57:35.130544] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257728, offset=39023435}, end_cursor:ObLogCursor{file_id=197, log_id=49257729, offset=39023717}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c48050, len:3253, log_data_len:282, seq:49257728, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:35.131643] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257729, offset=39023717}, end_cursor:ObLogCursor{file_id=197, log_id=49257730, offset=39023999}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2db8050, len:2971, log_data_len:282, seq:49257729, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:35.131720] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=35] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656886, miss_cnt:17, cache_read_size:8050156775, hit_ratio:"9.999997286811319608e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:35.132014] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650148, miss_cnt:32, cache_read_size:8050409291, hit_ratio:"9.999994892273253999e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:35.138019] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B989-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061855137079000, start_id=1729061855137079000, end_id=1729061855137079001) [2024-10-16 14:57:35.138203] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B98D-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061855138136000, start_id=1729061855138136000, end_id=1729061855138136001) [2024-10-16 14:57:35.138310] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B98C-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061855138136000, start_id=1729061855138136000, end_id=1729061855138136001) [2024-10-16 14:57:35.138492] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B98B-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061855138136000, start_id=1729061855138136000, end_id=1729061855138136001) [2024-10-16 14:57:35.143505] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=218, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:35.143522] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:35.143534] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=175, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:35.143575] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:35.143785] INFO [LIB] stat (utility.h:1165) [107148][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B909C-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:35.143792] INFO [LIB] stat (utility.h:1165) [107148][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B909C-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10573, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:35.143796] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107148][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B909C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=418, avg_log_submit_to_slide_time=10566) [2024-10-16 14:57:35.143800] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:35.153320] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.155099] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061855154987000, start_id=1729061855154987000, end_id=1729061855154987001) [2024-10-16 14:57:35.155177] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=26, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:35.155191] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:35.158446] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76F1-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:35.158474] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76F1-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:35.158480] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76F1-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18351, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:35.158490] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:35.172980] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=9] [APPLY STAT CB APPEND COST TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=46, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:57:35.172992] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=2159, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:57:35.172996] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:57:35.173000] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=-886, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:57:35.173003] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=24, stat_interval=5000000, avg (count/cost)=1319, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:57:35.173894] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=7] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:57:35.173907] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:57:35.173910] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:35.174513] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061865590307000, latest_log_ts:{val:1729061848964072001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:35.179012] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:35.180897] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5025-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:35.180913] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5025-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:35.181580] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D6A-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510323049974, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410588, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:35.187319] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15D0-0-0] [lt=11] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:35.189236] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31906][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB20CC6DA-0-0] [lt=29] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:35.189342] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15D6-0-0] [lt=10] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:35.191085] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480C-0-0] [lt=5] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061855158251, keepalive_ts:-1}) [2024-10-16 14:57:35.191101] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106892][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480C-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061855158251, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:35.201255] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061855000413001, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=200836, in_cluster_service=true, cluster_version={val:1729061854637437000, v:0}, min_cluster_version={val:1729061854637437000, v:0}, max_cluster_version={val:1729061854637437000, v:0}, get_cluster_version_err=0, cluster_version_delta=563812, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244145, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=464271052, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:35.202119] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061855201190000, start_id=1729061855201190000, end_id=1729061855201190001) [2024-10-16 14:57:35.202266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061855201190000, start_id=1729061855201190001, end_id=1729061855201190002) [2024-10-16 14:57:35.202360] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.253407] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.257170] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=57, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:35.257186] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:35.270052] INFO [LIB] stat (utility.h:1165) [107148][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BCF-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:35.272450] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2182, avg_group_log_size=198) [2024-10-16 14:57:35.272472] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2182) [2024-10-16 14:57:35.272507] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1210, avg_group_log_size=121) [2024-10-16 14:57:35.272513] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1210) [2024-10-16 14:57:35.272723] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:35.276593] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1033-0-0] [lt=15] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:35.285140] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F23-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:35.285160] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F23-0-0] [lt=20] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30773, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:35.285165] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F23-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497351, log_task={header:{begin_lsn:{lsn:207589539536}, end_lsn:{lsn:207589539658}, log_id:218497351, min_scn:{val:1729061855138136000, v:0}, max_scn:{val:1729061855138136000, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589539414}, prev_proposal_id:81, committed_end_lsn:{lsn:207589539536}, data_checksum:940009244, accum_checksum:2920245184, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061855185075, freeze_ts:1729061855185075, submit_ts:1729061855185081, flushed_ts:1729061855189307, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:4226}, fs_cb_begin_ts=1729061855285132, log_life_time=100057) [2024-10-16 14:57:35.288027] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061855287222000, start_id=1729061855287222000, end_id=1729061855287222001) [2024-10-16 14:57:35.290372] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107369][][T1001][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1001, ret=0, thread_count_=172) [2024-10-16 14:57:35.290405] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=15] worker thread created(tenant_->id()=1001, group_id_=1, token=1) [2024-10-16 14:57:35.290444] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107369][][T1001][Y0-0000000000000000-0-0] [lt=70] Init thread local success [2024-10-16 14:57:35.290614] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9465-0-0] [lt=4] init point thread id with(&point=0x55e3435c88c0, idx_=873, point=[thread id=107369, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107369) [2024-10-16 14:57:35.295747] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2044705, replace_num=31457) [2024-10-16 14:57:35.295764] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:35.297391] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=8, avg_log_batch_cnt=1, total_group_log_size=976, avg_group_log_size=122) [2024-10-16 14:57:35.297405] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=976) [2024-10-16 14:57:35.303076] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.314477] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6548, mem_used=12478464) [2024-10-16 14:57:35.322390] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=14] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:35.325242] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=38] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061855125766000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199469, in_cluster_service=true, cluster_version={val:1729061854886103000, v:0}, min_cluster_version={val:1729061854886103000, v:0}, max_cluster_version={val:1729061854886103000, v:0}, get_cluster_version_err=0, cluster_version_delta=439132, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244395, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=229721406, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:35.326182] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061855324153000, start_id=1729061855324153000, end_id=1729061855324153001) [2024-10-16 14:57:35.353441] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.353741] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAA-0-0] [lt=18] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:35.353757] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAA-0-0] [lt=11] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=17) [2024-10-16 14:57:35.355318] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061855354071000, start_id=1729061855354071000, end_id=1729061855354071001) [2024-10-16 14:57:35.355387] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=6155) [2024-10-16 14:57:35.356040] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=6155) [2024-10-16 14:57:35.356384] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=6155) [2024-10-16 14:57:35.356450] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BAB-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:35.356458] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BAB-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=983, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:35.356873] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=207, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:35.356890] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=47, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:35.357722] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:35.363772] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=7] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:35.364113] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107148][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=15] worker thread exit(tenant_->id()=1004, workers_.get_size()=2) [2024-10-16 14:57:35.364133] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107148][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=17] tenant thread end_run(id_=1004, ret=0, thread_count_=191) [2024-10-16 14:57:35.365898] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3857-0-0] [lt=3] MediumCheckTask(cost_ts=7) [2024-10-16 14:57:35.372912] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD157-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:35.372922] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD157-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1764, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:35.372927] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD157-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=1445, avg_log_submit_to_slide_time=1749) [2024-10-16 14:57:35.379302] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:35.380760] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=5] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:35.380782] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=213) [2024-10-16 14:57:35.390118] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:35.394111] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:35.398166] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=11426, avg_group_log_size=571) [2024-10-16 14:57:35.398188] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11897) [2024-10-16 14:57:35.398316] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=233, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:35.398328] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11426) [2024-10-16 14:57:35.398338] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=90, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:35.398352] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11897) [2024-10-16 14:57:35.398359] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:35.398731] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797BDE-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:35.398741] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797BDE-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=879, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:35.398745] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797BDE-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=35, avg_log_gen_to_submit_time=62, avg_log_submit_to_flush_time=431, avg_log_submit_to_slide_time=816) [2024-10-16 14:57:35.402197] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061855401465000, start_id=1729061855401465000, end_id=1729061855401465001) [2024-10-16 14:57:35.402355] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061855401465000, start_id=1729061855401465001, end_id=1729061855401465002) [2024-10-16 14:57:35.403134] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.451956] INFO [SHARE] get_server_tenant_configs (ob_unit_getter.cpp:231) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=8] get_server_tenant_configs(unit_infos=[{unit:unit_id:1, resource_pool_id:1, unit_group_id:1, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, pool:{resource_pool_id:1, name:"sys_pool", unit_count:1, unit_config_id:1, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1, replica_type:0}}, {unit:unit_id:1001, resource_pool_id:1001, unit_group_id:1001, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1001, name:"ocp_pool", unit_count:1, unit_config_id:1001, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1002, replica_type:0}}, {unit:unit_id:1005, resource_pool_id:1003, unit_group_id:1002, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1003, name:"pool_bddb_myoceanbase1_oav", unit_count:1, unit_config_id:1003, zone_list:["myoceanbase1"], tenant_id:1004, replica_type:0}}]) [2024-10-16 14:57:35.452006] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=46] jx_debug: get tenant compatibility mode(tenant_id=1002, compat_mode=0) [2024-10-16 14:57:35.452013] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=5] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:35.452024] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=10] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:35.452036] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=11] jx_debug: get tenant compatibility mode(tenant_id=1004, compat_mode=0) [2024-10-16 14:57:35.452040] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=4] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:35.452047] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=7] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:35.453351] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:191) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=9] refresh resource plan success(tenant_id=1, plan_name=, directives=[]) [2024-10-16 14:57:35.454106] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=14] refresh resource user mapping rule(tenant_id=1, plan=, user_rules=[]) [2024-10-16 14:57:35.454473] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get_max_applied_scn(scn={val:1729061855354071000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390099381}, palf_committed_end_scn_:{val:1729061855354071000, v:0}, last_check_scn_:{val:1729061855354071000, v:0}, max_applied_cb_scn_:{val:1729061855354071000, v:0}}) [2024-10-16 14:57:35.454495] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:35.454814] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=28] refresh_resource_function_mapping_rule(tenant_id=1, plan=, rules=[]) [2024-10-16 14:57:35.454827] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=12] refresh resource mapping rule success(tenant_id=1, plan=) [2024-10-16 14:57:35.455455] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=30, max_blocked_buffer_size_=9830400, max_buffer_size_=19660800) [2024-10-16 14:57:35.455513] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=14] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:35.455523] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=9] tenant dfc status(ret=0, get_tenant_id()=1, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=30, get_max_blocked_buffer_size()=9830400, get_max_buffer_size()=19660800, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:35.455624] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.455801] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:7668488755}, base_lsn={lsn:7582838784}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:35.455837] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=36] get_replay_process success(id={id:1}, replayed_log_size=85649971, unreplayed_log_size=0) [2024-10-16 14:57:35.455843] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:6198988121}, base_lsn={lsn:6106533888}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:35.455875] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=31] get_replay_process success(id={id:1001}, replayed_log_size=92454233, unreplayed_log_size=0) [2024-10-16 14:57:35.455880] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=8163527603, unreplayed_log_size=0) [2024-10-16 14:57:35.455884] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1003}, replayed_log_size=8111623618, unreplayed_log_size=0) [2024-10-16 14:57:35.455899] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1004, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=15691, last_replayed_log_size(MB)=15691, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:35.456622] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE76F7-0-0] [lt=19] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:35.457724] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=10] refresh resource user mapping rule(tenant_id=1001, plan=, user_rules=[]) [2024-10-16 14:57:35.458409] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=15] refresh_resource_function_mapping_rule(tenant_id=1001, plan=, rules=[]) [2024-10-16 14:57:35.458422] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=12] refresh resource mapping rule success(tenant_id=1001, plan=) [2024-10-16 14:57:35.459133] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=3] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:35.459197] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=10] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:35.459204] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=5] tenant dfc status(ret=0, get_tenant_id()=1001, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:35.461194] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=7] refresh resource user mapping rule(tenant_id=1002, plan=, user_rules=[]) [2024-10-16 14:57:35.461871] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=18] refresh_resource_function_mapping_rule(tenant_id=1002, plan=, rules=[]) [2024-10-16 14:57:35.461881] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=9] refresh resource mapping rule success(tenant_id=1002, plan=) [2024-10-16 14:57:35.462571] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:35.462645] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=10] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:35.462652] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] tenant dfc status(ret=0, get_tenant_id()=1002, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=4, get_max_size_per_channel()=655360) [2024-10-16 14:57:35.464464] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A79-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:35.464483] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A79-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:35.464908] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=7] refresh resource user mapping rule(tenant_id=1003, plan=, user_rules=[]) [2024-10-16 14:57:35.465561] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=12] refresh_resource_function_mapping_rule(tenant_id=1003, plan=, rules=[]) [2024-10-16 14:57:35.465572] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=10] refresh resource mapping rule success(tenant_id=1003, plan=) [2024-10-16 14:57:35.466193] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:35.466253] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=10] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:35.466267] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=13] tenant dfc status(ret=0, get_tenant_id()=1003, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:35.468234] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=9] refresh resource user mapping rule(tenant_id=1004, plan=, user_rules=[]) [2024-10-16 14:57:35.468888] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=12] refresh_resource_function_mapping_rule(tenant_id=1004, plan=, rules=[]) [2024-10-16 14:57:35.468899] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=11] refresh resource mapping rule success(tenant_id=1004, plan=) [2024-10-16 14:57:35.469537] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=3] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:35.469596] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=9] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:35.469608] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=12] tenant dfc status(ret=0, get_tenant_id()=1004, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:35.469620] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:108) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] refresh tenant units(sys_unit_cnt=3, units=[{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, {tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, {tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}], ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:35.469650] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=26] finish set nway(tenant_id_=1, nway=3) [2024-10-16 14:57:35.469654] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=4] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1, nway=3, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}) [2024-10-16 14:57:35.469664] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=8] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=1.638399963378906250e+02, this=0x2b0ddc2dc170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=819, Throttle Trigger(MB)=491, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.948457122979588263e+01) [2024-10-16 14:57:35.469675] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=11] succ to set_memstore_threshold of memstore allocator(tenant_id=1, ret=0) [2024-10-16 14:57:35.469681] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=4] finish set nway(tenant_id_=1002, nway=1) [2024-10-16 14:57:35.469683] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1002, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:35.469691] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e0c298170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:35.469697] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] succ to set_memstore_threshold of memstore allocator(tenant_id=1002, ret=0) [2024-10-16 14:57:35.469701] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=3] finish set nway(tenant_id_=1001, nway=1) [2024-10-16 14:57:35.469704] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1001, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:35.469711] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0df7098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:35.469716] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] succ to set_memstore_threshold of memstore allocator(tenant_id=1001, ret=0) [2024-10-16 14:57:35.469720] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=2] finish set nway(tenant_id_=1004, nway=1) [2024-10-16 14:57:35.469723] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1004, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:35.469730] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e40098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:35.469736] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] succ to set_memstore_threshold of memstore allocator(tenant_id=1004, ret=0) [2024-10-16 14:57:35.469740] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=2] finish set nway(tenant_id_=1003, nway=1) [2024-10-16 14:57:35.469742] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1003, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:35.469748] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e2c698170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:35.469754] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=5] succ to set_memstore_threshold of memstore allocator(tenant_id=1003, ret=0) [2024-10-16 14:57:35.470350] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:35.470369] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=18] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1001, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:35.470376] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=7] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:35.470380] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1003, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:35.470387] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=6] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:35.470410] INFO [SERVER] cal_all_part_disk_default_percentage (ob_server_utils.cpp:301) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=3] cal_all_part_disk_default_percentage succ(data_dir="/mnt/myoceanbase/oceanbase/data//sstable", clog_dir="/mnt/myoceanbase/oceanbase/data//clog", shared_mode=true, data_disk_total_size=214429704192, data_disk_default_percentage=60, clog_disk_total_size=214429704192, clog_disk_default_percentage=30) [2024-10-16 14:57:35.470419] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:337) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=10] decide disk size finished(suggested_disk_size=32212254720, suggested_disk_percentage=0, default_disk_percentage=30, total_space=214429704192, disk_size=32212254720) [2024-10-16 14:57:35.470424] INFO [SERVER] get_log_disk_info_in_config (ob_server_utils.cpp:88) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=4] get_log_disk_info_in_config(suggested_data_disk_size=32212254720, suggested_clog_disk_size=32212254720, suggested_data_disk_percentage=0, suggested_clog_disk_percentage=0, log_disk_size=32212254720, log_disk_percentage=0, total_log_disk_size=214429704192) [2024-10-16 14:57:35.470434] INFO [CLOG] try_resize (ob_server_log_block_mgr.cpp:800) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=8] try_resize success(ret=0, log_disk_size=32212254720, total_log_disk_size=214429704192, this={dir::"/mnt/myoceanbase/oceanbase/data//clog/log_pool", dir_fd:15, meta_fd:16, log_pool_meta:{curr_total_size:32212254720, next_total_size:32212254720, status:0}, min_block_id:15037, max_block_id:15335, min_log_disk_size_for_all_tenants_:15032385536, is_inited:true}) [2024-10-16 14:57:35.470445] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:132) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0B-0-0] [lt=11] refresh tenant config(tenants=[1, 1001, 1002, 1003, 1004], ret=0) [2024-10-16 14:57:35.472903] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1210) [2024-10-16 14:57:35.486448] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:35.486458] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:35.486468] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:35.486473] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:35.486476] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:35.486465] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED1-0-0] [lt=9] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:35.486479] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:35.486482] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:35.486484] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:35.486482] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED1-0-0] [lt=15] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:35.486487] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:35.486490] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:35.486493] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:35.488128] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061855487303000, start_id=1729061855487303000, end_id=1729061855487303001) [2024-10-16 14:57:35.489711] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] ====== check clog disk timer task ====== [2024-10-16 14:57:35.489724] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:35.489735] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70615955, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:35.495808] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:35.497477] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:35.497493] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:35.497761] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=244, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:35.497776] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=56, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:35.497796] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1524) [2024-10-16 14:57:35.497804] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:35.497916] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C422B-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:35.497922] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C422B-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=826, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:35.497960] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C422C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=354, avg_log_submit_to_slide_time=845) [2024-10-16 14:57:35.501678] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061855040596002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=461076, in_cluster_service=true, cluster_version={val:1729061854637437004, v:0}, min_cluster_version={val:1729061854637437004, v:0}, max_cluster_version={val:1729061854637437004, v:0}, get_cluster_version_err=0, cluster_version_delta=864235, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228174, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=400893193, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:35.503169] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.508858] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:45686897415}, base_lsn={lsn:45497032704}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:35.508907] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=49] get_replay_process success(id={id:1}, replayed_log_size=189864711, unreplayed_log_size=0) [2024-10-16 14:57:35.508925] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1001, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=181, last_replayed_log_size(MB)=181, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:35.526229] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061855524358000, start_id=1729061855524358000, end_id=1729061855524358001) [2024-10-16 14:57:35.533694] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BD-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:35.533715] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BD-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061855138136003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061855138136003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:35.535193] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9BD-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061855533192000, start_id=1729061855533192000, end_id=1729061855533192001) [2024-10-16 14:57:35.537290] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9A279FA9BD-0-0] [lt=16] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.054953432083129883e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=302, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.073582535704922066e+03) [2024-10-16 14:57:35.540213] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9BD-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061855538731000, start_id=1729061855538731000, end_id=1729061855538731001) [2024-10-16 14:57:35.541673] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=13] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:35.541688] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] [APPLY STAT CB APPEND COST TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=41, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:57:35.541695] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=2710, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:57:35.541698] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:57:35.541701] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=-621, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:57:35.541705] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=2130, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:57:35.542551] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061855542450000, start_id=1729061855542450000, end_id=1729061855542450001) [2024-10-16 14:57:35.542566] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] ts response task statistics(alloc_count=261364190, free_count=130682094) [2024-10-16 14:57:35.542576] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:35.544232] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D56-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:35.544277] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D56-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:35.544289] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D56-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:35.555278] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=38] get number(ret=0, service_type_=0, range=1, base_id=1729061855555231000, start_id=1729061855555231000, end_id=1729061855555231001) [2024-10-16 14:57:35.555693] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.557377] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:35.564613] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC40-0-0] [lt=15] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:35.564635] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC40-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:35.564653] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC40-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:35.568905] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.569324] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.569341] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.569802] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.570244] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BD5-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=9, avg_log_batch_cnt=0, total_group_log_size=1089, avg_group_log_size=121) [2024-10-16 14:57:35.572868] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=300, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:35.572887] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=89, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:35.572902] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:35.573112] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FDBA-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:35.573128] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FDBA-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1786, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:35.577680] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4A-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:35.577703] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4A-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061855138136004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061855138136004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:35.578673] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C4A-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061855578440000, start_id=1729061855578440000, end_id=1729061855578440001) [2024-10-16 14:57:35.579583] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:35.581370] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C4A-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061855580671000, start_id=1729061855580671000, end_id=1729061855580671001) [2024-10-16 14:57:35.584386] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=27] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1}, {id:1003}]) [2024-10-16 14:57:35.585406] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:35.586982] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:35.586997] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:35.587296] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061855115767000, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=471522, in_cluster_service=true, cluster_version={val:1729061854711458002, v:0}, min_cluster_version={val:1729061854711458002, v:0}, max_cluster_version={val:1729061854711458002, v:0}, get_cluster_version_err=0, cluster_version_delta=875831, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244095, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=526917128, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:35.599642] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0C-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061855590861, keepalive_ts:-1}) [2024-10-16 14:57:35.599658] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0C-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061855590861, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:35.601472] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] no need to update min/max version(new_version={val:1729061855128026000, v:0}, min_version={val:1729061855128026000, v:0}, max_version={val:1729061855128026000, v:0}) [2024-10-16 14:57:35.602588] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061855602252000, start_id=1729061855602252000, end_id=1729061855602252001) [2024-10-16 14:57:35.603352] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.608414] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4877, mem_used=10398720) [2024-10-16 14:57:35.624984] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:57:35.624998] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] try gc retain ctx [2024-10-16 14:57:35.625002] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:35.625595] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061867008855000, latest_log_ts:{val:1729061850421032001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:35.638431] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B98E-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061855638334000, start_id=1729061855638334000, end_id=1729061855638334001) [2024-10-16 14:57:35.638553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0C1B-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061855638334000, start_id=1729061855638334000, end_id=1729061855638334001) [2024-10-16 14:57:35.638575] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0C1C-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061855638334000, start_id=1729061855638334000, end_id=1729061855638334001) [2024-10-16 14:57:35.638683] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B992-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061855638334000, start_id=1729061855638334000, end_id=1729061855638334001) [2024-10-16 14:57:35.643858] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=47] success to update min reserved snapshot(reserved_snapshot=1729060050192549000, duration=1800, snapshot_gc_ts=1729061850192549000) [2024-10-16 14:57:35.644350] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=13] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060050192549000) [2024-10-16 14:57:35.644855] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=12] submit reserved snapshot log success(ls_id={id:1001}, new_snapshot_version=1729060050192549000) [2024-10-16 14:57:35.644867] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=10] update reserved snapshot finished(cost_ts=1012, reserved_snapshot=1729060050192549000) [2024-10-16 14:57:35.655752] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.666682] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:35.677522] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=6] query dest_info_ succ(dest_info=[[0, "file:///mnt/oceanbasedata/bddb/archive"]], archive_dest_changed=false) [2024-10-16 14:57:35.677538] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=14] finish to resize log external storage handler(current_ts=1729061855677536, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:35.685303] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F2B-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:35.688205] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061855687181000, start_id=1729061855687181000, end_id=1729061855687181001) [2024-10-16 14:57:35.695858] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:35.697853] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1636) [2024-10-16 14:57:35.697967] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4233-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:35.697977] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4233-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=815, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:35.698282] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=16] gc stale ls task succ [2024-10-16 14:57:35.702302] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061855702108000, start_id=1729061855702108000, end_id=1729061855702108001) [2024-10-16 14:57:35.702890] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get_max_applied_scn(scn={val:1729061855538731000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264666707}, palf_committed_end_scn_:{val:1729061855702108000, v:0}, last_check_scn_:{val:1729061855702108000, v:0}, max_applied_cb_scn_:{val:1729061855538731000, v:0}}) [2024-10-16 14:57:35.702913] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:35.703414] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.723864] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:35.726758] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061855726460000, start_id=1729061855726460000, end_id=1729061855726460001) [2024-10-16 14:57:35.726706] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:111799317, max_lsn:17493459931, max_scn:{val:1729061855542450001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7667642653}, reuse_lsn:{lsn:17493459931}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17451516891}}, last_submit_lsn:{lsn:17493459809}, last_submit_end_lsn:{lsn:17493459931}, last_submit_log_id:111799317, last_submit_log_pid:17, max_flushed_lsn:{lsn:17493459809}, max_flushed_end_lsn:{lsn:17493459931}, max_flushed_log_pid:17, committed_end_lsn:{lsn:17493459931}, last_slide_log_id:111799317, last_slide_scn:{val:1729061855542450001, v:0}, last_slide_lsn:{lsn:17493459809}, last_slide_end_lsn:{lsn:17493459931}, last_slide_log_pid:17, last_slide_log_accum_checksum:1505570313, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431d4530}, [StateMgr]={this:0x2b0e431d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:111799318, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261853177, palf_role_change_cb:0x2b0e431df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:18}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:18}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494261753074, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494261753074, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262053331, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728249494772393002, v:0}, min_block_id:241, min_block_min_scn:{val:1728207118648006001, v:0}, base_lsn_for_block_gc:{lsn:17380134912}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:17380134912}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:757760}, readable_log_tail:{lsn:757760}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17514344448}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/meta", dir_fd:193, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66347008, block_header_serialize_buf_:0x2b0e431da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17493459931}, readable_log_tail:{lsn:17493459931}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17447239680}, min_scn:{val:1729032503563174001, v:0}, curr_block_id:260, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/log", dir_fd:195, min_block_id:241, max_block_id:261, curr_writable_block_id:260}, logical_block_size_:67104768, curr_block_writable_size_:20884517, block_header_serialize_buf_:0x2b0e431dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261852794, this:0x2b0e431d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:7668488755}, saved_end_lsn:{lsn:7668488755}, last_submit_prepare_req_time_us:1722494261853360, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431d89b0}) [2024-10-16 14:57:35.726847] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=138] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061855672905, last_flushed_end_lsn:{lsn:17493459931}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061855673002, last_flushed_end_lsn:{lsn:17493459931}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061855673184, last_flushed_end_lsn:{lsn:17493459931}}]) [2024-10-16 14:57:35.726859] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=111799318, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061855672905, last_flushed_end_lsn:{lsn:17493459931}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061855673002, last_flushed_end_lsn:{lsn:17493459931}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061855673184, last_flushed_end_lsn:{lsn:17493459931}}]) [2024-10-16 14:57:35.735879] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:35.735899] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=14] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=213) [2024-10-16 14:57:35.754627] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] no need to update min/max version(new_version={val:1729061855354071000, v:0}, min_version={val:1729061855354071000, v:0}, max_version={val:1729061855354071000, v:0}) [2024-10-16 14:57:35.755729] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=2] gc stale ls task succ [2024-10-16 14:57:35.755753] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:35.755760] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:35.755815] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.770509] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:35.770855] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=15] gc stale ls task succ [2024-10-16 14:57:35.771374] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90A9-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:35.771383] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90A9-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8853, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:35.771388] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90A9-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=394, avg_log_submit_to_slide_time=8847) [2024-10-16 14:57:35.771392] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:35.771441] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:35.771449] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:35.772737] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:35.772750] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:35.776481] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D67-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:35.776508] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D67-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061855638334002, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061855638334002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:35.777524] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D67-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061855776723000, start_id=1729061855776723000, end_id=1729061855776723001) [2024-10-16 14:57:35.779880] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:35.780195] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D67-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061855779971000, start_id=1729061855779971000, end_id=1729061855779971001) [2024-10-16 14:57:35.780284] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D67-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:35.780298] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D67-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:35.781351] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=13] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:35.788107] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.788445] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.788464] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.788477] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.788488] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.788499] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.788716] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.789298] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.789298] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.789435] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.789577] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:35.793913] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15DA-0-0] [lt=6] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:35.794110] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:35.802682] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061855802002000, start_id=1729061855802002000, end_id=1729061855802002001) [2024-10-16 14:57:35.803475] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.808033] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=20] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}) [2024-10-16 14:57:35.823352] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=17] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:35.824777] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1002}, {id:1}]) [2024-10-16 14:57:35.825439] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:35.825699] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BB0-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=650, avg_log_submit_to_slide_time=1041) [2024-10-16 14:57:35.828209] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:35.828225] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:35.828238] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52352918, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:35.833694] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=61] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:35.834291] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:35.834340] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=14] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061854425066000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:35.852380] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4114, mem_used=8318976) [2024-10-16 14:57:35.855455] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061855854764000, start_id=1729061855854764000, end_id=1729061855854764001) [2024-10-16 14:57:35.855850] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.879426] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=11] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:35.880068] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=19] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:35.880121] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061851830492000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:35.886222] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=369, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:35.886242] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=348, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:35.886287] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:35.887389] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1002, min_version={val:1729061855138136004, v:0}, base_version={val:1729061855115767001, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061855542450001, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061855887375, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061855138136004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061855756407, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061855138136005, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061855791831, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:35.887672] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=12] ====== traversal_flush timer task ====== [2024-10-16 14:57:35.887847] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D46-0-0] [lt=7] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061855542450002, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061852531625004, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870098662000, latest_log_ts:{val:1729061853539570001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:35.887977] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D46-0-0] [lt=86] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061855542450002, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061852531625004, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:35.888041] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D46-0-0] [lt=31] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061855638334002, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061852531625004, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:35.888107] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D46-0-0] [lt=32] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061855638334002, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061852531625004, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:35.888137] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D46-0-0] [lt=29] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:35.888216] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061855886636000, start_id=1729061855886636000, end_id=1729061855886636001) [2024-10-16 14:57:35.895902] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:35.897712] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1210, avg_group_log_size=121) [2024-10-16 14:57:35.897731] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1210) [2024-10-16 14:57:35.898271] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C423A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=318, avg_log_submit_to_slide_time=688) [2024-10-16 14:57:35.899730] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=19] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:57:35.901004] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=10] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:35.902489] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=55] get number(ret=0, service_type_=0, range=1, base_id=1729061855901152000, start_id=1729061855901152000, end_id=1729061855901152001) [2024-10-16 14:57:35.903517] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.921660] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:35.926737] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061855925883000, start_id=1729061855925883000, end_id=1729061855925883001) [2024-10-16 14:57:35.926806] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:35.926818] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=9928, avg_group_log_size=522) [2024-10-16 14:57:35.926827] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9927) [2024-10-16 14:57:35.926834] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:35.927257] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=554, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:35.927268] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9928) [2024-10-16 14:57:35.927278] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=341, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:35.927292] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9927) [2024-10-16 14:57:35.927307] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:35.927522] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D57CB-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:35.927528] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D57CB-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=1227, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:35.927531] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D57CB-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=987, avg_log_submit_to_slide_time=1197) [2024-10-16 14:57:35.928729] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061851896611, last_first_register_time_us=1722494150998907, register_time_us=1729061851896611, curr_time_us=1729061855928728) [2024-10-16 14:57:35.928762] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061855928746, last_submit_register_req_time_us=1729061855928746, register_parent_reason=3) [2024-10-16 14:57:35.928773] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:35.941505] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=2660, mem_used=6239232) [2024-10-16 14:57:35.949407] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=4] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:35.949423] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:35.949431] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=148) [2024-10-16 14:57:35.954750] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:35.955865] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:35.957022] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7701-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:35.957258] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=595842, this={dio_aligned_buf:{buf_write_offset:2044, buf_padding_size:2052, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111160211, truncate_used_ts:41410929}, log_block_size:67108864, dir_fd:177, io_fd:142}, aligned_buf_len=4096, aligned_block_offset=593920, buf_len=122, total_write_size_=13366, total_write_size_after_dio_=421888, ob_pwrite_used_ts=53729, count=100, each_pwrite_cost=537) [2024-10-16 14:57:35.964954] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.95, rt: 278.25; ) [2024-10-16 14:57:35.964980] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:35.965018] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.95, rt: 849.50; ) [2024-10-16 14:57:35.965026] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:35.965037] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:35.973129] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=39] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=231, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:35.973150] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:35.973160] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:35.973303] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FDC6-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=351, avg_log_submit_to_slide_time=604) [2024-10-16 14:57:35.980771] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:35.985505] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F31-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:35.985527] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F31-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:35.985542] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F31-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497358, log_task={header:{begin_lsn:{lsn:207589540390}, end_lsn:{lsn:207589540512}, log_id:218497358, min_scn:{val:1729061855638334003, v:0}, max_scn:{val:1729061855638334003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589540268}, prev_proposal_id:81, committed_end_lsn:{lsn:207589540390}, data_checksum:940009244, accum_checksum:4041245380, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061855885401, freeze_ts:1729061855885401, submit_ts:1729061855885406, flushed_ts:1729061855886262, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:856}, fs_cb_begin_ts=1729061855985536, log_life_time=100135) [2024-10-16 14:57:35.989540] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061855989539, bytes: 4446811435424, bw: 0.331258 MB/s, add_ts: 1001407, add_bytes: 347838 [2024-10-16 14:57:35.997994] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:35.998011] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:35.998259] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=218, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:35.998271] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=48, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:35.998310] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:36.002785] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061856002326000, start_id=1729061856002326000, end_id=1729061856002326001) [2024-10-16 14:57:36.003533] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.019664] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061856019663, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007595, add_bytes: 0 [2024-10-16 14:57:36.023646] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061856023069000, start_id=1729061856023069000, end_id=1729061856023069001) [2024-10-16 14:57:36.025287] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=50] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3657, avg_group_log_size=332) [2024-10-16 14:57:36.025630] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=365, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:36.025652] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=141, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:36.026243] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=22][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fcaa35fa0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:199307155} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729057415940861 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:3518}, prev=this=0x2b0fcaa35180 trans_version={val:1729061736027587000, v:0} scn={val:1729061736027587000, v:0} tx_id={txid:199306951} prev=0x2b0fcaa35000 next=(nil) modify_count=73 acc_checksum=2321570609 version=1729057415940861 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:6315}, snapshot_version={val:1729061856025525000, v:0}, *this={this=0x2b0fcaa04a30 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fcaa35180 latest_compact_node=0x2b0fcaa33e60 max_trans_version={val:1729061736027587000, v:0} max_trans_id=199306951 max_elr_trans_version={val:1729061736027587000, v:0} max_elr_trans_id=199306951 latest_compact_ts=1729061616026649 last_compact_cnt=71 total_trans_node_cnt=74 max_modify_scn={val:1729061736027587000, v:0} min_modify_scn={val:1729057415944571000, v:0}}) [2024-10-16 14:57:36.026282] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=39][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fcaa35fa0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:199307155} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729057415940861 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:3518}, ctx={this:0x2b0e3f4d2080, ls_id:{id:1}, ls:0x2b0e3aec4150, branch:0, timeout:1729061885925760, tablet_id:{id:351}, table_iter:0x2b0e3f4d1210, table_version:1717582245152744, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061885925760, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061856025525000, v:0}, tx_id:{txid:199307155}, scn:{branch:0, seq:3515}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e3aed2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:199307155}, tx_desc:{this:0x2b0eb9fd7b50, tx_id:{txid:199307155}, state:2, addr:"10.0.4.8:2890", tenant_id:1003, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:-1, active_ts:1729061856014161, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1729061886014155, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061856022011, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e2c657750, mem_ctx:0x2b0e2c658cd0, tx_scn:{branch:0, seq:3518}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:36.026338] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:199307155} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:199307155}, data:0x2b0e3f4ce438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729057415940861, seq_no:{branch:0, seq:3518}, scn:{val:4611686018427387903, v:0}, column_cnt:17}) [2024-10-16 14:57:36.026356] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":1004},{"BIGINT":1004},{"BIGINT":3},{"BIGINT":74},{"BIGINT":1002}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:199307155} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:36.026371] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:36.026377] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=351, col_descs=[cnt:17, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=17,cells=[{"BIGINT":1004},{"BIGINT":1004},{"BIGINT":3},{"BIGINT":74},{"BIGINT":1002},{"TIMESTAMP":"2024-10-16 06:57:36.016281"},{"TIMESTAMP":"2024-10-16 06:57:36.016281"},{"BIGINT":1},{"BIGINT UNSIGNED":1729053449237798000},{"BIGINT UNSIGNED":14528194815},{"BIGINT UNSIGNED":14538479230},{"BIGINT UNSIGNED":1729061854137257001},{"VARCHAR":"DOING", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT":217},{"BIGINT":10284415},{"BIGINT":10284415},{"BIGINT":10284415},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:36.026437] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=38][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:36.026742] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4571}, to_seq_no={branch:0, seq:3516}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:199307155} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:36.026783] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:145) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=18] transaction rpc statistics(total_rt=161, total_process=3, avg_rt=40) [2024-10-16 14:57:36.028516] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061856027618000, start_id=1729061856027618000, end_id=1729061856027618001) [2024-10-16 14:57:36.042869] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061856042513000, start_id=1729061856042513000, end_id=1729061856042513001) [2024-10-16 14:57:36.042897] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061856042895], gts:1729061856042513001, latest_srr:[mts=1729061856042895]}) [2024-10-16 14:57:36.042888] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9986D1C20E-0-0] [lt=7][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fcaa363c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:199326023} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729057415940861 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:4211}, prev=this=0x2b0fcaa355a0 trans_version={val:1729061736042509000, v:0} scn={val:1729061736042509000, v:0} tx_id={txid:199325820} prev=0x2b0fcaa35420 next=(nil) modify_count=73 acc_checksum=1885359649 version=1729057415940861 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:5435}, snapshot_version={val:1729061856042513000, v:0}, *this={this=0x2b0fcaa0d670 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fcaa355a0 latest_compact_node=0x2b0fcaa34360 max_trans_version={val:1729061736042509000, v:0} max_trans_id=199325820 max_elr_trans_version={val:1729061736042509000, v:0} max_elr_trans_id=199325820 latest_compact_ts=1729061616044609 last_compact_cnt=71 total_trans_node_cnt=74 max_modify_scn={val:1729061736042509000, v:0} min_modify_scn={val:1729057415963433000, v:0}}) [2024-10-16 14:57:36.042920] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9986D1C20E-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fcaa363c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:199326023} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729057415940861 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:4211}, ctx={this:0x2b0e3f4d2080, ls_id:{id:1}, ls:0x2b0e3aec4150, branch:0, timeout:1729061885942468, tablet_id:{id:351}, table_iter:0x2b0e3f4d1210, table_version:1717582245152744, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061885942468, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061856042513000, v:0}, tx_id:{txid:199326023}, scn:{branch:0, seq:3169}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e3aed2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:199326023}, tx_desc:{this:0x2b0eb9fb8150, tx_id:{txid:199326023}, state:2, addr:"10.0.4.9:2890", tenant_id:1003, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:-1, active_ts:1729061856007212, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:1729061886007204, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061856038303, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e2c657750, mem_ctx:0x2b0e2c658cd0, tx_scn:{branch:0, seq:4211}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:36.042981] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9986D1C20E-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:199326023} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:199326023}, data:0x2b0e3f4ce438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729057415940861, seq_no:{branch:0, seq:4211}, scn:{val:4611686018427387903, v:0}, column_cnt:17}) [2024-10-16 14:57:36.042996] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9986D1C20E-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":1004},{"BIGINT":1004},{"BIGINT":3},{"BIGINT":74},{"BIGINT":1003}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:199326023} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:36.043008] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9986D1C20E-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:36.043014] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9986D1C20E-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=351, col_descs=[cnt:17, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=17,cells=[{"BIGINT":1004},{"BIGINT":1004},{"BIGINT":3},{"BIGINT":74},{"BIGINT":1003},{"TIMESTAMP":"2024-10-16 06:57:36.009331"},{"TIMESTAMP":"2024-10-16 06:57:36.009331"},{"BIGINT":1},{"BIGINT UNSIGNED":1729053449237798000},{"BIGINT UNSIGNED":14342072172},{"BIGINT UNSIGNED":14352364856},{"BIGINT UNSIGNED":1729061853636618001},{"VARCHAR":"DOING", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT":214},{"BIGINT":10292684},{"BIGINT":10292684},{"BIGINT":10292684},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:36.043074] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [22487][T1003_L5_G0][T1003][YB4A0A000409-00061E9986D1C20E-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:36.043540] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9986D1C20E-0-0] [lt=10] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4214}, to_seq_no={branch:0, seq:3170}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:199326023} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:36.049065] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:36.049148] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1002 and flag like "%DUPLICATE%", tenant_id=1002, exec_tenant_id=1001, need_member_list=false) [2024-10-16 14:57:36.055552] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061856054428000, start_id=1729061856054428000, end_id=1729061856054428001) [2024-10-16 14:57:36.055928] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.070701] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=283, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:36.070724] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:36.070757] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:36.072890] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:36.072905] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:36.073159] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:36.073175] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:36.081324] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D4-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061856072431, keepalive_ts:-1}) [2024-10-16 14:57:36.081340] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D4-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061856072431, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:36.083169] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] worker thread began to shrink(tenant_->id()=1002, group_id_=19, token=0) [2024-10-16 14:57:36.084371] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:36.084551] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D44-0-0] [lt=11] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061855542450004, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061853036726000, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870721265000, latest_log_ts:{val:1729061854137257001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:36.084739] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D44-0-0] [lt=98] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061855542450004, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061853036726000, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:36.084885] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D44-0-0] [lt=66] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061855638334005, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061853036726000, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:36.085020] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D44-0-0] [lt=59] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061855638334003, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061853036726000, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:36.085081] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D44-0-0] [lt=59] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:36.086029] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:36.086050] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598808, miss_cnt:9, cache_read_size:149777905197, hit_ratio:"9.999999345924609484e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:36.086305] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106793][T1002_L0_G19][T1002][Y0-0000000000000000-0-0] [lt=7] worker thread exit(tenant_->id()=1002, workers_.get_size()=1) [2024-10-16 14:57:36.086326] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106793][T1002_L0_G19][T0][Y0-0000000000000000-0-0] [lt=15] tenant thread end_run(id_=1002, ret=0, thread_count_=189) [2024-10-16 14:57:36.088347] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061856086423000, start_id=1729061856086423000, end_id=1729061856086423001) [2024-10-16 14:57:36.095978] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:36.097400] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=15] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:36.098275] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1210) [2024-10-16 14:57:36.098303] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=40] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1210) [2024-10-16 14:57:36.102487] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061856101561000, start_id=1729061856101561000, end_id=1729061856101561001) [2024-10-16 14:57:36.103596] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.109810] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601385, offset=9620974}, end_cursor:ObLogCursor{file_id=192, log_id=47601386, offset=9621258}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d88050, len:530, log_data_len:284, seq:47601385, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:36.112332] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA49E-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061856111152000, start_id=1729061856111152000, end_id=1729061856111152001) [2024-10-16 14:57:36.113416] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:36.113428] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:36.114705] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601386, offset=9621258}, end_cursor:ObLogCursor{file_id=192, log_id=47601387, offset=9621532}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dd4050, len:4342, log_data_len:274, seq:47601386, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:36.115689] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87A-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:36.115709] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87A-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061855638334005, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061855638334005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:36.117163] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA87A-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061856117109000, start_id=1729061856117109000, end_id=1729061856117109001) [2024-10-16 14:57:36.117621] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601387, offset=9621532}, end_cursor:ObLogCursor{file_id=192, log_id=47601388, offset=9621806}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2df0050, len:4068, log_data_len:274, seq:47601387, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:36.117858] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601388, offset=9621806}, end_cursor:ObLogCursor{file_id=192, log_id=47601389, offset=9622080}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bf8050, len:3794, log_data_len:274, seq:47601388, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:36.118111] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561219, miss_cnt:1, cache_read_size:152590670250, hit_ratio:"9.999999928346857692e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:36.119358] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA87A-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061856119224000, start_id=1729061856119224000, end_id=1729061856119224001) [2024-10-16 14:57:36.119850] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:36.126766] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061856125593000, start_id=1729061856125593000, end_id=1729061856125593001) [2024-10-16 14:57:36.127465] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=7] gc stale ls task succ [2024-10-16 14:57:36.129582] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1735, mem_used=4159488) [2024-10-16 14:57:36.130283] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [21543][T1_CdcSrv][T1][Y0-0000000000000000-0-0] [lt=11] finish to resize log external storage handler(current_ts=1729061856130281, tenant_max_cpu=3, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:36.130589] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0FC3-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061856130113000, start_id=1729061856130113000, end_id=1729061856130113001) [2024-10-16 14:57:36.134039] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257730, offset=39023999}, end_cursor:ObLogCursor{file_id=197, log_id=49257731, offset=39024282}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2df4050, len:2689, log_data_len:283, seq:49257730, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:36.136354] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB369-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061856135651000, start_id=1729061856135651000, end_id=1729061856135651001) [2024-10-16 14:57:36.137218] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257731, offset=39024282}, end_cursor:ObLogCursor{file_id=197, log_id=49257732, offset=39024564}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cf4050, len:2406, log_data_len:282, seq:49257731, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:36.138874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B995-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061856138655000, start_id=1729061856138655000, end_id=1729061856138655001) [2024-10-16 14:57:36.138894] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B993-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061856138655000, start_id=1729061856138655000, end_id=1729061856138655001) [2024-10-16 14:57:36.139013] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0C21-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061856138655000, start_id=1729061856138655000, end_id=1729061856138655001) [2024-10-16 14:57:36.139089] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B997-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061856138655000, start_id=1729061856138655000, end_id=1729061856138655001) [2024-10-16 14:57:36.139428] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257732, offset=39024564}, end_cursor:ObLogCursor{file_id=197, log_id=49257733, offset=39024846}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c6c050, len:2124, log_data_len:282, seq:49257732, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:36.139567] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257733, offset=39024846}, end_cursor:ObLogCursor{file_id=197, log_id=49257734, offset=39025128}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d2c050, len:1842, log_data_len:282, seq:49257733, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:36.139626] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650158, miss_cnt:32, cache_read_size:8050410502, hit_ratio:"9.999994892274070013e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:36.139652] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656895, miss_cnt:17, cache_read_size:8050157873, hit_ratio:"9.999997286811709296e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:36.144538] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D62-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:36.144553] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D62-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=32713, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:36.144557] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D62-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=357, avg_log_submit_to_slide_time=32705) [2024-10-16 14:57:36.144622] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:36.144725] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=265, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:36.144735] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:36.144743] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=59, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:36.144769] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:36.155986] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=39][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.157681] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7705-0-0] [lt=18][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938212, log_task={header:{begin_lsn:{lsn:213058230268}, end_lsn:{lsn:213058230390}, log_id:220938212, min_scn:{val:1729061855638334004, v:0}, max_scn:{val:1729061855638334004, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058230146}, prev_proposal_id:88, committed_end_lsn:{lsn:213058230268}, data_checksum:1715074492, accum_checksum:3717779399, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061856057350, freeze_ts:1729061856057350, submit_ts:1729061856057355, flushed_ts:1729061856064983, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:7628}, fs_cb_begin_ts=1729061856157674, log_life_time=100324) [2024-10-16 14:57:36.166378] INFO [SERVER.OMT] recv_group_request (ob_tenant.cpp:1419) [21192][pnio1][T0][YB4A0A000409-00061E9A33F4DC7E-0-0] [lt=3] worker thread created(id()=1002, group->group_id_=19) [2024-10-16 14:57:36.166378] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107454][][T1002][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1002, ret=0, thread_count_=190) [2024-10-16 14:57:36.166397] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107454][][T1002][Y0-0000000000000000-0-0] [lt=17] Init thread local success [2024-10-16 14:57:36.166422] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC7E-0-0] [lt=5] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:36.166439] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC7E-0-0] [lt=13] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:36.166457] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC7E-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:36.167365] INFO [LIB] runTimerTask (ob_work_queue.cpp:24) [21225][RSInspectTask][T0][Y0-0000000000000000-0-0] [lt=14] add async task(this=tasktype:N9oceanbase10rootserver21ObPurgeRecyclebinTaskE) [2024-10-16 14:57:36.169938] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:36.169957] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:36.169964] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:36.169967] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:36.169970] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:36.169973] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16724) [2024-10-16 14:57:36.171305] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BE1-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:36.181063] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:36.181685] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5026-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:36.181698] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5026-0-0] [lt=12] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:36.185673] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D6C-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510324054066, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410589, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:36.197017] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=33] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:36.203398] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061856203224000, start_id=1729061856203224000, end_id=1729061856203224001) [2024-10-16 14:57:36.203613] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.204009] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1729061855542450004, v:0}, new_max_version={val:1729061855542450004, v:0}, last_min_version={val:1729061855542450002, v:0}, last_max_version={val:1729061855542450002, v:0}, total_time_us=1389, delta_us=661558) [2024-10-16 14:57:36.204026] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1729061855542450004, v:0}, min_version={val:1729061855542450004, v:0}, max_version={val:1729061855542450004, v:0}, in_service=true, leader_epoch=17) [2024-10-16 14:57:36.225396] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=26, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:36.225418] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:36.228629] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:57:36.228648] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=16] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=83, cache_node_num=83) [2024-10-16 14:57:36.228654] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=83, cache_node_num=83) [2024-10-16 14:57:36.228659] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:36.230762] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=83, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}, {obj_id:78893917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}]) [2024-10-16 14:57:36.230873] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=109] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:36.241962] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:57:36.242720] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061856241349000, start_id=1729061856241349000, end_id=1729061856241349001) [2024-10-16 14:57:36.255724] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061856254673000, start_id=1729061856254673000, end_id=1729061856254673001) [2024-10-16 14:57:36.256047] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.257316] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7707-0-0] [lt=26] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:36.257735] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F405F-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:36.257746] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F405F-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=25441, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:36.257748] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1455, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:36.257782] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=25] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=62, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:36.257790] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:36.261598] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:36.262112] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21658][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3DEA-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510324130507, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410590, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:36.270542] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BE3-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:36.273019] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:36.273045] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:36.273070] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:36.273076] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:36.273215] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:36.286287] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4242-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:36.286303] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4242-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9761, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:36.288595] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061856286441000, start_id=1729061856286441000, end_id=1729061856286441001) [2024-10-16 14:57:36.289220] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=21] get_max_applied_scn(scn={val:1729061856042513000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959688511}, palf_committed_end_scn_:{val:1729061856042513000, v:0}, last_check_scn_:{val:1729061856042513000, v:0}, max_applied_cb_scn_:{val:1729061856042513000, v:0}}) [2024-10-16 14:57:36.289244] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:36.289300] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=43] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:57:36.289308] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] get_min_unreplayed_log_info(lsn={lsn:213058230634}, scn={val:1729061856138655003, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061856257729, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058230634}, next_to_submit_scn_:{val:1729061856138655003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058230512}, end_lsn:{lsn:213058230634}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061856138655002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061856138655002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:1178765681, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061856138655002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561222, miss_cnt:1, cache_read_size:152590670616, hit_ratio:"9.999999928346858802e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:36.295890] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] worker thread began to shrink(tenant_->id()=1001, group_id_=1, token=0) [2024-10-16 14:57:36.296050] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:36.298125] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1210, avg_group_log_size=121) [2024-10-16 14:57:36.298138] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1210) [2024-10-16 14:57:36.298275] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106892][T1001_L0_G1][T1001][Y0-0000000000000000-0-0] [lt=7] worker thread exit(tenant_->id()=1001, workers_.get_size()=2) [2024-10-16 14:57:36.298291] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106892][T1001_L0_G1][T0][Y0-0000000000000000-0-0] [lt=12] tenant thread end_run(id_=1001, ret=0, thread_count_=171) [2024-10-16 14:57:36.301858] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1003, min_version={val:1729061856101561000, v:0}, base_version={val:1729061855901152000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061856101561000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061856301846, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061856135651000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061856255868, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061856101561000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061856210463, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:36.302673] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061856302009000, start_id=1729061856302009000, end_id=1729061856302009001) [2024-10-16 14:57:36.303673] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.311536] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:36.311554] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:36.311572] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373584125, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:36.311733] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:36.311819] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA9-0-0] [lt=12] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061869029444000, latest_log_ts:{val:1729061852392712001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:36.311875] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DA9-0-0] [lt=53] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:36.314597] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=31] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6643, mem_used=12478464) [2024-10-16 14:57:36.322473] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=17] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:36.327053] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061856326611000, start_id=1729061856326611000, end_id=1729061856326611001) [2024-10-16 14:57:36.328325] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [WRS] update tenant weak read server version(tenant_id=1001, server_version={version:{val:1729061856125593000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061856328305}, version_delta=202731) [2024-10-16 14:57:36.342645] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55C6-0-0] [lt=17] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10006, add_timestamp:1729061856342636}) [2024-10-16 14:57:36.342674] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E31-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10006, locations=[{table_id:10006, partition_id:10006, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856342660, sql_renew_time:1729061856342660, is_mark_fail:false}]) [2024-10-16 14:57:36.342696] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E31-0-0] [lt=22] success to process renew task(task={tenant_id:1, table_id:10006, add_timestamp:1729061856342636}, locations=[{table_id:10006, partition_id:10006, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856342660, sql_renew_time:1729061856342660, is_mark_fail:false}]) [2024-10-16 14:57:36.342799] WDIAG [SHARE] inner_to_show_str (ob_system_variable.cpp:516) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55C6-0-0] [lt=15][errcode=0] sys var casted obj is not ObVarcharType(ret=0, value={"NULL":"NULL"}, *res_obj={"NULL":"NULL"}, get_name()=secure_file_priv) [2024-10-16 14:57:36.352212] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E04-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061856352042000, start_id=1729061856352042000, end_id=1729061856352042001) [2024-10-16 14:57:36.356110] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.357572] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=931, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:36.357595] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:36.371860] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=20] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:57:36.373554] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FDD2-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:36.373566] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FDD2-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=573, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:36.373571] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FDD2-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=233, avg_log_submit_to_slide_time=557) [2024-10-16 14:57:36.381345] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:36.383005] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:36.383021] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:36.383040] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=10] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:36.387912] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] get_max_applied_scn(scn={val:1729061856042513000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023094250}, palf_committed_end_scn_:{val:1729061856130113000, v:0}, last_check_scn_:{val:1729061856130113000, v:0}, max_applied_cb_scn_:{val:1729061856042513000, v:0}}) [2024-10-16 14:57:36.387935] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=20] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:36.388706] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=6] ====== tenant freeze timer task ====== [2024-10-16 14:57:36.388719] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:36.388730] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:36.394174] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=17] gc stale ls task succ [2024-10-16 14:57:36.396214] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=18] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:36.398988] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=16] new mysql sessid created(conn.sessid_=3221735410, support_ssl=false) [2024-10-16 14:57:36.399021] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12] sm conn init succ(conn.sessid_=3221735410, sess.client_addr_="127.0.0.1:33530") [2024-10-16 14:57:36.399029] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0f27406030, session_id:3221735410, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399045] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] new mysql sessid created(conn.sessid_=3221737422, support_ssl=false) [2024-10-16 14:57:36.399057] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221737422, sess.client_addr_="127.0.0.1:33540") [2024-10-16 14:57:36.399060] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f27414df0, session_id:3221737422, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:214}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399071] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221737478, support_ssl=false) [2024-10-16 14:57:36.399081] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221737478, sess.client_addr_="127.0.0.1:33546") [2024-10-16 14:57:36.399089] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] accept one succ(*s={this:0x2b0f2742c030, session_id:3221737478, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:215}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399097] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221649591, support_ssl=false) [2024-10-16 14:57:36.399107] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221649591, sess.client_addr_="127.0.0.1:33552") [2024-10-16 14:57:36.399111] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f2742cdf0, session_id:3221649591, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:218}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399119] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221739053, support_ssl=false) [2024-10-16 14:57:36.399129] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221739053, sess.client_addr_="127.0.0.1:33564") [2024-10-16 14:57:36.399132] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f2749c030, session_id:3221739053, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:221}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399170] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=22] new mysql sessid created(conn.sessid_=3221736972, support_ssl=false) [2024-10-16 14:57:36.399184] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] sm conn init succ(conn.sessid_=3221736972, sess.client_addr_="127.0.0.1:33538") [2024-10-16 14:57:36.399187] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e2917e030, session_id:3221736972, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:223}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399196] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221728400, support_ssl=false) [2024-10-16 14:57:36.399206] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221728400, sess.client_addr_="127.0.0.1:33544") [2024-10-16 14:57:36.399210] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2] accept one succ(*s={this:0x2b0e5cdcc030, session_id:3221728400, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:224}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399218] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221737879, support_ssl=false) [2024-10-16 14:57:36.399228] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221737879, sess.client_addr_="127.0.0.1:33554") [2024-10-16 14:57:36.399232] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdb6df0, session_id:3221737879, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:225}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399240] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221737567, support_ssl=false) [2024-10-16 14:57:36.399251] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221737567, sess.client_addr_="127.0.0.1:33556") [2024-10-16 14:57:36.399254] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdf8df0, session_id:3221737567, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:227}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399249] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D6E-0-0] [lt=14] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12276, add_timestamp:1729061856399243}) [2024-10-16 14:57:36.399274] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=11] new mysql sessid created(conn.sessid_=3221657713, support_ssl=false) [2024-10-16 14:57:36.399287] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] sm conn init succ(conn.sessid_=3221657713, sess.client_addr_="127.0.0.1:33534") [2024-10-16 14:57:36.399290] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdb8df0, session_id:3221657713, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:229}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399303] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=10] new mysql sessid created(conn.sessid_=3221739142, support_ssl=false) [2024-10-16 14:57:36.399314] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221739142, sess.client_addr_="127.0.0.1:33558") [2024-10-16 14:57:36.399317] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] accept one succ(*s={this:0x2b0e5cdfcdf0, session_id:3221739142, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:230}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399365] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=8] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:57:36.399370] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=5] try gc retain ctx [2024-10-16 14:57:36.399373] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:36.399557] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E32-0-0] [lt=8] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12276, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:36.399672] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=43] new mysql sessid created(conn.sessid_=3221736398, support_ssl=false) [2024-10-16 14:57:36.399699] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=7] sm conn init succ(conn.sessid_=3221736398, sess.client_addr_="127.0.0.1:33532") [2024-10-16 14:57:36.399706] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=4] accept one succ(*s={this:0x2b0f2749cdf0, session_id:3221736398, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:232}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399720] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=9] new mysql sessid created(conn.sessid_=3221726815, support_ssl=false) [2024-10-16 14:57:36.399732] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221726815, sess.client_addr_="127.0.0.1:33536") [2024-10-16 14:57:36.399736] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f2749e030, session_id:3221726815, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:235}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399744] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221649181, support_ssl=false) [2024-10-16 14:57:36.399754] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221649181, sess.client_addr_="127.0.0.1:33542") [2024-10-16 14:57:36.399758] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f2749edf0, session_id:3221649181, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:236}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399767] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221739050, support_ssl=false) [2024-10-16 14:57:36.399778] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221739050, sess.client_addr_="127.0.0.1:33548") [2024-10-16 14:57:36.399781] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f03c0e030, session_id:3221739050, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:237}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399789] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221734926, support_ssl=false) [2024-10-16 14:57:36.399799] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221734926, sess.client_addr_="127.0.0.1:33550") [2024-10-16 14:57:36.399802] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f03c0edf0, session_id:3221734926, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:238}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399811] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221735918, support_ssl=false) [2024-10-16 14:57:36.399821] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221735918, sess.client_addr_="127.0.0.1:33560") [2024-10-16 14:57:36.399825] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] accept one succ(*s={this:0x2b0f03c10030, session_id:3221735918, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:240}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399834] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221568553, support_ssl=false) [2024-10-16 14:57:36.399846] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221568553, sess.client_addr_="127.0.0.1:33562") [2024-10-16 14:57:36.399850] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f03c10df0, session_id:3221568553, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:241}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.399967] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061869029444000, latest_log_ts:{val:1729061852392712001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:36.400068] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=48] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=11137, avg_group_log_size=655) [2024-10-16 14:57:36.400076] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11146) [2024-10-16 14:57:36.400323] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E32-0-0] [lt=10] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12276, locations=[{table_id:12276, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856400310, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856400310, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856400310, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.400349] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E32-0-0] [lt=25] success to process renew task(task={tenant_id:1, table_id:12276, add_timestamp:1729061856399243}, locations=[{table_id:12276, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856400310, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856400310, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856400310, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.400813] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=300, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:36.400851] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=38] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11137) [2024-10-16 14:57:36.400861] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=97, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:36.400880] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11146) [2024-10-16 14:57:36.400892] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:36.401008] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD0DB-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:36.401015] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD0DB-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=803, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:36.401019] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD0DB-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=16, avg_log_gen_to_submit_time=43, avg_log_submit_to_flush_time=610, avg_log_submit_to_slide_time=759) [2024-10-16 14:57:36.401117] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.401132] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=7] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.401142] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.401147] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.401153] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.401158] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.401167] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.401171] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.401177] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.401181] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.401187] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.401191] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.401208] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.401212] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.401296] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB6-0-0] [lt=16] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.401322] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9C-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.401779] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9C-0-0] [lt=8] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221735918, proxy_sessid=0, client_sessid=3221735918, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.401779] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB6-0-0] [lt=5] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221568553, proxy_sessid=0, client_sessid=3221568553, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.401830] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9D-0-0] [lt=15] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.401832] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB7-0-0] [lt=15] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.402180] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9D-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221734926, proxy_sessid=0, client_sessid=3221734926, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.402185] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB7-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221739050, proxy_sessid=0, client_sessid=3221739050, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.402218] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9E-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.402222] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB8-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.402552] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9E-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221649181, proxy_sessid=0, client_sessid=3221649181, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.402589] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9F-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.402629] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB8-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221726815, proxy_sessid=0, client_sessid=3221726815, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.402934] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2C9F-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221736398, proxy_sessid=0, client_sessid=3221736398, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.402978] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.402990] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=5] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.402999] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.403005] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.403045] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB9-0-0] [lt=12] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.403054] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.403061] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=4] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.403067] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.403071] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.403077] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.403081] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.403087] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.403090] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.403139] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AB-0-0] [lt=15] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.403505] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AB-0-0] [lt=5] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221657713, proxy_sessid=0, client_sessid=3221657713, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.403512] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DB9-0-0] [lt=6] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221739142, proxy_sessid=0, client_sessid=3221739142, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.403546] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AC-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.403564] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBA-0-0] [lt=13] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.403901] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AC-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737567, proxy_sessid=0, client_sessid=3221737567, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.403940] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AD-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.403990] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBA-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737879, proxy_sessid=0, client_sessid=3221737879, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.404044] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBB-0-0] [lt=13] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.404284] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AD-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221728400, proxy_sessid=0, client_sessid=3221728400, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.404324] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.404407] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBB-0-0] [lt=5] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221736972, proxy_sessid=0, client_sessid=3221736972, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.405069] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAA-0-0] [lt=3] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11018, add_timestamp:1729061856405063}) [2024-10-16 14:57:36.405112] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061856400719000, start_id=1729061856400719000, end_id=1729061856400719001) [2024-10-16 14:57:36.405300] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21668][T1_L0_G0][T1][YB4A0A000407-00061E99793CF611-0-0] [lt=13] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11021, add_timestamp:1729061856405298}) [2024-10-16 14:57:36.405650] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1729061856405647}) [2024-10-16 14:57:36.406619] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.406629] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.407266] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.407288] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.407295] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.407309] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=12] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.407314] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.407353] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:36.407356] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.407367] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.407370] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.407377] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:36.407380] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.407385] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.407388] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAC-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.407620] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=3][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:10008, add_timestamp:1729061856407618}) [2024-10-16 14:57:36.407628] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=8][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:10008, add_timestamp:1729061856407618}) [2024-10-16 14:57:36.407633] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=4][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=10008, vtable_locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841415025, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841415025, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841415025, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.408533] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=17] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.408542] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.409164] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.409183] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.409189] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.409194] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.409198] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.409229] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:36.409232] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.409248] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=13] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.409251] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.409259] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:36.409262] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.409266] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.409269] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAD-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.410210] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.410218] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.410849] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.410869] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.410875] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.410884] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.410889] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.410908] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:36.410911] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.410919] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.410922] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.410929] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:36.410932] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.410936] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.410939] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.412000] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.412009] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.412639] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.412658] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.412664] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.412669] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.412673] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.412845] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:36.412850] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.412861] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.412864] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.412874] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:36.412877] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.412882] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.412885] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EAF-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.413087] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:290) [21668][T1_L0_G0][T1][YB4A0A000407-00061E99793CF611-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:36.413393] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E33-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11018, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:36.413780] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.413800] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=11] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.413809] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.413814] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.413820] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.413824] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.413831] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.413835] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.413843] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:36.413847] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:36.413925] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBC-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.414144] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AE-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.414396] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBC-0-0] [lt=7] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221739053, proxy_sessid=0, client_sessid=3221739053, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.414448] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBD-0-0] [lt=15] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.414526] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AE-0-0] [lt=6] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221649591, proxy_sessid=0, client_sessid=3221649591, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.414571] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AF-0-0] [lt=12] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.414816] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBD-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737478, proxy_sessid=0, client_sessid=3221737478, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.414862] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBE-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:36.414917] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37AF-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737422, proxy_sessid=0, client_sessid=3221737422, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.415255] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBE-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221735410, proxy_sessid=0, client_sessid=3221735410, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:36.415643] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E33-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11018, locations=[{table_id:11018, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415631, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415631, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415631, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.415666] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E33-0-0] [lt=22] success to process renew task(task={tenant_id:1, table_id:11018, add_timestamp:1729061856405063}, locations=[{table_id:11018, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415631, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415631, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415631, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.415691] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E34-0-0] [lt=13] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11021, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:36.415835] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E34-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11021, locations=[{table_id:11021, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415829, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415829, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415829, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.415848] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E34-0-0] [lt=14] success to process renew task(task={tenant_id:1, table_id:11021, add_timestamp:1729061856405298}, locations=[{table_id:11021, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415829, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415829, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856415829, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.415864] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E35-0-0] [lt=11] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:36.416102] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=13] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12142, add_timestamp:1729061856416098}) [2024-10-16 14:57:36.416117] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12068, add_timestamp:1729061856416116}) [2024-10-16 14:57:36.416124] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12084, add_timestamp:1729061856416123}) [2024-10-16 14:57:36.417084] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC91F-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11008, add_timestamp:1729061856417082}) [2024-10-16 14:57:36.419438] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC920-0-0] [lt=9][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12142, add_timestamp:1729061856419437}) [2024-10-16 14:57:36.419448] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC920-0-0] [lt=9][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12142, add_timestamp:1729061856419437}) [2024-10-16 14:57:36.419453] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC920-0-0] [lt=4][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12142, vtable_locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414871, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.419468] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC920-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12217, add_timestamp:1729061856419467}) [2024-10-16 14:57:36.419474] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC920-0-0] [lt=4][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12068, add_timestamp:1729061856419474}) [2024-10-16 14:57:36.419478] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC920-0-0] [lt=3][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12068, add_timestamp:1729061856419474}) [2024-10-16 14:57:36.419481] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC920-0-0] [lt=3][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12068, vtable_locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414899, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.419495] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC920-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11035, add_timestamp:1729061856419494}) [2024-10-16 14:57:36.419732] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9C-0-0] [lt=8] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11003, add_timestamp:1729061856419730}) [2024-10-16 14:57:36.420173] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=9] get_stable_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:36.420193] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1001][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=7] get_stable_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:36.420203] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1002][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=4] get_stable_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:36.420211] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1003][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] get_stable_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:36.420220] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1004][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] get_stable_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:36.420986] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.420995] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.421607] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:36.421627] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=8] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.421633] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.421638] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.421642] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.421675] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:36.421679] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.421687] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:36.421690] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.421699] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:36.421702] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.421706] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:36.421709] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:36.421961] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D71-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11037, add_timestamp:1729061856421959}) [2024-10-16 14:57:36.422207] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12325, add_timestamp:1729061856422205}) [2024-10-16 14:57:36.422266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061856415145000, start_id=1729061856415145000, end_id=1729061856415145001) [2024-10-16 14:57:36.422923] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9A-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1729061856422920}) [2024-10-16 14:57:36.423975] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D72-0-0] [lt=9][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:11017, add_timestamp:1729061856423974}) [2024-10-16 14:57:36.423984] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D72-0-0] [lt=9][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:11017, add_timestamp:1729061856423974}) [2024-10-16 14:57:36.423989] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D72-0-0] [lt=5][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=11017, vtable_locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414772, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414772, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061841414772, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.429706] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9A-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:36.430045] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3658) [2024-10-16 14:57:36.430693] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3193) [2024-10-16 14:57:36.430719] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3658) [2024-10-16 14:57:36.431534] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D72-0-0] [lt=15][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:36.431734] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061856430804000, start_id=1729061856430804000, end_id=1729061856430804001) [2024-10-16 14:57:36.431827] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=11] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:133533222, max_lsn:66390102574, max_scn:{val:1729061856415145000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:33216076378}, reuse_lsn:{lsn:66390102574}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:66348159534}}, last_submit_lsn:{lsn:66390102452}, last_submit_end_lsn:{lsn:66390102574}, last_submit_log_id:133533222, last_submit_log_pid:48, max_flushed_lsn:{lsn:66390102452}, max_flushed_end_lsn:{lsn:66390102574}, max_flushed_log_pid:48, committed_end_lsn:{lsn:66390102452}, last_slide_log_id:133533221, last_slide_scn:{val:1729061856254673000, v:0}, last_slide_lsn:{lsn:66390101865}, last_slide_end_lsn:{lsn:66390102452}, last_slide_log_pid:48, last_slide_log_accum_checksum:1399240953, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0de0acc530}, [StateMgr]={this:0x2b0de0ad0830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:133533222, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261740374, palf_role_change_cb:0x2b0de0ad7230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, reconfig_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, persistent_config_version:{proposal_id:48, config_seq:11}, ms_ack_list:0, resend_config_version:{proposal_id:48, config_seq:11}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494260838312, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494260838312, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0de0acdc70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261941355, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728741813401825001, v:0}, min_block_id:965, min_block_min_scn:{val:1728727703672931001, v:0}, base_lsn_for_block_gc:{lsn:66098196480}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, log_config_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, log_snapshot_meta:{version:1, base_lsn:{lsn:66098196480}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1634304}, readable_log_tail:{lsn:1634304}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66433720320}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/meta", dir_fd:128, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65470464, block_header_serialize_buf_:0x2b0de0ad29f8, flashback_version:0}, log_storage:{log_tail:{lsn:66390102574}, readable_log_tail:{lsn:66390102574}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66366615552}, min_scn:{val:1729057321384354001, v:0}, curr_block_id:989, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/log", dir_fd:134, min_block_id:965, max_block_id:990, curr_writable_block_id:989}, logical_block_size_:67104768, curr_block_writable_size_:43617746, block_header_serialize_buf_:0x2b0de0ad46b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261739251, this:0x2b0de0ad0f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:48, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:47, majority_max_lsn:{lsn:33216819320}, saved_end_lsn:{lsn:33216819320}, last_submit_prepare_req_time_us:1722494261740552, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0de0ad09b0}) [2024-10-16 14:57:36.431992] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=163] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061856430716, last_flushed_end_lsn:{lsn:66390102574}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061856257833, last_flushed_end_lsn:{lsn:66390102452}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061856256351, last_flushed_end_lsn:{lsn:66390102452}}]) [2024-10-16 14:57:36.432003] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=133533222, log_task={header:{begin_lsn:{lsn:66390102452}, end_lsn:{lsn:66390102574}, log_id:133533222, min_scn:{val:1729061856415145000, v:0}, max_scn:{val:1729061856415145000, v:0}, data_len:66, proposal_id:48, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:66390102452}, data_checksum:667309872, accum_checksum:506348163, is_padding_log:false, is_raw_write:false}, state_map:{val:30}, ref_cnt:0, gen_ts:1729061856430008, freeze_ts:1729061856430011, submit_ts:1729061856430037, flushed_ts:1729061856430714, gen_to_freeze cost time:3, gen_to_submit cost time:29, submit_to_flush cost time:677}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061856430716, last_flushed_end_lsn:{lsn:66390102574}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061856257833, last_flushed_end_lsn:{lsn:66390102452}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061856256351, last_flushed_end_lsn:{lsn:66390102452}}]) [2024-10-16 14:57:36.432616] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2749edf0, session_id:3221649181, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:236}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:793, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432638] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=16] kill and revert session(conn.sessid_=3221649181, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.432644] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f03c10df0, session_id:3221568553, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:241}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432673] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=27] kill and revert session(conn.sessid_=3221568553, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.432677] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f03c10030, session_id:3221735918, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:240}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:738, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432684] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221735918, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.432687] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f03c0edf0, session_id:3221734926, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:238}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:534, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432694] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221734926, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.432697] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f03c0e030, session_id:3221739050, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:237}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432703] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] kill and revert session(conn.sessid_=3221739050, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.432706] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2749e030, session_id:3221726815, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:235}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:610, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432713] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] kill and revert session(conn.sessid_=3221726815, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.432716] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2749cdf0, session_id:3221736398, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:232}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:284, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432722] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] kill and revert session(conn.sessid_=3221736398, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.432725] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0f2749edf0, session_id:3221649181, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:236}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:793, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432738] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221649181, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.432743] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221649181, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E99793CF611-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:688, rec_:688, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:370, seq_:120}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:121}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.432778] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=34] sock ref clean, do destroy(*s={this:0x2b0f03c10df0, session_id:3221568553, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:241}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432788] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221568553, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.432791] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221568553, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA18D5EAF-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:452, rec_:452, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.432804] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0f03c10030, session_id:3221735918, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:240}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:738, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432806] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221649181) [2024-10-16 14:57:36.432814] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221735918, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.432814] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=8] mark session id unused(sessid=3221649181) [2024-10-16 14:57:36.432816] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221735918, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA18D5EAE-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:633, rec_:633, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.432829] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0f03c0edf0, session_id:3221734926, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:238}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:534, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432838] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221734926, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.432846] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221568553) [2024-10-16 14:57:36.432849] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221568553) [2024-10-16 14:57:36.432841] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] connection close(sessid=3221734926, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA18D5EAD-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:429, rec_:429, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.432853] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0f03c0e030, session_id:3221739050, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:237}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432862] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=4] unlock session of tenant(conn.sessid_=3221739050, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.432864] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221739050, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA18D5EAC-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:452, rec_:452, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.432876] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221735918) [2024-10-16 14:57:36.432878] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221735918) [2024-10-16 14:57:36.432876] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=11] sock ref clean, do destroy(*s={this:0x2b0f2749e030, session_id:3221726815, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:235}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:610, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432885] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221726815, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.432887] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221726815, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA18D5EAB-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:505, rec_:505, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:7, seq_:9}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:13}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:14}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.432899] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221734926) [2024-10-16 14:57:36.432902] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221734926) [2024-10-16 14:57:36.432899] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0f2749cdf0, session_id:3221736398, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:232}, err:5, last_decode_time:1729061856400719, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:284, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.432909] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221736398, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.432911] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221736398, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA18D5EAA-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:179, rec_:179, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:5, seq_:7}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:8}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:13}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:14}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.432928] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221739050) [2024-10-16 14:57:36.432931] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221739050) [2024-10-16 14:57:36.432953] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221736398) [2024-10-16 14:57:36.432955] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221736398) [2024-10-16 14:57:36.432967] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=11] free session successfully(sessid=3221726815) [2024-10-16 14:57:36.432973] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221726815) [2024-10-16 14:57:36.433021] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FED-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:36.433028] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FED-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1058, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:36.433063] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E35-0-0] [lt=3] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10008, locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433051, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433051, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433051, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.433081] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E35-0-0] [lt=17] success to process renew task(task={tenant_id:1, table_id:10008, add_timestamp:1729061856405647}, locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433051, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433051, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433051, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.433103] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E36-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12142, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433100, sql_renew_time:1729061856433100, is_mark_fail:false}]) [2024-10-16 14:57:36.433109] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E36-0-0] [lt=6] success to process renew task(task={tenant_id:1, table_id:12142, add_timestamp:1729061856416098}, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433100, sql_renew_time:1729061856433100, is_mark_fail:false}]) [2024-10-16 14:57:36.433118] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E37-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12068, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433116, sql_renew_time:1729061856433116, is_mark_fail:false}]) [2024-10-16 14:57:36.433124] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E37-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12068, add_timestamp:1729061856416116}, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433116, sql_renew_time:1729061856433116, is_mark_fail:false}]) [2024-10-16 14:57:36.433133] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E38-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12084, locations=[{table_id:12084, partition_id:12084, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433130, sql_renew_time:1729061856433130, is_mark_fail:false}]) [2024-10-16 14:57:36.433138] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E38-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12084, add_timestamp:1729061856416123}, locations=[{table_id:12084, partition_id:12084, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433130, sql_renew_time:1729061856433130, is_mark_fail:false}]) [2024-10-16 14:57:36.433145] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E39-0-0] [lt=5] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11008, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:36.434638] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=16][errcode=-4008] query response time size is 0(MTL_ID()=1, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.434651] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21658][T1_L0_G0][T1001][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=11][errcode=-4008] query response time size is 0(MTL_ID()=1001, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.434655] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21658][T1_L0_G0][T1002][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=4][errcode=-4008] query response time size is 0(MTL_ID()=1002, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.434659] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21658][T1_L0_G0][T1003][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=3][errcode=-4008] query response time size is 0(MTL_ID()=1003, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.434662] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21658][T1_L0_G0][T1004][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=3][errcode=-4008] query response time size is 0(MTL_ID()=1004, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.434665] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:105) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=3][errcode=-4008] fail to execute(ret=-4008) [2024-10-16 14:57:36.436764] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436776] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=11][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436792] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436795] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436803] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436806] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436811] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436814] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436819] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436822] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436826] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436829] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436834] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436837] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436842] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436844] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436849] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436852] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436857] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436859] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436864] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436867] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436872] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436874] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436879] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436881] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436886] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436889] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436894] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436896] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436901] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436904] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436909] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436911] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436916] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436919] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436924] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436926] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436931] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436934] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436938] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436941] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436946] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436948] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436953] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436956] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436961] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436963] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436968] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436971] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436976] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436978] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436983] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436986] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436991] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.436993] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.436998] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437000] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437008] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437010] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437016] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437018] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437023] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437026] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437030] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437041] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437046] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437048] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437053] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437055] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437060] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437063] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437068] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437071] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437075] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437078] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437083] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437085] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437090] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437093] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437098] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437100] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437105] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437107] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437113] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437115] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437120] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437122] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437127] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437130] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437135] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437137] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437142] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437145] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437150] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437152] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437157] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437160] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437165] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437167] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437172] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437174] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437179] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437182] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437186] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437189] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437194] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437197] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437202] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437204] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437209] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437211] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437216] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437219] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437224] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437226] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437231] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437234] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437239] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437241] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437246] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437249] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437253] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437256] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437261] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437263] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437268] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437271] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437276] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437278] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437283] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437285] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437290] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437293] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437298] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437300] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2024-10-16 14:57:36.437305] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11608) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2024-10-16 14:57:36.437307] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11621) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED795-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:36.439973] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:36.440005] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=27] kill and revert session(conn.sessid_=3221657713, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.440012] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=1] kill and revert session(conn.sessid_=3221739142, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.440016] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0e5cdb8df0, session_id:3221657713, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:229}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:285, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.440028] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221657713, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.440033] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221657713, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B304F3D71-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:180, rec_:180, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:2, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:6}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.440055] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=21] sock ref clean, do destroy(*s={this:0x2b0e5cdfcdf0, session_id:3221739142, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:230}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:466, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.440068] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221739142, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.440071] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221739142, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA26D4E9E-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:361, rec_:361, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:7, seq_:9}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.440136] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221657713) [2024-10-16 14:57:36.440143] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=6] mark session id unused(sessid=3221657713) [2024-10-16 14:57:36.440166] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221739142) [2024-10-16 14:57:36.440169] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221739142) [2024-10-16 14:57:36.440195] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:36.440208] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=11] kill and revert session(conn.sessid_=3221737567, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.440213] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=0] kill and revert session(conn.sessid_=3221737879, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.440218] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=0] kill and revert session(conn.sessid_=3221728400, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.440221] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0e5cdf8df0, session_id:3221737567, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:227}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:654, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.440231] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221737567, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.440234] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221737567, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B304F3D72-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:549, rec_:549, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:25, seq_:32}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:33}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.440250] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=14] sock ref clean, do destroy(*s={this:0x2b0e5cdb6df0, session_id:3221737879, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:225}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:346, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.440262] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221737879, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.440265] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2] connection close(sessid=3221737879, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B2D49CF9A-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:241, rec_:241, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.440277] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0e5cdcc030, session_id:3221728400, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:224}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:239, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.440287] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221728400, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.440290] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2] connection close(sessid=3221728400, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B2D49CF99-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:134, rec_:134, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:7}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:8}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.440331] INFO [SERVER] run (obmp_disconnect.cpp:82) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=14] free session successfully(sessid=3221737567) [2024-10-16 14:57:36.440335] INFO [SERVER] run (obmp_disconnect.cpp:89) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] mark session id unused(sessid=3221737567) [2024-10-16 14:57:36.440357] INFO [SERVER] run (obmp_disconnect.cpp:82) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221737879) [2024-10-16 14:57:36.440359] INFO [SERVER] run (obmp_disconnect.cpp:89) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221737879) [2024-10-16 14:57:36.440385] INFO [SERVER] run (obmp_disconnect.cpp:82) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221728400) [2024-10-16 14:57:36.440387] INFO [SERVER] run (obmp_disconnect.cpp:89) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221728400) [2024-10-16 14:57:36.440425] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:36.440438] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=10] kill and revert session(conn.sessid_=3221739053, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.440443] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=0] kill and revert session(conn.sessid_=3221649591, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.440447] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=0] kill and revert session(conn.sessid_=3221737422, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.440451] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0f2749c030, session_id:3221739053, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:221}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:667, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.440464] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9] unlock session of tenant(conn.sessid_=3221739053, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.440467] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221739053, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA26D4E9D-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:562, rec_:562, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:10}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:11}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.440482] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=14] sock ref clean, do destroy(*s={this:0x2b0f2742cdf0, session_id:3221649591, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:218}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:203, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.440491] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221649591, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.440494] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] connection close(sessid=3221649591, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA26D4E9C-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:98, rec_:98, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.440506] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0f27414df0, session_id:3221737422, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:214}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:231, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.440516] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221737422, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.440518] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221737422, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B196AC91F-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:126, rec_:126, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:64, seq_:70}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:71}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.440572] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221739053) [2024-10-16 14:57:36.440576] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221739053) [2024-10-16 14:57:36.440622] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221649591) [2024-10-16 14:57:36.440624] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221649591) [2024-10-16 14:57:36.440647] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221737422) [2024-10-16 14:57:36.440649] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221737422) [2024-10-16 14:57:36.444524] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=6][errcode=-4008] query response time size is 0(MTL_ID()=1, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.444539] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21654][T1_L0_G0][T1001][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=14][errcode=-4008] query response time size is 0(MTL_ID()=1001, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.444544] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21654][T1_L0_G0][T1002][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=4][errcode=-4008] query response time size is 0(MTL_ID()=1002, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.444547] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21654][T1_L0_G0][T1003][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=3][errcode=-4008] query response time size is 0(MTL_ID()=1003, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.444550] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21654][T1_L0_G0][T1004][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=2][errcode=-4008] query response time size is 0(MTL_ID()=1004, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:36.444565] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:105) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85C5-0-0] [lt=2][errcode=-4008] fail to execute(ret=-4008) [2024-10-16 14:57:36.444860] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E39-0-0] [lt=4] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11008, locations=[{table_id:11008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444846, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444846, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444846, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.444881] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E39-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:11008, add_timestamp:1729061856417082}, locations=[{table_id:11008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444846, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444846, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444846, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.444909] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3A-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12217, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444907, sql_renew_time:1729061856444907, is_mark_fail:false}]) [2024-10-16 14:57:36.444915] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3A-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12217, add_timestamp:1729061856419467}, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444907, sql_renew_time:1729061856444907, is_mark_fail:false}]) [2024-10-16 14:57:36.444925] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3B-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11035, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444923, sql_renew_time:1729061856444923, is_mark_fail:false}]) [2024-10-16 14:57:36.444941] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3B-0-0] [lt=16] success to process renew task(task={tenant_id:1, table_id:11035, add_timestamp:1729061856419494}, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856444923, sql_renew_time:1729061856444923, is_mark_fail:false}]) [2024-10-16 14:57:36.444948] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3C-0-0] [lt=6] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11003, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:36.445047] INFO [COMMON] generate_mod_stat_task (memory_dump.cpp:218) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] task info(*task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:36.445055] INFO [COMMON] handle (memory_dump.cpp:550) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=8] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:36.445899] INFO [COMMON] update_check_range (ob_sql_mem_leak_checker.cpp:62) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] update_check_range(min_check_version=652898, max_check_version=652899, global_version=652900) [2024-10-16 14:57:36.456450] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061856456295000, start_id=1729061856456295000, end_id=1729061856456295001) [2024-10-16 14:57:36.456540] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.457277] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3C-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11003, locations=[{table_id:11003, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856457264, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856457264, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856457264, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.457317] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3C-0-0] [lt=39] success to process renew task(task={tenant_id:1, table_id:11003, add_timestamp:1729061856419730}, locations=[{table_id:11003, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856457264, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856457264, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856457264, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.457337] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3D-0-0] [lt=13] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11037, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:36.457449] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE770B-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:36.458072] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3D-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11037, locations=[{table_id:11037, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856458060, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856458060, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856458060, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.458090] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3D-0-0] [lt=17] success to process renew task(task={tenant_id:1, table_id:11037, add_timestamp:1729061856421959}, locations=[{table_id:11037, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856458060, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856458060, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856458060, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.458108] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3E-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12325, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:36.458353] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=1] kill and revert session(conn.sessid_=3221736972, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.458362] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=9] sock ref clean, do destroy(*s={this:0x2b0e2917e030, session_id:3221736972, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:223}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:1099, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.458379] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=10] unlock session of tenant(conn.sessid_=3221736972, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.458384] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221736972, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B34FA85C5-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:5, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:994, rec_:994, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:5, seq_:7}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.458447] INFO [SERVER] run (obmp_disconnect.cpp:82) [21680][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=11] free session successfully(sessid=3221736972) [2024-10-16 14:57:36.458452] INFO [SERVER] run (obmp_disconnect.cpp:89) [21680][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221736972) [2024-10-16 14:57:36.467369] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7A-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:36.467382] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7A-0-0] [lt=11] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:36.470411] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=39] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:36.471016] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=10] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:36.471043] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:36.471060] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061848719447000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:36.471703] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=5] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:57:36.471715] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=10] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:36.471721] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:36.471726] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:36.473778] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:4, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:57:36.473905] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=126] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:36.474484] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3E-0-0] [lt=4] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12325, locations=[{table_id:12325, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856474470, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856474470, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856474470, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.474505] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3E-0-0] [lt=21] success to process renew task(task={tenant_id:1, table_id:12325, add_timestamp:1729061856422205}, locations=[{table_id:12325, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856474470, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856474470, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856474470, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.474526] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3F-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:36.475750] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=0] kill and revert session(conn.sessid_=3221735410, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:36.475789] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=22] sock ref clean, do destroy(*s={this:0x2b0f27406030, session_id:3221735410, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061856415145, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:228, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:36.475804] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] unlock session of tenant(conn.sessid_=3221735410, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:36.475809] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221735410, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED795-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:123, rec_:123, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:1, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:36.475841] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:36.475930] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3F-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11017, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856475919, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856475919, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856475919, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.475949] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E3F-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:11017, add_timestamp:1729061856422920}, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856475919, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856475919, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856475919, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:36.477178] INFO [SERVER] run (obmp_disconnect.cpp:82) [21653][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=10] free session successfully(sessid=3221735410) [2024-10-16 14:57:36.477187] INFO [SERVER] run (obmp_disconnect.cpp:89) [21653][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=9] mark session id unused(sessid=3221735410) [2024-10-16 14:57:36.489842] INFO handle (memory_dump.cpp:677) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 203 265 0 1 2 117 1 0 1 4 1 1 0 1 5 38 4 0 1 6 2 7 0 1 7 1 2 0 1 8 65 1 0 1 12 18 3 0 1 13 6 1 0 1 14 1 2 0 1 15 1 1 0 1 16 2 2 0 1 18 26 3 0 1 20 2 2 0 1 22 1 1 0 500 0 139 255 0 500 7 7 4 0 500 8 50 1 0 500 9 2 1 0 500 10 10 2 0 500 16 3 2 0 500 17 10 7 0 500 22 112 63 0 500 23 61 11 0 508 0 4 13 0 508 8 5 1 0 509 0 3 8 0 509 8 5 1 0 1001 0 129 232 0 1001 2 21 1 0 1001 4 1 1 0 1001 5 23 4 0 1001 8 44 1 0 1001 12 10 3 0 1001 13 3 1 0 1001 14 1 2 0 1001 18 14 3 0 1001 20 1 2 0 1002 0 173 258 0 1002 2 21 1 0 1002 4 2 1 0 1002 5 24 4 0 1002 8 50 1 0 1002 12 19 3 0 1002 13 7 1 0 1002 14 1 2 0 1002 18 14 3 0 1002 20 7 2 0 1003 0 141 237 0 1003 2 29 1 0 1003 4 1 1 0 1003 5 23 4 0 1003 8 45 1 0 1003 12 10 3 0 1003 13 3 1 0 1003 14 1 2 0 1003 18 13 3 0 1003 20 1 2 0 1004 0 291 258 0 1004 2 19 1 0 1004 4 2 1 0 1004 5 34 4 0 1004 8 52 1 0 1004 12 19 3 0 1004 13 10 1 0 1004 14 2 2 0 1004 15 2 1 0 1004 16 4 4 0 1004 18 25 3 0 1004 20 11 2 0 cost_time: 43972 [2024-10-16 14:57:36.489881] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=10] [MEMORY] tenant: 1, limit: 2,147,483,648 hold: 1,982,304,256 rpc_hold: 0 cache_hold: 861,929,472 cache_used: 861,929,472 cache_item_count: 411 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 537,346,048 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MEMSTORE_CTX_ID hold_bytes= 245,366,784 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TRANS_CTX_MGR_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PLAN_CACHE_CTX_ID hold_bytes= 79,691,776 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= WORK_AREA hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 136,314,880 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= META_OBJ_CTX_ID hold_bytes= 37,748,736 limit= 429,496,720 [MEMORY] ctx_id= TX_CALLBACK_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOB_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PS_CACHE_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TX_DATA_TABLE hold_bytes= 54,546,432 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MDS_DATA_ID hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:36.490132] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= DEFAULT_CTX_ID hold= 537,346,048 used= 425,993,312 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 31 [MEMORY] hold= 56,180,736 used= 56,153,088 count= 27 avg_used= 2,079,744 block_cnt= 27 chunk_cnt= 27 mod=CACHE_MAP_NODE [MEMORY] hold= 41,963,520 used= 41,943,040 count= 1 avg_used= 41,943,040 block_cnt= 1 chunk_cnt= 1 mod=LogGroupBuffer [MEMORY] hold= 30,965,760 used= 27,525,120 count= 420 avg_used= 65,536 block_cnt= 420 chunk_cnt= 50 mod=MinorMergeMgr [MEMORY] hold= 23,511,040 used= 23,433,792 count= 8 avg_used= 2,929,224 block_cnt= 8 chunk_cnt= 7 mod=IoControl [MEMORY] hold= 22,880,256 used= 21,568,040 count= 161 avg_used= 133,962 block_cnt= 161 chunk_cnt= 38 mod=SqlPlanMon [MEMORY] hold= 18,874,368 used= 16,967,424 count= 256 avg_used= 66,279 block_cnt= 256 chunk_cnt= 63 mod=SqlDtlBuf [MEMORY] hold= 17,694,720 used= 16,942,080 count= 240 avg_used= 70,592 block_cnt= 240 chunk_cnt= 30 mod=[T]ObSessionDIB [MEMORY] hold= 16,797,696 used= 16,777,336 count= 1 avg_used= 16,777,336 block_cnt= 1 chunk_cnt= 1 mod=MrgMemPoolChk [MEMORY] hold= 15,034,848 used= 14,886,881 count= 98 avg_used= 151,906 block_cnt= 31 chunk_cnt= 7 mod=OmtTenant [MEMORY] hold= 11,771,904 used= 9,838,592 count= 236 avg_used= 41,688 block_cnt= 236 chunk_cnt= 61 mod=[T]char [MEMORY] hold= 10,911,744 used= 9,699,328 count= 148 avg_used= 65,536 block_cnt= 148 chunk_cnt= 38 mod=MajorMergeMgr [MEMORY] hold= 8,777,728 used= 8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue [MEMORY] hold= 8,409,088 used= 8,388,640 count= 1 avg_used= 8,388,640 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 6,094,848 used= 6,033,408 count= 8 avg_used= 754,176 block_cnt= 8 chunk_cnt= 7 mod=ResourceGroup [MEMORY] hold= 4,980,736 used= 4,970,432 count= 4 avg_used= 1,242,608 block_cnt= 4 chunk_cnt= 4 mod=MysqlRequesReco [MEMORY] hold= 4,964,352 used= 4,959,488 count= 3 avg_used= 1,653,162 block_cnt= 3 chunk_cnt= 3 mod=SqlFltSpanRec [MEMORY] hold= 4,943,872 used= 4,915,456 count= 2 avg_used= 2,457,728 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe [MEMORY] hold= 4,235,264 used= 4,194,304 count= 2 avg_used= 2,097,152 block_cnt= 2 chunk_cnt= 2 mod=SHARE_BLOCK [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMem [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 12 mod=CkptDgnMemCU [MEMORY] hold= 3,772,416 used= 3,751,936 count= 1 avg_used= 3,751,936 block_cnt= 1 chunk_cnt= 1 mod=DiskIteratorSto [MEMORY] hold= 3,723,264 used= 3,698,688 count= 2 avg_used= 1,849,344 block_cnt= 2 chunk_cnt= 2 mod=LogDIOAligned [MEMORY] hold= 3,162,112 used= 3,145,808 count= 2 avg_used= 1,572,904 block_cnt= 2 chunk_cnt= 2 mod=TenantDLMgr [MEMORY] hold= 3,145,728 used= 2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 5 mod=SqlDtlQueue [MEMORY] hold= 2,828,928 used= 2,708,352 count= 604 avg_used= 4,484 block_cnt= 576 chunk_cnt= 72 mod=WorkerMap [MEMORY] hold= 2,777,088 used= 2,760,192 count= 1 avg_used= 2,760,192 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo [MEMORY] hold= 2,415,648 used= 2,387,720 count= 104 avg_used= 22,958 block_cnt= 104 chunk_cnt= 4 mod=TabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=FastFrezCkr [MEMORY] hold= 2,375,680 used= 2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta [MEMORY] hold= 2,248,704 used= 2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb [MEMORY] hold= 2,198,864 used= 416,700 count= 25,277 avg_used= 16 block_cnt= 270 chunk_cnt= 2 mod=Number [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=LogReplayTask [MEMORY] hold= 1,843,200 used= 1,770,696 count= 9 avg_used= 196,744 block_cnt= 9 chunk_cnt= 2 mod=DagMap [MEMORY] hold= 1,777,664 used= 1,770,696 count= 1 avg_used= 1,770,696 block_cnt= 1 chunk_cnt= 1 mod=TableProc [MEMORY] hold= 1,548,288 used= 1,488,096 count= 9 avg_used= 165,344 block_cnt= 9 chunk_cnt= 8 mod=SQLSessionInfo [MEMORY] hold= 1,548,288 used= 1,507,118 count= 5 avg_used= 301,423 block_cnt= 5 chunk_cnt= 1 mod=LDIOSetup [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=XATimeWheel [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=TransTimeWheel [MEMORY] hold= 1,261,568 used= 681,912 count= 77 avg_used= 8,856 block_cnt= 77 chunk_cnt= 31 mod=[T]ObDSActionAr [MEMORY] hold= 1,253,376 used= 1,122,952 count= 17 avg_used= 66,056 block_cnt= 17 chunk_cnt= 16 mod=CommSysVarFac [MEMORY] hold= 1,196,032 used= 1,182,720 count= 2 avg_used= 591,360 block_cnt= 2 chunk_cnt= 2 mod=ServerObjecPool [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsCache [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsInfo [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=RewriteRuleMap [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPlanCac [MEMORY] hold= 1,024,000 used= 1,015,808 count= 1 avg_used= 1,015,808 block_cnt= 1 chunk_cnt= 1 mod=CommitVersions [MEMORY] hold= 983,488 used= 960,256 count= 121 avg_used= 7,936 block_cnt= 121 chunk_cnt= 9 mod=TltMemtablMgr [MEMORY] hold= 958,464 used= 950,272 count= 1 avg_used= 950,272 block_cnt= 1 chunk_cnt= 1 mod=IOWorkerLQ [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=ArcSenderQueue [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=TransService [MEMORY] hold= 802,816 used= 394,352 count= 49 avg_used= 8,048 block_cnt= 49 chunk_cnt= 22 mod=[T]ObPerfEventR [MEMORY] hold= 798,400 used= 779,200 count= 100 avg_used= 7,792 block_cnt= 100 chunk_cnt= 20 mod=CkptDgnMemNode [MEMORY] hold= 798,112 used= 789,624 count= 3 avg_used= 263,208 block_cnt= 2 chunk_cnt= 2 mod=PartitTableTask [MEMORY] hold= 786,688 used= 524,352 count= 33 avg_used= 15,889 block_cnt= 33 chunk_cnt= 4 mod=LogAlloc [MEMORY] hold= 765,440 used= 730,112 count= 184 avg_used= 3,968 block_cnt= 184 chunk_cnt= 44 mod=CkpDgn [MEMORY] hold= 663,552 used= 655,360 count= 1 avg_used= 655,360 block_cnt= 1 chunk_cnt= 1 mod=FetchLog [MEMORY] hold= 663,552 used= 659,200 count= 1 avg_used= 659,200 block_cnt= 1 chunk_cnt= 1 mod=MulLevelQueue [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=ElectTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MdsT [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DupTbLease [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=FrzTrigger [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTR [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DetectorTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MultiVersionGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=OBJLockGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTF [MEMORY] hold= 598,016 used= 590,232 count= 1 avg_used= 590,232 block_cnt= 1 chunk_cnt= 1 mod=DagNetIdMap [MEMORY] hold= 507,904 used= 491,520 count= 2 avg_used= 245,760 block_cnt= 2 chunk_cnt= 2 mod=SqlExecutor [MEMORY] hold= 458,752 used= 440,928 count= 3 avg_used= 146,976 block_cnt= 3 chunk_cnt= 3 mod=bf_queue [MEMORY] hold= 421,504 used= 404,416 count= 89 avg_used= 4,544 block_cnt= 89 chunk_cnt= 31 mod=[T]ObTraceEvent [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ApplySrv [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ReplaySrv [MEMORY] hold= 406,400 used= 396,800 count= 50 avg_used= 7,936 block_cnt= 50 chunk_cnt= 29 mod=LinearHashMapNo [MEMORY] hold= 399,200 used= 389,600 count= 50 avg_used= 7,792 block_cnt= 50 chunk_cnt= 32 mod=CkptDgnCUNode [MEMORY] hold= 385,024 used= 377,120 count= 1 avg_used= 377,120 block_cnt= 1 chunk_cnt= 1 mod=ClogGe [MEMORY] hold= 338,112 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 44 chunk_cnt= 2 mod=LSMap [MEMORY] hold= 311,296 used= 262,144 count= 256 avg_used= 1,024 block_cnt= 44 chunk_cnt= 1 mod=TabletToLS [MEMORY] hold= 311,104 used= 303,904 count= 6 avg_used= 50,650 block_cnt= 6 chunk_cnt= 2 mod=PoolFreeList [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompactMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=DmlStatHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompStMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 294,912 used= 261,632 count= 4 avg_used= 65,408 block_cnt= 4 chunk_cnt= 3 mod=MemTblObj [MEMORY] hold= 276,352 used= 269,824 count= 34 avg_used= 7,936 block_cnt= 34 chunk_cnt= 27 mod=SqlSession [MEMORY] hold= 268,224 used= 261,888 count= 33 avg_used= 7,936 block_cnt= 33 chunk_cnt= 18 mod=CtxBlockPl [MEMORY] hold= 260,224 used= 253,952 count= 31 avg_used= 8,192 block_cnt= 31 chunk_cnt= 22 mod=ParamCharsType [MEMORY] hold= 212,992 used= 212,160 count= 1 avg_used= 212,160 block_cnt= 1 chunk_cnt= 1 mod=MetaMemMgr [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 19 mod=QueryAllocator [MEMORY] hold= 204,800 used= 196,744 count= 1 avg_used= 196,744 block_cnt= 1 chunk_cnt= 1 mod=DagNetMap [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=TenDLBucket [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=ResourMapLock [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=T3MBucket [MEMORY] hold= 163,776 used= 156,741 count= 2 avg_used= 78,370 block_cnt= 2 chunk_cnt= 2 mod=CsSstableReader [MEMORY] hold= 147,456 used= 139,264 count= 1 avg_used= 139,264 block_cnt= 1 chunk_cnt= 1 mod=RFLTaskQueue [MEMORY] hold= 147,456 used= 138,240 count= 2 avg_used= 69,120 block_cnt= 2 chunk_cnt= 1 mod=LDBlockBitMap [MEMORY] hold= 147,456 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 2 mod=ResultSet [MEMORY] hold= 146,304 used= 142,848 count= 18 avg_used= 7,936 block_cnt= 18 chunk_cnt= 15 mod=SqlSessiVarMap [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 15 mod=PlanVaIdx [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 11 mod=RawPl [MEMORY] hold= 128,576 used= 33,168 count= 471 avg_used= 70 block_cnt= 175 chunk_cnt= 33 mod=Coro [MEMORY] hold= 128,000 used= 124,928 count= 16 avg_used= 7,808 block_cnt= 16 chunk_cnt= 2 mod=TabletIDSetNode [MEMORY] hold= 122,880 used= 110,608 count= 2 avg_used= 55,304 block_cnt= 2 chunk_cnt= 1 mod=TabletIDSetBkt [MEMORY] hold= 114,688 used= 111,096 count= 1 avg_used= 111,096 block_cnt= 1 chunk_cnt= 1 mod=AsyncFreezeTabl [MEMORY] hold= 114,688 used= 106,496 count= 1 avg_used= 106,496 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlMgr [MEMORY] hold= 106,496 used= 98,528 count= 1 avg_used= 98,528 block_cnt= 1 chunk_cnt= 1 mod=LSSvr [MEMORY] hold= 106,496 used= 98,432 count= 1 avg_used= 98,432 block_cnt= 1 chunk_cnt= 1 mod=tenant_mf_mgr [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=DictPl [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=RlePl [MEMORY] hold= 81,920 used= 73,896 count= 1 avg_used= 73,896 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairCache [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 10 mod=ConstPl [MEMORY] hold= 77,824 used= 65,536 count= 64 avg_used= 1,024 block_cnt= 12 chunk_cnt= 3 mod=TxCtxMgr [MEMORY] hold= 73,728 used= 65,536 count= 1 avg_used= 65,536 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushMeta [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushLog [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=DEVICE_MANAGER [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=StorMetaCacheIO [MEMORY] hold= 73,728 used= 66,176 count= 1 avg_used= 66,176 block_cnt= 1 chunk_cnt= 1 mod=Rpc [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxDataMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxCtxMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LockMemObj [MEMORY] hold= 73,728 used= 72,736 count= 1 avg_used= 72,736 block_cnt= 1 chunk_cnt= 1 mod=LogSharedQueueT [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 9 mod=IntDiffPl [MEMORY] hold= 72,240 used= 1,024 count= 896 avg_used= 1 block_cnt= 10 chunk_cnt= 1 mod=CharsetUtil [MEMORY] hold= 65,216 used= 60,712 count= 4 avg_used= 15,178 block_cnt= 4 chunk_cnt= 4 mod=HB_SERVICE [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 8 mod=MdsTableMgr [MEMORY] hold= 64,000 used= 62,464 count= 8 avg_used= 7,808 block_cnt= 8 chunk_cnt= 8 mod=HashNodeLCSta [MEMORY] hold= 58,400 used= 55,296 count= 16 avg_used= 3,456 block_cnt= 11 chunk_cnt= 8 mod=CACHE_INST [MEMORY] hold= 57,344 used= 50,496 count= 1 avg_used= 50,496 block_cnt= 1 chunk_cnt= 1 mod=palf_env [MEMORY] hold= 54,000 used= 5,760 count= 240 avg_used= 24 block_cnt= 146 chunk_cnt= 27 mod=[T]MemoryContex [MEMORY] hold= 48,960 used= 44,840 count= 2 avg_used= 22,420 block_cnt= 2 chunk_cnt= 2 mod=DynamicFactor [MEMORY] hold= 48,880 used= 44,760 count= 2 avg_used= 22,380 block_cnt= 2 chunk_cnt= 2 mod=Autoincrement [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 4 mod=HexStrPl [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 5 mod=ColEqualPl [MEMORY] hold= 45,792 used= 35,616 count= 53 avg_used= 672 block_cnt= 51 chunk_cnt= 23 mod=[T]ObWarningBuf [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DagWarnHisBkt [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=StrDiffPl [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 1 mod=ObDMMDL [MEMORY] hold= 34,816 used= 31,744 count= 16 avg_used= 1,984 block_cnt= 2 chunk_cnt= 2 mod=CreateContext [MEMORY] hold= 32,912 used= 22,464 count= 54 avg_used= 416 block_cnt= 23 chunk_cnt= 7 mod=OMT_Worker [MEMORY] hold= 32,768 used= 17,408 count= 2 avg_used= 8,704 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTTP [MEMORY] hold= 32,768 used= 25,664 count= 1 avg_used= 25,664 block_cnt= 1 chunk_cnt= 1 mod=TSQLSessionMgr [MEMORY] hold= 32,768 used= 24,688 count= 2 avg_used= 12,344 block_cnt= 2 chunk_cnt= 2 mod=TLD_AssignedMgr [MEMORY] hold= 32,704 used= 26,079 count= 2 avg_used= 13,039 block_cnt= 2 chunk_cnt= 2 mod=CGReadInfoMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 3 mod=StrPrefixPl [MEMORY] hold= 32,000 used= 31,232 count= 4 avg_used= 7,808 block_cnt= 4 chunk_cnt= 4 mod=HashNodePlanCac [MEMORY] hold= 24,576 used= 22,592 count= 1 avg_used= 22,592 block_cnt= 1 chunk_cnt= 1 mod=IO_MGR [MEMORY] hold= 24,576 used= 16,384 count= 1 avg_used= 16,384 block_cnt= 1 chunk_cnt= 1 mod=SlogWriteBuffer [MEMORY] hold= 24,368 used= 20,136 count= 2 avg_used= 10,068 block_cnt= 2 chunk_cnt= 2 mod=TabletStats [MEMORY] hold= 24,336 used= 20,104 count= 2 avg_used= 10,052 block_cnt= 2 chunk_cnt= 2 mod=TLD_ResourceMgr [MEMORY] hold= 21,152 used= 16,896 count= 22 avg_used= 768 block_cnt= 15 chunk_cnt= 12 mod=TGTimer [MEMORY] hold= 17,056 used= 8,448 count= 3 avg_used= 2,816 block_cnt= 2 chunk_cnt= 2 mod=BaseLogWriter [MEMORY] hold= 16,384 used= 8,448 count= 1 avg_used= 8,448 block_cnt= 1 chunk_cnt= 1 mod=PalfEnv [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=SlogNopLog [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TabletSvrBucket [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_ClientTask [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MedCheckSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=DiaTabletMap [MEMORY] hold= 16,384 used= 11,584 count= 1 avg_used= 11,584 block_cnt= 1 chunk_cnt= 1 mod=LogApplyStatus [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_TableCtxMgr [MEMORY] hold= 16,384 used= 13,824 count= 1 avg_used= 13,824 block_cnt= 1 chunk_cnt= 1 mod=LogReplayStatus [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=TLDecoderCtxArr [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=OB_MICB_DECODER [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=ColSubStrPl [MEMORY] hold= 15,648 used= 15,264 count= 2 avg_used= 7,632 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoNode [MEMORY] hold= 12,560 used= 9,262 count= 17 avg_used= 544 block_cnt= 16 chunk_cnt= 14 mod=SqlSessiQuerSql [MEMORY] hold= 11,984 used= 11,592 count= 2 avg_used= 5,796 block_cnt= 2 chunk_cnt= 1 mod=StopWordSet [MEMORY] hold= 10,656 used= 7,208 count= 17 avg_used= 424 block_cnt= 16 chunk_cnt= 11 mod=PackStateMap [MEMORY] hold= 10,640 used= 7,208 count= 17 avg_used= 424 block_cnt= 16 chunk_cnt= 10 mod=SequenceIdMap [MEMORY] hold= 10,624 used= 7,208 count= 17 avg_used= 424 block_cnt= 16 chunk_cnt= 11 mod=SequenceMap [MEMORY] hold= 10,608 used= 7,208 count= 17 avg_used= 424 block_cnt= 16 chunk_cnt= 10 mod=ContextsMap [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=Scheduler [MEMORY] hold= 9,376 used= 8,984 count= 2 avg_used= 4,492 block_cnt= 2 chunk_cnt= 1 mod=FTPluginMap [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapDi [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapCn [MEMORY] hold= 8,192 used= 7,760 count= 2 avg_used= 3,880 block_cnt= 1 chunk_cnt= 1 mod=HashBucPsSessIn [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=PurgeThrottle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FetchLogTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompCkmItems [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSIter [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LockWaitMgr [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairs [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=APPLY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=REPLAY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LsElectRefInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MatchLsnMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MdsDebugMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=HandleSubmit [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ShareBlksMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=DASIDCache [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ls_id [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FTPluginHandle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=IORunners [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LCLSender [MEMORY] hold= 8,096 used= 7,904 count= 1 avg_used= 7,904 block_cnt= 1 chunk_cnt= 1 mod=SqlSessionSbloc [MEMORY] hold= 8,096 used= 7,904 count= 1 avg_used= 7,904 block_cnt= 1 chunk_cnt= 1 mod=PsSessionInfo [MEMORY] hold= 8,000 used= 7,808 count= 1 avg_used= 7,808 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsInfo [MEMORY] hold= 7,984 used= 7,792 count= 1 avg_used= 7,792 block_cnt= 1 chunk_cnt= 1 mod=ColUsagNode [MEMORY] hold= 7,920 used= 7,728 count= 1 avg_used= 7,728 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsCache [MEMORY] hold= 7,904 used= 7,712 count= 1 avg_used= 7,712 block_cnt= 1 chunk_cnt= 1 mod=DmlStatNode [MEMORY] hold= 7,824 used= 7,632 count= 1 avg_used= 7,632 block_cnt= 1 chunk_cnt= 1 mod=CheckMapNode [MEMORY] hold= 7,136 used= 66 count= 33 avg_used= 2 block_cnt= 31 chunk_cnt= 11 mod=[T]TSIUseWeak [MEMORY] hold= 6,800 used= 6,216 count= 3 avg_used= 2,072 block_cnt= 2 chunk_cnt= 2 mod=DetectManager [MEMORY] hold= 5,056 used= 4,656 count= 2 avg_used= 2,328 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=PxPoolBkt [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResGrpIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResRuleIdMap [MEMORY] hold= 4,288 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 4,160 used= 3,968 count= 1 avg_used= 3,968 block_cnt= 1 chunk_cnt= 1 mod=TinyAllocator [MEMORY] hold= 3,328 used= 1,792 count= 8 avg_used= 224 block_cnt= 5 chunk_cnt= 2 mod=LogIOTask [MEMORY] hold= 3,328 used= 256 count= 16 avg_used= 16 block_cnt= 16 chunk_cnt= 14 mod=[T]TSILastOper [MEMORY] hold= 3,200 used= 3,008 count= 1 avg_used= 3,008 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlDfc [MEMORY] hold= 2,944 used= 1,920 count= 5 avg_used= 384 block_cnt= 5 chunk_cnt= 3 mod=TableLockNode [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=MedCheckMap [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=IRMMemHashBuck [MEMORY] hold= 2,048 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=LogIOWS [MEMORY] hold= 2,000 used= 1,800 count= 1 avg_used= 1,800 block_cnt= 1 chunk_cnt= 1 mod=PxResMgr [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=MSTXCTX [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TXBFRetryId [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TabStatMgrLock [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=DUP_LS_SET [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GROUP_INDEX_MAP [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GCMemtableMap [MEMORY] hold= 1,344 used= 960 count= 2 avg_used= 480 block_cnt= 1 chunk_cnt= 1 mod=PoolArenaArray [MEMORY] hold= 1,280 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=ModuleInitCtx [MEMORY] hold= 1,248 used= 1,056 count= 1 avg_used= 1,056 block_cnt= 1 chunk_cnt= 1 mod=LOG_HASH_MAP [MEMORY] hold= 1,216 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=LockMemtableMgr [MEMORY] hold= 1,152 used= 960 count= 1 avg_used= 960 block_cnt= 1 chunk_cnt= 1 mod=FTParserMgr [MEMORY] hold= 1,120 used= 120 count= 5 avg_used= 24 block_cnt= 2 chunk_cnt= 1 mod=FreezeTask [MEMORY] hold= 976 used= 776 count= 1 avg_used= 776 block_cnt= 1 chunk_cnt= 1 mod=DepTabletSet [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=CheckPointTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TLD_TIMER [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=FlushTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=tenant_rm_mgr [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TabShellTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=DiskUsageTimer [MEMORY] hold= 848 used= 648 count= 1 avg_used= 648 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfo [MEMORY] hold= 752 used= 552 count= 1 avg_used= 552 block_cnt= 1 chunk_cnt= 1 mod=LSFreeze [MEMORY] hold= 672 used= 472 count= 1 avg_used= 472 block_cnt= 1 chunk_cnt= 1 mod=Election [MEMORY] hold= 672 used= 288 count= 2 avg_used= 144 block_cnt= 2 chunk_cnt= 2 mod=ObTxLSLogCb [MEMORY] hold= 576 used= 384 count= 1 avg_used= 384 block_cnt= 1 chunk_cnt= 1 mod=HAScheduler [MEMORY] hold= 544 used= 144 count= 2 avg_used= 72 block_cnt= 1 chunk_cnt= 1 mod=TntSrvObjPool [MEMORY] hold= 512 used= 120 count= 2 avg_used= 60 block_cnt= 2 chunk_cnt= 2 mod=UserResourceMgr [MEMORY] hold= 448 used= 64 count= 2 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfFSCbNode [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTHandler [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 2 mod=2dArray [MEMORY] hold= 400 used= 208 count= 1 avg_used= 208 block_cnt= 1 chunk_cnt= 1 mod=[T]ObMacroBlock [MEMORY] hold= 320 used= 102 count= 1 avg_used= 102 block_cnt= 1 chunk_cnt= 1 mod=TxLSLogBuf [MEMORY] hold= 304 used= 112 count= 1 avg_used= 112 block_cnt= 1 chunk_cnt= 1 mod=Coordinator [MEMORY] hold= 304 used= 104 count= 1 avg_used= 104 block_cnt= 1 chunk_cnt= 1 mod=RedoLogBuckLock [MEMORY] hold= 256 used= 64 count= 1 avg_used= 64 block_cnt= 1 chunk_cnt= 1 mod=NegParamIdex [MEMORY] hold= 256 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfoMap [MEMORY] hold= 256 used= 48 count= 1 avg_used= 48 block_cnt= 1 chunk_cnt= 1 mod=[T]TSITabletTab [MEMORY] hold= 240 used= 41 count= 1 avg_used= 41 block_cnt= 1 chunk_cnt= 1 mod=KeepAliveBuf [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfRCCbNode [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=RebuildCbNode [MEMORY] hold= 208 used= 16 count= 1 avg_used= 16 block_cnt= 1 chunk_cnt= 1 mod=logservice [MEMORY] hold= 425,993,312 used= 408,726,518 count= 33,146 avg_used= 12,331 mod=SUMMARY [2024-10-16 14:57:36.490198] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=58] [MEMORY] tenant_id= 1 ctx_id= MEMSTORE_CTX_ID hold= 245,366,784 used= 243,449,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 block_cnt= 117 chunk_cnt= 117 mod=Memstore [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 mod=SUMMARY [2024-10-16 14:57:36.490209] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TRANS_CTX_MGR_ID hold= 2,097,152 used= 729,088 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 block_cnt= 1 chunk_cnt= 1 mod=PartTranCtxMgr [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 mod=SUMMARY [2024-10-16 14:57:36.490219] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= PLAN_CACHE_CTX_ID hold= 79,691,776 used= 73,331,136 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 68,663,616 used= 62,143,518 count= 2,832 avg_used= 21,943 block_cnt= 2,832 chunk_cnt= 37 mod=SqlPhyPlan [MEMORY] hold= 3,414,336 used= 3,146,720 count= 326 avg_used= 9,652 block_cnt= 326 chunk_cnt= 37 mod=SqlPlanCache [MEMORY] hold= 1,251,200 used= 1,140,800 count= 575 avg_used= 1,984 block_cnt= 192 chunk_cnt= 37 mod=CreateContext [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 73,331,136 used= 66,432,638 count= 3,735 avg_used= 17,786 mod=SUMMARY [2024-10-16 14:57:36.490232] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= WORK_AREA hold= 2,097,152 used= 862,784 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 745,216 used= 658,927 count= 113 avg_used= 5,831 block_cnt= 36 chunk_cnt= 2 mod=ArenaHashJoin [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=SqlAggrFuncRow [MEMORY] hold= 42,112 used= 33,513 count= 3 avg_used= 11,171 block_cnt= 2 chunk_cnt= 1 mod=HashNodeGrouRow [MEMORY] hold= 624 used= 3 count= 3 avg_used= 1 block_cnt= 3 chunk_cnt= 1 mod=SqlMergeJoin [MEMORY] hold= 448 used= 64 count= 2 avg_used= 32 block_cnt= 2 chunk_cnt= 2 mod=HtOpAlloc [MEMORY] hold= 416 used= 8 count= 2 avg_used= 4 block_cnt= 2 chunk_cnt= 2 mod=HashBloomFilter [MEMORY] hold= 240 used= 44 count= 1 avg_used= 44 block_cnt= 1 chunk_cnt= 1 mod=SqlSortRow [MEMORY] hold= 862,784 used= 757,967 count= 125 avg_used= 6,063 mod=SUMMARY [2024-10-16 14:57:36.490242] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 1 ctx_id= GLIBC hold= 2,097,152 used= 81,040 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 80,944 used= 59,018 count= 263 avg_used= 224 block_cnt= 20 chunk_cnt= 1 mod=PlJit [MEMORY] hold= 96 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PlCodeGen [MEMORY] hold= 81,040 used= 59,050 count= 264 avg_used= 223 mod=SUMMARY [2024-10-16 14:57:36.490249] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= CO_STACK hold= 136,314,880 used= 133,668,864 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 block_cnt= 259 chunk_cnt= 65 mod=CoStack [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 mod=SUMMARY [2024-10-16 14:57:36.490262] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= KVSTORE_CACHE_ID hold= 861,929,472 used= 861,929,472 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 mod=SUMMARY [2024-10-16 14:57:36.490271] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= META_OBJ_CTX_ID hold= 37,748,736 used= 36,290,496 limit= 429,496,720 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 29,491,200 used= 26,236,400 count= 400 avg_used= 65,591 block_cnt= 400 chunk_cnt= 18 mod=L_TabletPool [MEMORY] hold= 6,709,248 used= 5,952,128 count= 91 avg_used= 65,408 block_cnt= 91 chunk_cnt= 15 mod=N_TabletPool [MEMORY] hold= 90,048 used= 84,768 count= 2 avg_used= 42,384 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 36,290,496 used= 32,273,296 count= 493 avg_used= 65,463 mod=SUMMARY [2024-10-16 14:57:36.490279] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TX_CALLBACK_CTX_ID hold= 12,582,912 used= 6,234,176 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 block_cnt= 767 chunk_cnt= 6 mod=MemtableCallbac [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:36.490287] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= LOB_CTX_ID hold= 2,097,152 used= 81,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LobAllocator [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 81,856 used= 73,344 count= 2 avg_used= 36,672 mod=SUMMARY [2024-10-16 14:57:36.490294] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= PS_CACHE_CTX_ID hold= 2,097,152 used= 16,288 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 block_cnt= 4 chunk_cnt= 1 mod=SqlPsCache [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 mod=SUMMARY [2024-10-16 14:57:36.490302] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= RPC_CTX_ID hold= 0 used= 49,152 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 24,576 used= 16,256 count= 1 avg_used= 16,256 block_cnt= 1 chunk_cnt= 1 mod=OB_LOG_COMMITTE [MEMORY] hold= 24,576 used= 16,256 count= 1 avg_used= 16,256 block_cnt= 1 chunk_cnt= 1 mod=OB_PX_FAST_INIT [MEMORY] hold= 49,152 used= 32,512 count= 2 avg_used= 16,256 mod=SUMMARY [2024-10-16 14:57:36.490314] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TX_DATA_TABLE hold= 54,546,432 used= 52,763,328 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 49,572,672 used= 48,401,664 count= 6,099 avg_used= 7,936 block_cnt= 6,099 chunk_cnt= 25 mod=TX_DATA_SLICE [MEMORY] hold= 3,182,528 used= 3,153,728 count= 3 avg_used= 1,051,242 block_cnt= 3 chunk_cnt= 3 mod=MEMTABLE_ARENA [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 52,763,328 used= 51,563,328 count= 6,103 avg_used= 8,448 mod=SUMMARY [2024-10-16 14:57:36.490325] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= MDS_DATA_ID hold= 4,194,304 used= 398,272 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 260,096 used= 253,952 count= 32 avg_used= 7,936 block_cnt= 32 chunk_cnt= 2 mod=MdsTable [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 2 mod=TX_OP [MEMORY] hold= 398,272 used= 388,864 count= 49 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:36.490339] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= SCHEMA_SERVICE hold= 2,097,152 used= 8,128 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=NonPartTenCac [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:36.490356] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,084,416,000 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 371,380,224 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 14,680,064 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 104,857,600 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 20,971,520 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= RPC_CTX_ID hold_bytes= 6,291,456 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 26,357,760 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 239,882,240 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= UNEXPECTED_IN_500 hold_bytes= 295,800,832 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:36.490565] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 371,380,224 used= 312,971,936 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 66,584,784 used= 66,551,824 count= 33 avg_used= 2,016,721 block_cnt= 33 chunk_cnt= 33 mod=Compressor [MEMORY] hold= 31,477,760 used= 31,458,304 count= 1 avg_used= 31,458,304 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 23,584,768 used= 23,499,200 count= 9 avg_used= 2,611,022 block_cnt= 9 chunk_cnt= 8 mod=IoControl [MEMORY] hold= 15,925,248 used= 15,247,872 count= 216 avg_used= 70,592 block_cnt= 216 chunk_cnt= 42 mod=[T]ObSessionDIB [MEMORY] hold= 12,873,728 used= 12,856,352 count= 1 avg_used= 12,856,352 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 9,457,664 used= 9,437,784 count= 1 avg_used= 9,437,784 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 5,505,024 used= 5,238,784 count= 32 avg_used= 163,712 block_cnt= 32 chunk_cnt= 22 mod=Iteratorget_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4906, mem_used=10398720) [2024-10-16 14:57:36.618622] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=33] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:36.631915] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061856631165000, start_id=1729061856631165000, end_id=1729061856631165001) [2024-10-16 14:57:36.639444] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0C25-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061856638564000, start_id=1729061856638564000, end_id=1729061856638564001) [2024-10-16 14:57:36.639552] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B998-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061856638564000, start_id=1729061856638564001, end_id=1729061856638564002) [2024-10-16 14:57:36.639553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0C23-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061856638564000, start_id=1729061856638564000, end_id=1729061856638564001) [2024-10-16 14:57:36.639581] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B99C-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061856638564000, start_id=1729061856638564000, end_id=1729061856638564001) [2024-10-16 14:57:36.655852] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] no need to update min/max version(new_version={val:1729061856054428001, v:0}, min_version={val:1729061856054428001, v:0}, max_version={val:1729061856054428001, v:0}) [2024-10-16 14:57:36.656642] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=24] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:36.656660] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:36.656667] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.657742] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:36.666791] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:36.670820] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BEB-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:36.696159] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:36.696243] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061856695581000, start_id=1729061856695581000, end_id=1729061856695581001) [2024-10-16 14:57:36.696960] INFO [STORAGE] runTimerTask (ob_locality_manager.cpp:708) [21260][LocalityReload][T0][Y0-0000000000000000-0-0] [lt=9] runTimer to refresh locality_info(ret=0) [2024-10-16 14:57:36.697948] INFO [SHARE] set_server_locality_array (ob_server_locality_cache.cpp:346) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=10] set_server_locality_array success(server_locality_array=[{inited:true, addr:"10.0.4.7:2890", zone:"myoceanbase1", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494256558000, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.8:2890", zone:"myoceanbase2", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494430723273, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.9:2890", zone:"myoceanbase3", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494593087951, server_stop_time:0, server_status:1}], has_readonly_zone=false) [2024-10-16 14:57:36.697986] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=29] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1, primary_zone=RANDOM) [2024-10-16 14:57:36.697993] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=6] add_locality_zone success(locality_zone={tenant_id:1, region_priority:18446744073709551615}) [2024-10-16 14:57:36.698002] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=5] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1001, primary_zone=RANDOM) [2024-10-16 14:57:36.698006] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=4] add_locality_zone success(locality_zone={tenant_id:1001, region_priority:18446744073709551615}) [2024-10-16 14:57:36.698022] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1002, primary_zone=RANDOM) [2024-10-16 14:57:36.698030] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=8] add_locality_zone success(locality_zone={tenant_id:1002, region_priority:18446744073709551615}) [2024-10-16 14:57:36.698034] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1003, primary_zone=RANDOM) [2024-10-16 14:57:36.698037] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1003, region_priority:18446744073709551615}) [2024-10-16 14:57:36.698040] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1004, primary_zone=RANDOM) [2024-10-16 14:57:36.698042] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1004, region_priority:18446744073709551615}) [2024-10-16 14:57:36.698046] INFO [SHARE] load_region (ob_locality_table_operator.cpp:373) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] load region(ret=0, locality_info={version:0, local_region:"sys_region", local_zone:"myoceanbase1", local_idc:"", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["myoceanbase1", "myoceanbase2", "myoceanbase3"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:18446744073709551615}, {tenant_id:1002, region_priority:18446744073709551615}, {tenant_id:1003, region_priority:18446744073709551615}, {tenant_id:1004, region_priority:18446744073709551615}]}) [2024-10-16 14:57:36.698399] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=7] gc stale ls task succ [2024-10-16 14:57:36.700666] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:57:36.700680] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] try gc retain ctx [2024-10-16 14:57:36.700683] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:36.701282] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061853638892005, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870098662000, latest_log_ts:{val:1729061853539570001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:36.701583] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1089) [2024-10-16 14:57:36.701989] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=46] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061854637437004, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:36.702640] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=35] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061854637437004, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:36.702692] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C424F-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:36.702703] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C424F-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=911, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:36.703237] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=28] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061854637437004, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:36.703984] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061856702990000, start_id=1729061856702990000, end_id=1729061856702990001) [2024-10-16 14:57:36.704492] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.723974] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:36.755759] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:36.755792] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=18] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:36.755806] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:36.756707] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.756800] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061856756345000, start_id=1729061856756345000, end_id=1729061856756345001) [2024-10-16 14:57:36.770913] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:36.774829] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:36.774845] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:36.781391] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=18] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:36.781893] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:36.783226] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D68-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:36.783250] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D68-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061856638564002, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061856638564002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:36.784264] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D68-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061856783612000, start_id=1729061856783612000, end_id=1729061856783612001) [2024-10-16 14:57:36.786464] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F41-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:36.786668] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D68-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061856785730000, start_id=1729061856785730000, end_id=1729061856785730001) [2024-10-16 14:57:36.786729] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D68-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:36.786743] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D68-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:36.788061] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.788174] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.788184] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.788200] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.788211] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.788218] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.788708] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.789169] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.789259] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.789279] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.789298] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:36.804538] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.805497] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061856804787000, start_id=1729061856804787000, end_id=1729061856804787001) [2024-10-16 14:57:36.812910] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC585-0-0] [lt=22] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:57:36.813627] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC585-0-0] [lt=15] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1002) [2024-10-16 14:57:36.813639] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC585-0-0] [lt=10] no transfer partition job [2024-10-16 14:57:36.813645] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC585-0-0] [lt=4] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:57:36.814896] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC585-0-0] [lt=6] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1002, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:57:36.814922] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC585-0-0] [lt=19] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:57:36.816073] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:500) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC585-0-0] [lt=15][errcode=-4152] get obj error(ret=-4152) [2024-10-16 14:57:36.816112] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC585-0-0] [lt=13] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1001, status:0}], ls_array_=[{tenant_id:1002, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729057174083520, last_statistic_bg_stat_time=1729008957751717, last_statistic_schema_version=1729008953019688, last_statistic_max_transfer_task_id={id:-1}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:57:36.824880] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1}, {id:1003}]) [2024-10-16 14:57:36.826358] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=16] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:36.831923] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1729061856831841000, start_id=1729061856831841000, end_id=1729061856831841001) [2024-10-16 14:57:36.832330] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061852819198, last_first_register_time_us=1722494157071428, register_time_us=1729061852819198, curr_time_us=1729061856832329) [2024-10-16 14:57:36.832360] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061856832346, last_submit_register_req_time_us=1729061856832346, register_parent_reason=3) [2024-10-16 14:57:36.832367] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=6] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:36.845730] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=32, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:36.845751] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:36.852501] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4188, mem_used=8318976) [2024-10-16 14:57:36.856770] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.861109] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22703][T1004_TntShared][T1004][Y0-0000000000000000-0-0] [lt=18] clear dtl interm result cost(us)(clear_cost=969, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:36.871491] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:36.872844] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90BF-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:36.872856] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90BF-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18189, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:36.872860] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90BF-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=1129, avg_log_submit_to_slide_time=18183) [2024-10-16 14:57:36.879660] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:36.890806] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=220, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:36.890822] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1187, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:36.890851] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:36.896278] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061856895316000, start_id=1729061856895316000, end_id=1729061856895316001) [2024-10-16 14:57:36.896366] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:36.900429] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:36.900442] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:57:36.900456] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=12] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:36.901046] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=14] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:36.901733] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=25] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1089, avg_group_log_size=121) [2024-10-16 14:57:36.901745] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1089) [2024-10-16 14:57:36.902558] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4255-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=267, avg_log_submit_to_slide_time=732) [2024-10-16 14:57:36.904050] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061856903077000, start_id=1729061856903077000, end_id=1729061856903077001) [2024-10-16 14:57:36.904596] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.915883] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBDA-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061856883663, keepalive_ts:-1}) [2024-10-16 14:57:36.915900] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBDA-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061856883663, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:36.920270] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:36.920295] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=212) [2024-10-16 14:57:36.921118] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DD8-0-0] [lt=7] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:57:36.921757] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:36.930756] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:36.931143] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FF1-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=377, avg_log_submit_to_slide_time=972) [2024-10-16 14:57:36.932302] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:36.932371] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061852921764, last_first_register_time_us=1722494150998907, register_time_us=1729061852921764, curr_time_us=1729061856932370) [2024-10-16 14:57:36.932399] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061856932384, last_submit_register_req_time_us=1729061856932384, register_parent_reason=3) [2024-10-16 14:57:36.932406] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:36.941626] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=2703, mem_used=6239232) [2024-10-16 14:57:36.950265] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=14] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:36.954482] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061846949291, end_time=1729061856954480, dag_status={}) [2024-10-16 14:57:36.954501] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=17] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:36.954508] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:36.954512] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:36.954515] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:36.954520] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:36.954526] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:36.954530] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:36.954533] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:36.954536] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:36.954539] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954549] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954553] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954560] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954564] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954571] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954574] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954582] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954586] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954589] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954596] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954606] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954612] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954616] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954620] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954624] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954628] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954631] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954635] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954639] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954643] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954647] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954651] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954654] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954658] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954662] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954672] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954676] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954679] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954683] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954687] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954691] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954694] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954698] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954702] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954705] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954709] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954713] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954718] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954722] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954726] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954729] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954733] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954740] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954744] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954750] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954754] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954761] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954765] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954772] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954775] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954779] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954786] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954790] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954797] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954801] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954808] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954812] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:36.954819] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:36.954823] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:36.954829] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:36.954832] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:36.954834] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:36.954837] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:36.954842] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:36.954845] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:36.954847] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:36.954852] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:36.954856] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:36.956019] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] no need to update min/max version(new_version={val:1729061856415145000, v:0}, min_version={val:1729061856415145000, v:0}, max_version={val:1729061856415145000, v:0}) [2024-10-16 14:57:36.956830] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:36.957667] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7715-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:36.958231] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBB-0-0] [lt=15] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:36.958252] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBB-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:36.967110] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15D6-0-0] [lt=3] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:36.971882] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.97, rt: 209.50; ) [2024-10-16 14:57:36.971910] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:36.971959] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.97, rt: 164.75; ) [2024-10-16 14:57:36.971972] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:36.971988] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:36.975259] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=271, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:36.975265] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:36.975276] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=330, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:36.975502] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD185-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=1759, avg_log_submit_to_slide_time=2339) [2024-10-16 14:57:36.982167] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:36.983981] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9989241701-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061856983632000, start_id=1729061856983632000, end_id=1729061856983632001) [2024-10-16 14:57:36.985371] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Cache wash clean map node details(ret=0, clean_node_count=43, clean_time=0, clean_start_pos=880796, clean_num=62914) [2024-10-16 14:57:36.986077] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F45-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:36.986099] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F45-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:36.995388] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061856995387, bytes: 4446811978544, bw: 0.514948 MB/s, add_ts: 1005848, add_bytes: 543120 [2024-10-16 14:57:36.995465] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061856995226000, start_id=1729061856995226000, end_id=1729061856995226001) [2024-10-16 14:57:36.995928] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f6521f180 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:192998632} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059893180432 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:2124}, prev=this=0x2b0f65219da0 trans_version={val:1729061256953030000, v:0} scn={val:1729061256953030000, v:0} tx_id={txid:192997736} prev=0x2b0f65219b00 next=(nil) modify_count=5 acc_checksum=2925942435 version=1729059893180432 type=1 flag=12 snapshot_barrier=4611686018427387803 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:4331}, snapshot_version={val:1729061856995226000, v:0}, *this={this=0x2b0f6520dc10 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=0 list_head=0x2b0f65219da0 latest_compact_node=0x2b0f65219da0 max_trans_version={val:1729061256953030000, v:0} max_trans_id=192997736 max_elr_trans_version={val:1729061256953030000, v:0} max_elr_trans_id=192997736 latest_compact_ts=1729061256953928 last_compact_cnt=5 total_trans_node_cnt=6 max_modify_scn={val:1729061256953030000, v:0} min_modify_scn={val:1729060056817495000, v:0}}) [2024-10-16 14:57:36.995964] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=35][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f6521f180 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:192998632} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059893180432 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:2124}, ctx={this:0x2b0e0b6d2080, ls_id:{id:1}, ls:0x2b0e076c4150, branch:0, timeout:1729061886895067, tablet_id:{id:444}, table_iter:0x2b0e0b6d1210, table_version:1717560275661600, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061886895067, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061856995226000, v:0}, tx_id:{txid:192998632}, scn:{branch:0, seq:2121}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e076d2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:192998632}, tx_desc:{this:0x2b0e0fd1a0f0, tx_id:{txid:192998632}, state:1, addr:"10.0.4.9:2890", tenant_id:1001, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061856995226, commit_ts:-1, finish_ts:-1, timeout_us:29999990, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061856993106, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0df705ba10, mem_ctx:0x2b0df705cf90, tx_scn:{branch:0, seq:2124}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:36.996024] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560275661600 max_table_version=1717560275661600 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:192998632} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:192998632}, data:0x2b0e0b6ce438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729059893180432, seq_no:{branch:0, seq:2124}, scn:{val:4611686018427387903, v:0}, column_cnt:7}) [2024-10-16 14:57:36.996042] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=16][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:36.996060] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:36.996068] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=444, col_descs=[cnt:7, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=7,cells=[{"BIGINT":1002},{"BIGINT UNSIGNED":1},{"VARCHAR":"10.0.4.9", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT UNSIGNED":2890},{"BIGINT UNSIGNED":1729061856962275},{"BIGINT UNSIGNED":1729061856983632000},{"BIGINT UNSIGNED":0},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:36.996126] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:36.996145] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=8][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f6521f300 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:192998632} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059893180432 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:2124}, prev=this=0x2b0f65219c50 trans_version={val:1729061256953030000, v:0} scn={val:1729061256953030000, v:0} tx_id={txid:192997736} prev=0x2b0f65219800 next=(nil) modify_count=5 acc_checksum=2317121052 version=1729059893180432 type=1 flag=12 snapshot_barrier=4611686018427387803 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:4331}, snapshot_version={val:1729061856995226000, v:0}, *this={this=0x2b0f6520ddb0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=0 list_head=0x2b0f65219c50 latest_compact_node=0x2b0f65219c50 max_trans_version={val:1729061256953030000, v:0} max_trans_id=192997736 max_elr_trans_version={val:1729061256953030000, v:0} max_elr_trans_id=192997736 latest_compact_ts=1729061256953925 last_compact_cnt=5 total_trans_node_cnt=6 max_modify_scn={val:1729061256953030000, v:0} min_modify_scn={val:1729060056817495000, v:0}}) [2024-10-16 14:57:36.996162] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=16][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f6521f300 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:192998632} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059893180432 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:2124}, ctx={this:0x2b0e0b6d2080, ls_id:{id:1}, ls:0x2b0e076c4150, branch:0, timeout:1729061886895067, tablet_id:{id:444}, table_iter:0x2b0e0b6d1210, table_version:1717560275661600, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061886895067, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061856995226000, v:0}, tx_id:{txid:192998632}, scn:{branch:0, seq:2121}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e076d2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:192998632}, tx_desc:{this:0x2b0e0fd1a0f0, tx_id:{txid:192998632}, state:1, addr:"10.0.4.9:2890", tenant_id:1001, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061856995226, commit_ts:-1, finish_ts:-1, timeout_us:29999990, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061856993106, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0df705ba10, mem_ctx:0x2b0df705cf90, tx_scn:{branch:0, seq:2124}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:36.996200] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560275661600 max_table_version=1717560275661600 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:192998632} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:192998632}, data:0x2b0e0b6ce438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729059893180432, seq_no:{branch:0, seq:2124}, scn:{val:4611686018427387903, v:0}, column_cnt:7}) [2024-10-16 14:57:36.996213] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=0][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:36.996217] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=444, col_descs=[cnt:7, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=7,cells=[{"BIGINT":1002},{"BIGINT UNSIGNED":2},{"VARCHAR":"10.0.4.9", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT UNSIGNED":2890},{"BIGINT UNSIGNED":1729061856962275},{"BIGINT UNSIGNED":1729061856138655005},{"BIGINT UNSIGNED":0},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:36.996245] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=19][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:36.996258] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=5][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f6521f480 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:192998632} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059893180432 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:2124}, prev=this=0x2b0f65219cf8 trans_version={val:1729061256953030000, v:0} scn={val:1729061256953030000, v:0} tx_id={txid:192997736} prev=0x2b0f65219980 next=(nil) modify_count=5 acc_checksum=1207149433 version=1729059893180432 type=1 flag=12 snapshot_barrier=4611686018427387803 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:4331}, snapshot_version={val:1729061856995226000, v:0}, *this={this=0x2b0f6520df50 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=0 list_head=0x2b0f65219cf8 latest_compact_node=0x2b0f65219cf8 max_trans_version={val:1729061256953030000, v:0} max_trans_id=192997736 max_elr_trans_version={val:1729061256953030000, v:0} max_elr_trans_id=192997736 latest_compact_ts=1729061256953926 last_compact_cnt=5 total_trans_node_cnt=6 max_modify_scn={val:1729061256953030000, v:0} min_modify_scn={val:1729060056817495000, v:0}}) [2024-10-16 14:57:36.996274] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=16][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f6521f480 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:192998632} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059893180432 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:2124}, ctx={this:0x2b0e0b6d2080, ls_id:{id:1}, ls:0x2b0e076c4150, branch:0, timeout:1729061886895067, tablet_id:{id:444}, table_iter:0x2b0e0b6d1210, table_version:1717560275661600, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061886895067, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061856995226000, v:0}, tx_id:{txid:192998632}, scn:{branch:0, seq:2121}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e076d2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:192998632}, tx_desc:{this:0x2b0e0fd1a0f0, tx_id:{txid:192998632}, state:1, addr:"10.0.4.9:2890", tenant_id:1001, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061856995226, commit_ts:-1, finish_ts:-1, timeout_us:29999990, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061856993106, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0df705ba10, mem_ctx:0x2b0df705cf90, tx_scn:{branch:0, seq:2124}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:36.996305] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560275661600 max_table_version=1717560275661600 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:192998632} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:192998632}, data:0x2b0e0b6ce438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729059893180432, seq_no:{branch:0, seq:2124}, scn:{val:4611686018427387903, v:0}, column_cnt:7}) [2024-10-16 14:57:36.996317] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=1][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:36.996320] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=2][errcode=-5024] failed to write table row(ret=-5024, table id=444, col_descs=[cnt:7, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=7,cells=[{"BIGINT":1002},{"BIGINT UNSIGNED":3},{"VARCHAR":"10.0.4.9", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT UNSIGNED":2890},{"BIGINT UNSIGNED":1729061856962275},{"BIGINT UNSIGNED":1729061856983632000},{"BIGINT UNSIGNED":0},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:36.996345] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=18][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:36.996357] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=6][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f6521f600 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:192998632} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059893180432 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:2124}, prev=this=0x2b0f65219ba8 trans_version={val:1729061256953030000, v:0} scn={val:1729061256953030000, v:0} tx_id={txid:192997736} prev=0x2b0f65219680 next=(nil) modify_count=5 acc_checksum=2006411149 version=1729059893180432 type=1 flag=12 snapshot_barrier=4611686018427387803 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:4331}, snapshot_version={val:1729061856995226000, v:0}, *this={this=0x2b0f6520e0f0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=0 list_head=0x2b0f65219ba8 latest_compact_node=0x2b0f65219ba8 max_trans_version={val:1729061256953030000, v:0} max_trans_id=192997736 max_elr_trans_version={val:1729061256953030000, v:0} max_elr_trans_id=192997736 latest_compact_ts=1729061256953923 last_compact_cnt=5 total_trans_node_cnt=6 max_modify_scn={val:1729061256953030000, v:0} min_modify_scn={val:1729060056817495000, v:0}}) [2024-10-16 14:57:36.996372] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=15][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f6521f600 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:192998632} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059893180432 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:60} seq_no={branch:0, seq:2124}, ctx={this:0x2b0e0b6d2080, ls_id:{id:1}, ls:0x2b0e076c4150, branch:0, timeout:1729061886895067, tablet_id:{id:444}, table_iter:0x2b0e0b6d1210, table_version:1717560275661600, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061886895067, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061856995226000, v:0}, tx_id:{txid:192998632}, scn:{branch:0, seq:2121}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e076d2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:192998632}, tx_desc:{this:0x2b0e0fd1a0f0, tx_id:{txid:192998632}, state:1, addr:"10.0.4.9:2890", tenant_id:1001, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061856995226, commit_ts:-1, finish_ts:-1, timeout_us:29999990, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061856993106, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0df705ba10, mem_ctx:0x2b0df705cf90, tx_scn:{branch:0, seq:2124}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:36.996403] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560275661600 max_table_version=1717560275661600 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:192998632} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:192998632}, data:0x2b0e0b6ce438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729059893180432, seq_no:{branch:0, seq:2124}, scn:{val:4611686018427387903, v:0}, column_cnt:7}) [2024-10-16 14:57:36.996416] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=0][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:36.996419] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=2][errcode=-5024] failed to write table row(ret=-5024, table id=444, col_descs=[cnt:7, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=7,cells=[{"BIGINT":1002},{"BIGINT UNSIGNED":4},{"VARCHAR":"10.0.4.9", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT UNSIGNED":2890},{"BIGINT UNSIGNED":1729061856962275},{"BIGINT UNSIGNED":4611686018427387903},{"BIGINT UNSIGNED":0},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:36.996448] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:36.996708] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9989241703-0-0] [lt=27] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3177}, to_seq_no={branch:0, seq:2122}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560275661600 max_table_version=1717560275661600 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:192998632} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:37.001811] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:37.001826] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:37.002192] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=209, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:37.002207] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=53, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:37.002213] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:37.004624] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.005849] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061857005790000, start_id=1729061857005790000, end_id=1729061857005790001) [2024-10-16 14:57:37.006103] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9989241704-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061857005790000, start_id=1729061857005790000, end_id=1729061857005790001) [2024-10-16 14:57:37.006191] INFO [LIB] stat (utility.h:1165) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9989241704-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:37.006205] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9989241704-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=12275, avg_group_log_size=681) [2024-10-16 14:57:37.006228] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9989241704-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11140) [2024-10-16 14:57:37.006257] INFO [LIB] stat (utility.h:1165) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9989241704-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:37.006285] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:142) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9989241704-0-0] [lt=8] transaction rpc statistics(total_rt=1786, total_process=19, avg_rt=89) [2024-10-16 14:57:37.006617] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=458, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:37.006642] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=24] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=12275) [2024-10-16 14:57:37.006660] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=127, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:37.006681] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D57DD-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11140) [2024-10-16 14:57:37.006728] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:37.006738] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:37.006744] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=990, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:37.006752] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=703, avg_log_submit_to_slide_time=958) [2024-10-16 14:57:37.006759] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:37.010654] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=4] Garbage Collector is running(seq_=656698, gc_interval=10000000) [2024-10-16 14:57:37.010685] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=14] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:37.010699] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=8] gc_check_member_list_ cost time(ret=0, time_us=31) [2024-10-16 14:57:37.010707] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:37.011429] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:37.011450] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=16] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:37.011464] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=14] execute_gc cost time(ret=0, time_us=15) [2024-10-16 14:57:37.011472] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] ObSafeDestroyHandler start process [2024-10-16 14:57:37.020079] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:37.027281] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061857027279, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007616, add_bytes: 0 [2024-10-16 14:57:37.030424] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=2958, avg_group_log_size=328) [2024-10-16 14:57:37.030714] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=272, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:37.030729] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=86, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:37.032103] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061857030958000, start_id=1729061857030958000, end_id=1729061857030958001) [2024-10-16 14:57:37.042053] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:37.042069] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:37.042084] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291907634, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:37.043592] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061857042680000, start_id=1729061857042680000, end_id=1729061857042680001) [2024-10-16 14:57:37.043629] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=23] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061857043626], gts:1729061857042680000, latest_srr:[mts=1729061857043626]}) [2024-10-16 14:57:37.056878] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.056911] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061857056369000, start_id=1729061857056369000, end_id=1729061857056369001) [2024-10-16 14:57:37.071768] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=347, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:37.071788] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=864, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:37.071825] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:37.075003] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:37.075022] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:37.075325] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:37.075343] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:37.085860] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:37.085881] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=14] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151993940, tenant_id=1) [2024-10-16 14:57:37.085895] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:37.086748] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:37.086775] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598818, miss_cnt:9, cache_read_size:149777906417, hit_ratio:"9.999999345924657224e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:37.096443] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:37.096494] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061857095507000, start_id=1729061857095507000, end_id=1729061857095507001) [2024-10-16 14:57:37.097243] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB APPEND COST TIME](cur_stat_count=53, stat_interval=5000000, avg (count/cost)=41, this=0x2b0e07718280, extra_info="") [2024-10-16 14:57:37.097257] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=13] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=53, stat_interval=5000000, avg (count/cost)=2515, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:57:37.097260] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=53, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:57:37.097264] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=53, stat_interval=5000000, avg (count/cost)=-845, this=0x2b0e07718910, extra_info="") [2024-10-16 14:57:37.097267] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=53, stat_interval=5000000, avg (count/cost)=1711, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:57:37.100450] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA4-0-0] [lt=19] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061857068196, keepalive_ts:-1}) [2024-10-16 14:57:37.100469] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA4-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061857068196, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:37.100507] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA5-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061857068242, keepalive_ts:-1}) [2024-10-16 14:57:37.100512] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA5-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061857068242, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:37.102337] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=18][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:0, dropped:13968, tid:22487}]) [2024-10-16 14:57:37.102572] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1089) [2024-10-16 14:57:37.102610] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:57:37.104086] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061857102554000, start_id=1729061857102554000, end_id=1729061857102554001) [2024-10-16 14:57:37.104696] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.113469] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061846112316, end_time=1729061857113468, dag_status={}) [2024-10-16 14:57:37.113485] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=14] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:37.113483] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:37.113492] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:37.113496] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:37.113496] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:37.113500] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:37.113503] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:37.113507] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:37.113510] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:37.113513] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:37.113517] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:37.113520] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113526] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113530] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113534] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113538] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113542] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113546] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113550] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113554] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113557] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113561] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113565] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113569] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113573] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113577] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113581] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113585] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113589] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113592] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113596] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113608] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113612] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113616] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113620] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113623] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113627] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113631] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113634] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113638] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113642] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113649] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113655] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113665] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113669] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113674] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113678] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113682] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113686] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113690] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113694] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113700] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113704] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113711] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113714] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113721] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113728] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113732] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113738] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113742] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113745] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113750] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113757] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113761] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113765] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113772] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113776] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113783] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113786] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:37.113798] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=11] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:37.113801] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:37.113807] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:37.113810] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:37.113815] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:37.113818] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:37.113823] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:37.113825] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:37.113828] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:37.113830] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:37.113836] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:37.118297] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=7][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:37.120238] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601389, offset=9622080}, end_cursor:ObLogCursor{file_id=192, log_id=47601390, offset=9622364}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2de0050, len:3520, log_data_len:284, seq:47601389, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:37.120807] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87B-0-0] [lt=40] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:37.120825] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87B-0-0] [lt=9] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061856983632001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061856983632001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:37.121980] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA49F-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061857120912000, start_id=1729061857120912000, end_id=1729061857120912001) [2024-10-16 14:57:37.122133] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA87B-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061857121982000, start_id=1729061857121982000, end_id=1729061857121982001) [2024-10-16 14:57:37.124986] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA87B-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061857124396000, start_id=1729061857124396001, end_id=1729061857124396002) [2024-10-16 14:57:37.125206] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=30] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561230, miss_cnt:1, cache_read_size:152590671592, hit_ratio:"9.999999928346863243e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:37.125511] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=4, total_batch_msg_count=0) [2024-10-16 14:57:37.126179] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=10] current monitor number(seq_=-1) [2024-10-16 14:57:37.127527] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:37.129690] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=26] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1825, mem_used=4159488) [2024-10-16 14:57:37.138572] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0FD5-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061857138218000, start_id=1729061857138218000, end_id=1729061857138218001) [2024-10-16 14:57:37.139591] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B99D-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061857139277000, start_id=1729061857139277000, end_id=1729061857139277001) [2024-10-16 14:57:37.139650] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B99F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061857139277000, start_id=1729061857139277000, end_id=1729061857139277001) [2024-10-16 14:57:37.139776] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0C2B-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061857139277000, start_id=1729061857139277000, end_id=1729061857139277001) [2024-10-16 14:57:37.139836] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9A1-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061857139277000, start_id=1729061857139277000, end_id=1729061857139277001) [2024-10-16 14:57:37.144360] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257734, offset=39025128}, end_cursor:ObLogCursor{file_id=197, log_id=49257735, offset=39025411}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e6c050, len:1560, log_data_len:283, seq:49257734, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:37.146409] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB36A-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061857145632000, start_id=1729061857145632000, end_id=1729061857145632001) [2024-10-16 14:57:37.146618] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D76-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:37.146629] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D76-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=39298, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:37.146634] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D76-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=1613, avg_log_submit_to_slide_time=39291) [2024-10-16 14:57:37.146641] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D76-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173104, log_task={header:{begin_lsn:{lsn:14352368872}, end_lsn:{lsn:14352368994}, log_id:115173104, min_scn:{val:1729061856638564004, v:0}, max_scn:{val:1729061856638564004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352368750}, prev_proposal_id:17, committed_end_lsn:{lsn:14352368872}, data_checksum:940009244, accum_checksum:2825539429, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061857045386, freeze_ts:1729061857045386, submit_ts:1729061857045395, flushed_ts:1729061857046411, gen_to_freeze cost time:0, gen_to_submit cost time:9, submit_to_flush cost time:1016}, fs_cb_begin_ts=1729061857146611, log_life_time=101225) [2024-10-16 14:57:37.146645] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:37.146672] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656906, miss_cnt:17, cache_read_size:8050159215, hit_ratio:"9.999997286812185582e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:37.146948] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=622, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:37.146958] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:37.146972] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=981, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:37.146997] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:37.147295] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257735, offset=39025411}, end_cursor:ObLogCursor{file_id=197, log_id=49257736, offset=39025693}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d94050, len:1277, log_data_len:282, seq:49257735, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:37.149244] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=31] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257736, offset=39025693}, end_cursor:ObLogCursor{file_id=197, log_id=49257737, offset=39025975}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d5c050, len:995, log_data_len:282, seq:49257736, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:37.149417] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257737, offset=39025975}, end_cursor:ObLogCursor{file_id=197, log_id=49257738, offset=39026257}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c18050, len:713, log_data_len:282, seq:49257737, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:37.149468] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650168, miss_cnt:32, cache_read_size:8050411722, hit_ratio:"9.999994892274884917e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:37.156941] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.171815] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90C5-0-0] [lt=17] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:37.182524] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5027-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:37.182544] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5027-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:37.185679] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:37.188867] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D74-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510325057252, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410591, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:37.197128] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:37.197280] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D84-0-0] [lt=14] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061867008855000, latest_log_ts:{val:1729061850421032001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:37.197338] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D84-0-0] [lt=55] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:37.198345] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=41] success to update min reserved snapshot(reserved_snapshot=1729060054425066000, duration=1800, snapshot_gc_ts=1729061854425066000) [2024-10-16 14:57:37.198932] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=13] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060054425066000) [2024-10-16 14:57:37.198948] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=15] update reserved snapshot finished(cost_ts=606, reserved_snapshot=1729060054425066000) [2024-10-16 14:57:37.205400] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.206407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061857206144000, start_id=1729061857206144000, end_id=1729061857206144001) [2024-10-16 14:57:37.207323] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1729061856610006002, v:0}, new_max_version={val:1729061856610006002, v:0}, last_min_version={val:1729061856610006000, v:0}, last_max_version={val:1729061856610006000, v:0}, total_time_us=1814, delta_us=597315) [2024-10-16 14:57:37.207350] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=25] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1729061856610006002, v:0}, min_version={val:1729061856610006002, v:0}, max_version={val:1729061856610006002, v:0}, in_service=true, leader_epoch=17) [2024-10-16 14:57:37.211050] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5878-0-0] [lt=15] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:37.211145] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5878-0-0] [lt=16][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:37.230542] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:37.230578] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:37.232276] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061857231381000, start_id=1729061857231381000, end_id=1729061857231381001) [2024-10-16 14:57:37.248806] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD09-0-0] [lt=7][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:37.257002] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.258758] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061857257139000, start_id=1729061857257139000, end_id=1729061857257139001) [2024-10-16 14:57:37.259346] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE771B-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:37.261252] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21531][T1_TntSharedTim][T1][Y0-0000000000000000-0-0] [lt=11] clear dtl interm result cost(us)(clear_cost=1861, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:37.262511] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F407B-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:37.262525] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F407B-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10663, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:37.264609] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:37.265954] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=192, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:37.265972] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:37.274155] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BF7-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:37.275840] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:37.275856] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:37.275880] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:37.275885] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:37.276333] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:37.282246] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1034-0-0] [lt=8] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:37.283042] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB97-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:37.289554] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=15] start to print blacklist info [2024-10-16 14:57:37.289728] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=12] ls blacklist refresh finish(cost_time=12459) [2024-10-16 14:57:37.296516] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061857295616000, start_id=1729061857295616000, end_id=1729061857295616001) [2024-10-16 14:57:37.297000] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:37.301978] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1089, avg_group_log_size=121) [2024-10-16 14:57:37.301991] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1089) [2024-10-16 14:57:37.304302] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061857302574000, start_id=1729061857302574000, end_id=1729061857302574001) [2024-10-16 14:57:37.305463] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.315806] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6724, mem_used=12478464) [2024-10-16 14:57:37.328770] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:37.328831] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=84] success to update min reserved snapshot(reserved_snapshot=1729060048719447000, duration=1800, snapshot_gc_ts=1729061848719447000) [2024-10-16 14:57:37.338892] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=8] update reserved snapshot finished(cost_ts=10062, reserved_snapshot=1729060048719447000) [2024-10-16 14:57:37.351143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B6EFF3DF1-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061857351113000, start_id=1729061857351113000, end_id=1729061857351113001) [2024-10-16 14:57:37.351541] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:37.351562] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:37.351570] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:37.351574] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:37.351578] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:37.351581] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17114) [2024-10-16 14:57:37.354222] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAB-0-0] [lt=15] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:37.354235] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAB-0-0] [lt=9] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=13) [2024-10-16 14:57:37.357065] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.362808] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:14352369238}], scn_array=[cnt:1, {val:1729061857139277002, v:0}], curr_size=122, accum_size=244, time_cost=11579) [2024-10-16 14:57:37.364936] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21656][T1_L0_G0][T1][YB4A0A000408-00061E9B4C9C294C-0-0] [lt=12][errcode=-4008] query response time size is 0(MTL_ID()=1, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:37.364953] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21656][T1_L0_G0][T1001][YB4A0A000408-00061E9B4C9C294C-0-0] [lt=14][errcode=-4008] query response time size is 0(MTL_ID()=1001, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:37.364957] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21656][T1_L0_G0][T1002][YB4A0A000408-00061E9B4C9C294C-0-0] [lt=4][errcode=-4008] query response time size is 0(MTL_ID()=1002, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:37.364962] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21656][T1_L0_G0][T1003][YB4A0A000408-00061E9B4C9C294C-0-0] [lt=3][errcode=-4008] query response time size is 0(MTL_ID()=1003, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:37.364965] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21656][T1_L0_G0][T1004][YB4A0A000408-00061E9B4C9C294C-0-0] [lt=3][errcode=-4008] query response time size is 0(MTL_ID()=1004, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:37.364968] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:105) [21656][T1_L0_G0][T1][YB4A0A000408-00061E9B4C9C294C-0-0] [lt=2][errcode=-4008] fail to execute(ret=-4008) [2024-10-16 14:57:37.368338] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=880, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:37.368356] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1179, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:37.384434] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:37.384448] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=2513, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:37.384452] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=1588, avg_log_submit_to_slide_time=2499) [2024-10-16 14:57:37.385983] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:37.388994] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F4D-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:37.389004] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F4D-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40311, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:37.394195] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:37.405522] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.406424] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061857405126000, start_id=1729061857405126000, end_id=1729061857405126001) [2024-10-16 14:57:37.406497] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=13008, avg_group_log_size=722) [2024-10-16 14:57:37.406507] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=12528) [2024-10-16 14:57:37.406740] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=273, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:37.406750] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=13008) [2024-10-16 14:57:37.406759] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=821, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:37.406772] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=12528) [2024-10-16 14:57:37.406779] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:37.407111] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD0ED-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:37.407127] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD0ED-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=1514, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:37.407132] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD0ED-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=1241, avg_log_submit_to_slide_time=1485) [2024-10-16 14:57:37.430732] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3069) [2024-10-16 14:57:37.430949] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3069) [2024-10-16 14:57:37.430982] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3069) [2024-10-16 14:57:37.432397] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061857431904000, start_id=1729061857431904000, end_id=1729061857431904001) [2024-10-16 14:57:37.432676] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:37.457125] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.461972] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE771F-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:37.462975] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061857462582000, start_id=1729061857462582000, end_id=1729061857462582001) [2024-10-16 14:57:37.463468] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BC1-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:37.463479] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BC1-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1057, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:37.468159] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7B-0-0] [lt=12] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:37.468183] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7B-0-0] [lt=22] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:37.471782] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=43857211, this={dio_aligned_buf:{buf_write_offset:1461, buf_padding_size:2635, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35369079, truncate_used_ts:14641048}, log_block_size:67108864, dir_fd:201, io_fd:197}, aligned_buf_len=4096, aligned_block_offset=43855872, buf_len=122, total_write_size_=12172, total_write_size_after_dio_=421888, ob_pwrite_used_ts=31712, count=100, each_pwrite_cost=317) [2024-10-16 14:57:37.476158] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:37.486877] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:37.486888] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:37.486898] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:37.486894] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED2-0-0] [lt=7] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:37.486903] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:37.486907] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:37.486909] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:37.486908] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED2-0-0] [lt=11] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:37.486912] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:37.486915] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:37.486917] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:37.486919] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:37.486923] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:37.489806] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:37.489821] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:37.489834] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70637788, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:37.495808] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] no need to update min/max version(new_version={val:1729061856542245004, v:0}, min_version={val:1729061856542245004, v:0}, max_version={val:1729061856542245004, v:0}) [2024-10-16 14:57:37.498504] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2076162, replace_num=31457) [2024-10-16 14:57:37.498534] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=29] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:37.502068] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:37.502085] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:37.502354] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=237, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:37.502364] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=47, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:37.502375] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:57:37.502410] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=33] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:37.502632] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4265-0-0] [lt=15] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=306, avg_log_submit_to_slide_time=2189) [2024-10-16 14:57:37.502796] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4266-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:37.502816] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4266-0-0] [lt=19] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=2041, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:37.504420] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061857503675000, start_id=1729061857503675000, end_id=1729061857503675001) [2024-10-16 14:57:37.505581] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.524355] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061857523717000, start_id=1729061857523717000, end_id=1729061857523717001) [2024-10-16 14:57:37.524456] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fcaa367e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:199664148} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729057415940861 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:4}, prev=this=0x2b0fcaa359c0 trans_version={val:1729061737517586000, v:0} scn={val:1729061737517586000, v:0} tx_id={txid:199662417} prev=0x2b0fcaa35840 next=(nil) modify_count=73 acc_checksum=1537021403 version=1729057415940861 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:7}, snapshot_version={val:1729061857523717000, v:0}, *this={this=0x2b0fcaa0dab0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fcaa359c0 latest_compact_node=0x2b0fcaa34860 max_trans_version={val:1729061737517586000, v:0} max_trans_id=199662417 max_elr_trans_version={val:1729061737517586000, v:0} max_elr_trans_id=199662417 latest_compact_ts=1729061617512899 last_compact_cnt=71 total_trans_node_cnt=74 max_modify_scn={val:1729061737517586000, v:0} min_modify_scn={val:1729057417374933002, v:0}}) [2024-10-16 14:57:37.524487] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fcaa367e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:199664148} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729057415940861 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:4}, ctx={this:0x2b0e5e44a150, ls_id:{id:1}, ls:0x2b0e3aec4150, branch:0, timeout:1729061887424321, tablet_id:{id:351}, table_iter:0x2b0e5e4492d0, table_version:1717582245152744, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061887424321, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061857523717000, v:0}, tx_id:{txid:199664148}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e3aed2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:199664148}, tx_desc:{this:0x2b0ed9a7d910, tx_id:{txid:199664148}, state:2, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061857523717, active_ts:1729061857523717, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061887523712, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061857521598, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e2c678d50, mem_ctx:0x2b0e2c67a2d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:37.524543] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:199664148} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:199664148}, data:0x2b0e5e4464f8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729057415940861, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:17}) [2024-10-16 14:57:37.524559] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":1004},{"BIGINT":1004},{"BIGINT":3},{"BIGINT":74},{"BIGINT":1}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:199664148} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:37.524576] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:37.524581] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=351, col_descs=[cnt:17, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=17,cells=[{"BIGINT":1004},{"BIGINT":1004},{"BIGINT":3},{"BIGINT":74},{"BIGINT":1},{"TIMESTAMP":"2024-10-16 06:57:37.523717"},{"TIMESTAMP":"2024-10-16 06:57:37.523717"},{"BIGINT":1},{"BIGINT UNSIGNED":1729053449237798000},{"BIGINT UNSIGNED":17479894340},{"BIGINT UNSIGNED":17493458833},{"BIGINT UNSIGNED":1729061854598998002},{"VARCHAR":"DOING", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT":261},{"BIGINT":13564493},{"BIGINT":13564493},{"BIGINT":13564493},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:37.524657] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=53][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:37.524682] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:199664148} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:37.524826] INFO [SHARE] do_insert_or_update_row_ (ob_inner_table_operator.cpp:696) [22806][T1004_ArcSrv][T1004][YB4A0A000407-00061E9972A960C1-0-0] [lt=10] insert/update one row(row={key:{tenant_id:1004, dest_id:1004, round_id:3, piece_id:74, ls_id:{id:1}}, incarnation:1, start_scn:{val:1729053449237798000, v:0}, start_lsn:17479894340, checkpoint_scn:{val:1729061854598998002, v:0}, lsn:17493458833, archive_file_id:261, archive_file_offset:13564493, input_bytes:13564493, output_bytes:13564493, state:{status:"DOING"}}, affected_rows=2, sql=INSERT INTO __all_ls_log_archive_progress (tenant_id, dest_id, round_id, piece_id, ls_id, incarnation, start_scn, min_lsn, checkpoint_scn, max_lsn, file_id, file_offset, input_bytes, output_bytes, status) VALUES (1004, 1004, 3, 74, 1, 1, 1729053449237798000, 17479894340, 1729061854598998002, 17493458833, 261, 13564493, 13564493, 13564493, 'DOING') ON DUPLICATE KEY UPDATE incarnation = 1, start_scn = 1729053449237798000, min_lsn = 17479894340, checkpoint_scn = 1729061854598998002, max_lsn = 17493458833, file_id = 261, file_offset = 13564493, input_bytes = 13564493, output_bytes = 13564493, status = 'DOING') [2024-10-16 14:57:37.525528] INFO [ARCHIVE] do_persist_ (ob_archive_persist_mgr.cpp:694) [22806][T1004_ArcSrv][T1004][YB4A0A000407-00061E9972A960C1-0-0] [lt=20] [LOG_STREAM] [ARCHIVE_PROGRESS] persist archive progress succ(info={key:{tenant_id:1004, dest_id:1004, round_id:3, piece_id:74, ls_id:{id:1}}, incarnation:1, start_scn:{val:1729053449237798000, v:0}, start_lsn:17479894340, checkpoint_scn:{val:1729061854598998002, v:0}, lsn:17493458833, archive_file_id:261, archive_file_offset:13564493, input_bytes:13564493, output_bytes:13564493, state:{status:"DOING"}}, total_time=1960) [2024-10-16 14:57:37.526414] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fcaa36c00 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:199664149} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729057415940861 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:4}, prev=this=0x2b0fcaa35de0 trans_version={val:1729061737519707001, v:0} scn={val:1729061737519707001, v:0} tx_id={txid:199662418} prev=0x2b0fcaa35c60 next=(nil) modify_count=73 acc_checksum=533321968 version=1729057415940861 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:7}, snapshot_version={val:1729061857525861000, v:0}, *this={this=0x2b0fcaa0def0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fcaa35de0 latest_compact_node=0x2b0fcaa34d60 max_trans_version={val:1729061737519707001, v:0} max_trans_id=199662418 max_elr_trans_version={val:1729061737519707001, v:0} max_elr_trans_id=199662418 latest_compact_ts=1729061617515118 last_compact_cnt=71 total_trans_node_cnt=74 max_modify_scn={val:1729061737519707001, v:0} min_modify_scn={val:1729057417377272001, v:0}}) [2024-10-16 14:57:37.526440] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=26][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fcaa36c00 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:199664149} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729057415940861 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:120} seq_no={branch:0, seq:4}, ctx={this:0x2b0e5e44a150, ls_id:{id:1}, ls:0x2b0e3aec4150, branch:0, timeout:1729061887426330, tablet_id:{id:351}, table_iter:0x2b0e5e4492d0, table_version:1717582245152744, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061887426330, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061857525861000, v:0}, tx_id:{txid:199664149}, scn:{branch:0, seq:2}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e3aed2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:199664149}, tx_desc:{this:0x2b0ed9a837f0, tx_id:{txid:199664149}, state:2, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061857525861, active_ts:1729061857525861, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061887525856, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061857524804, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, tx_ctx:0x2b0e2c678d50, mem_ctx:0x2b0e2c67a2d0, tx_scn:{branch:0, seq:4}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:37.526486] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=45][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:199664149} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:199664149}, data:0x2b0e5e4464f8, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729057415940861, seq_no:{branch:0, seq:4}, scn:{val:4611686018427387903, v:0}, column_cnt:17}) [2024-10-16 14:57:37.526498] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=12][errcode=0] primary key duplication conflict(key={"BIGINT":1004},{"BIGINT":1004},{"BIGINT":3},{"BIGINT":74},{"BIGINT":1001}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:199664149} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:37.526510] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:37.526515] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=351, col_descs=[cnt:17, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=17,cells=[{"BIGINT":1004},{"BIGINT":1004},{"BIGINT":3},{"BIGINT":74},{"BIGINT":1001},{"TIMESTAMP":"2024-10-16 06:57:37.525861"},{"TIMESTAMP":"2024-10-16 06:57:37.525861"},{"BIGINT":1},{"BIGINT UNSIGNED":1729053449237798000},{"BIGINT UNSIGNED":13635346983},{"BIGINT UNSIGNED":13645410827},{"BIGINT UNSIGNED":1729061854598998000},{"VARCHAR":"DOING", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT":204},{"BIGINT":10063844},{"BIGINT":10063844},{"BIGINT":10063844},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:37.526564] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=33][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:37.526582] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5}, to_seq_no={branch:0, seq:3}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717582245152744 max_table_version=1717582245152744 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:199664149} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:37.526715] INFO [SHARE] do_insert_or_update_row_ (ob_inner_table_operator.cpp:696) [22806][T1004_ArcSrv][T1004][YB4A0A000407-00061E9972A960C1-0-0] [lt=16] insert/update one row(row={key:{tenant_id:1004, dest_id:1004, round_id:3, piece_id:74, ls_id:{id:1001}}, incarnation:1, start_scn:{val:1729053449237798000, v:0}, start_lsn:13635346983, checkpoint_scn:{val:1729061854598998000, v:0}, lsn:13645410827, archive_file_id:204, archive_file_offset:10063844, input_bytes:10063844, output_bytes:10063844, state:{status:"DOING"}}, affected_rows=2, sql=INSERT INTO __all_ls_log_archive_progress (tenant_id, dest_id, round_id, piece_id, ls_id, incarnation, start_scn, min_lsn, checkpoint_scn, max_lsn, file_id, file_offset, input_bytes, output_bytes, status) VALUES (1004, 1004, 3, 74, 1001, 1, 1729053449237798000, 13635346983, 1729061854598998000, 13645410827, 204, 10063844, 10063844, 10063844, 'DOING') ON DUPLICATE KEY UPDATE incarnation = 1, start_scn = 1729053449237798000, min_lsn = 13635346983, checkpoint_scn = 1729061854598998000, max_lsn = 13645410827, file_id = 204, file_offset = 10063844, input_bytes = 10063844, output_bytes = 10063844, status = 'DOING') [2024-10-16 14:57:37.533221] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=17) [2024-10-16 14:57:37.543979] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061857543807000, start_id=1729061857543807000, end_id=1729061857543807001) [2024-10-16 14:57:37.544000] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:37.544177] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061857544166], gts:1729061857543807000, latest_srr:[mts=1729061857544166]}) [2024-10-16 14:57:37.549788] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BF-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:37.549815] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9BF-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061857139277003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061857139277003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:37.550857] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9BF-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061857550162000, start_id=1729061857550162000, end_id=1729061857550162001) [2024-10-16 14:57:37.553364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9BF-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061857553311000, start_id=1729061857553311000, end_id=1729061857553311001) [2024-10-16 14:57:37.553980] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:37.557188] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.562013] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7721-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938226, log_task={header:{begin_lsn:{lsn:213058231976}, end_lsn:{lsn:213058232098}, log_id:220938226, min_scn:{val:1729061857139277004, v:0}, max_scn:{val:1729061857139277004, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058231854}, prev_proposal_id:88, committed_end_lsn:{lsn:213058231976}, data_checksum:1715074492, accum_checksum:3861363882, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061857461958, freeze_ts:1729061857461958, submit_ts:1729061857461963, flushed_ts:1729061857463127, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:1164}, fs_cb_begin_ts=1729061857562007, log_life_time=100049) [2024-10-16 14:57:37.562337] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:37.567646] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCABF-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061857567082000, start_id=1729061857567082000, end_id=1729061857567082001) [2024-10-16 14:57:37.568672] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.569058] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.569075] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.569438] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.576231] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1125, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:37.576248] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=347, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:37.576448] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:37.576554] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD197-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:37.576564] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD197-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=2354, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:37.582589] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D4-0-0] [lt=17] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061857573775, keepalive_ts:-1}) [2024-10-16 14:57:37.582624] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D4-0-0] [lt=34] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061857573775, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:37.584596] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1002}, {id:1001}]) [2024-10-16 14:57:37.586250] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:37.586686] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [22451][T1003_ObXAHbWor][T1003][Y0-0000000000000000-0-0] [lt=12] XA scheduler heartbeat task statistics(avg_time=1) [2024-10-16 14:57:37.587252] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:37.587271] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:37.587387] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=8] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:57:37.592536] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4C-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:37.592562] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4C-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061857567082000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061857567082000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:37.593790] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C4C-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061857592990000, start_id=1729061857592990000, end_id=1729061857592990001) [2024-10-16 14:57:37.596594] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061857596468000, start_id=1729061857596468000, end_id=1729061857596468001) [2024-10-16 14:57:37.596737] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C4C-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061857596468000, start_id=1729061857596468001, end_id=1729061857596468002) [2024-10-16 14:57:37.597339] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061857596468000, start_id=1729061857596468002, end_id=1729061857596468003) [2024-10-16 14:57:37.598911] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D5-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061857566684, keepalive_ts:-1}) [2024-10-16 14:57:37.598928] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D5-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061857566684, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:37.605642] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.606510] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061857604905000, start_id=1729061857604905000, end_id=1729061857604905001) [2024-10-16 14:57:37.608658] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4920, mem_used=10398720) [2024-10-16 14:57:37.632451] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061857632198000, start_id=1729061857632198000, end_id=1729061857632198001) [2024-10-16 14:57:37.640220] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9A2-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061857639595000, start_id=1729061857639595000, end_id=1729061857639595001) [2024-10-16 14:57:37.640344] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9A6-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061857639595000, start_id=1729061857639595000, end_id=1729061857639595001) [2024-10-16 14:57:37.645764] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D80-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:37.645785] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D80-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:37.645794] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D80-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:37.645808] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D80-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173109, log_task={header:{begin_lsn:{lsn:14352369482}, end_lsn:{lsn:14352369604}, log_id:115173109, min_scn:{val:1729061857139277004, v:0}, max_scn:{val:1729061857139277004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352369360}, prev_proposal_id:17, committed_end_lsn:{lsn:14352369482}, data_checksum:940009244, accum_checksum:3047054623, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061857545677, freeze_ts:1729061857545677, submit_ts:1729061857545682, flushed_ts:1729061857545902, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:220}, fs_cb_begin_ts=1729061857645801, log_life_time=100124) [2024-10-16 14:57:37.647280] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=11] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:57:37.657247] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.658042] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:57:37.658064] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=8] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:37.658071] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:37.658076] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:37.660153] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=111, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:57:37.660314] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=160] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:37.663121] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3ACFEA-0-0] [lt=15] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.738636016845703125e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=869, Throttle Trigger(MB)=521, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.115848223054147326e+01) [2024-10-16 14:57:37.663302] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061857662430000, start_id=1729061857662430000, end_id=1729061857662430001) [2024-10-16 14:57:37.666897] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:37.671655] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2BFF-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:37.671689] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=9] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:57:37.672520] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061857671914000, start_id=1729061857671914000, end_id=1729061857671914001) [2024-10-16 14:57:37.686583] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:37.698467] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:37.698611] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:37.702463] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:37.702705] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=14, stat_interval=5000000, avg (count/cost)=28, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:57:37.702716] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=14, stat_interval=5000000, avg (count/cost)=1658, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:57:37.702720] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=14, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:57:37.702723] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=14, stat_interval=5000000, avg (count/cost)=-940, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:57:37.702727] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=14, stat_interval=5000000, avg (count/cost)=747, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:57:37.703820] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=12] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:37.704598] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061857704185000, start_id=1729061857704185000, end_id=1729061857704185001) [2024-10-16 14:57:37.705702] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.724081] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=20] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:37.756225] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:37.756252] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=14] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:37.756260] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:37.757630] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.762319] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1938) [2024-10-16 14:57:37.770978] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:37.771742] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C01-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178416, log_task={header:{begin_lsn:{lsn:14538483247}, end_lsn:{lsn:14538483369}, log_id:115178416, min_scn:{val:1729061857640639000, v:0}, max_scn:{val:1729061857640639000, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538483125}, prev_proposal_id:23, committed_end_lsn:{lsn:14538483247}, data_checksum:1715074492, accum_checksum:135184642, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061857671643, freeze_ts:1729061857671643, submit_ts:1729061857671647, flushed_ts:1729061857673085, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1438}, fs_cb_begin_ts=1729061857771735, log_life_time=100092) [2024-10-16 14:57:37.776200] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:37.776220] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:37.779808] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:37.779827] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:37.779835] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=50) [2024-10-16 14:57:37.780594] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15DA-0-0] [lt=11] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:37.781485] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:37.786568] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:37.787783] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.787784] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.787798] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.787992] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.788433] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.789028] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.789122] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.789146] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.789159] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.789170] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.789181] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:37.792843] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D69-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:37.792872] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D69-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061857640639001, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061857640639001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:37.793867] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D69-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061857793139000, start_id=1729061857793139000, end_id=1729061857793139001) [2024-10-16 14:57:37.796776] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061857796322000, start_id=1729061857796322000, end_id=1729061857796322001) [2024-10-16 14:57:37.797188] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D69-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061857796322000, start_id=1729061857796322000, end_id=1729061857796322001) [2024-10-16 14:57:37.797248] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D69-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:37.797264] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D69-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:37.797583] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:207589542830}, scn={val:1729061857639595002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061857786910, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589542830}, next_to_submit_scn_:{val:1729061857639595002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589542708}, end_lsn:{lsn:207589542830}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061857639595001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061857639595001, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:2826140267, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061857639595001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598825, miss_cnt:9, cache_read_size:149777907271, hit_ratio:"9.999999345924690530e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:57:37.802686] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C426E-0-0] [lt=25] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:37.802698] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C426E-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1970, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:37.805733] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.806745] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061857804828000, start_id=1729061857804828000, end_id=1729061857804828001) [2024-10-16 14:57:37.820693] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=17] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:57:37.824983] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1}, {id:1001}]) [2024-10-16 14:57:37.828261] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:37.828277] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:37.828290] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52378711, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:37.829677] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=12] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:37.832584] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061857832117000, start_id=1729061857832117000, end_id=1729061857832117001) [2024-10-16 14:57:37.833632] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061853824118, last_first_register_time_us=1722494157071428, register_time_us=1729061853824118, curr_time_us=1729061857833631) [2024-10-16 14:57:37.833660] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=14] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061857833647, last_submit_register_req_time_us=1729061857833647, register_parent_reason=3) [2024-10-16 14:57:37.833672] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=11] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:37.833821] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:37.846370] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:37.846391] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:37.852616] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4250, mem_used=8318976) [2024-10-16 14:57:37.857690] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.858262] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22136][T1002_TntTransf][T1002][YB4A0A000407-00061E9AB24CC6BC-0-0] [lt=13] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:57:37.863407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061857861735000, start_id=1729061857861735000, end_id=1729061857861735001) [2024-10-16 14:57:37.871991] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:37.884783] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=69] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:37.885431] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:37.885490] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=14] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061857241966000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:37.886572] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F57-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:37.898686] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:37.901118] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=7] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:37.902300] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:37.902319] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:37.903029] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4272-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=13, avg_log_submit_to_flush_time=858, avg_log_submit_to_slide_time=2026) [2024-10-16 14:57:37.904682] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061857904206000, start_id=1729061857904206000, end_id=1729061857904206001) [2024-10-16 14:57:37.905820] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.911050] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=22] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:37.913598] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=14] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:37.914064] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=38] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=238, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:37.914085] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=205, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:37.914128] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:37.921844] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:37.941731] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2735, mem_used=6239232) [2024-10-16 14:57:37.943947] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=30] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:37.943990] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:57:37.943999] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=7] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:37.944005] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:37.944010] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:37.946290] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:57:37.946668] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=375] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:37.954893] INFO [COMMON] loop_running_dag_net_list (ob_tenant_dag_scheduler.cpp:3738) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] loop running dag net list(ret=0, running_dag_net_list_size=0, slow_dag_net_cnt=0, finished_dag_net_list_size=0) [2024-10-16 14:57:37.954909] INFO [COMMON] loop_finished_dag_net_list (ob_tenant_dag_scheduler.cpp:3761) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=16] loop finsihed dag net list(ret=0, finished_dag_net_list_size=0) [2024-10-16 14:57:37.957446] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:37.957465] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:37.957474] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=158) [2024-10-16 14:57:37.957752] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:37.962244] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7727-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1816, avg_group_log_size=181) [2024-10-16 14:57:37.972785] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90D5-0-0] [lt=29] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:37.972799] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90D5-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18138, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:37.972803] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90D5-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=515, avg_log_submit_to_slide_time=18131) [2024-10-16 14:57:37.973004] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:37.976475] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:37.976664] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1018, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:37.976677] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=198, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:37.976963] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1A3-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=1351, avg_log_submit_to_slide_time=2038) [2024-10-16 14:57:37.980716] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=25] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 0.99, rt: 197.00; ) [2024-10-16 14:57:37.980733] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:37.980768] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.97, rt: 155.00; ) [2024-10-16 14:57:37.980773] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:37.980781] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:37.983652] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=33] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:37.983840] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:37.984577] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:37.984609] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=31] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:37.984633] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=19] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:37.984693] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061869029444000, latest_log_ts:{val:1729061852392712001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:37.984740] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:37.984795] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:57:37.984804] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=0, rec_log_ts={val:1729061852392712001, v:0}) [2024-10-16 14:57:37.984808] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:37.984812] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:37.984818] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:37.984827] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:57:37.984835] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:37.984840] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:37.986568] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F59-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:37.986665] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:207589543008}, scn={val:1729061857639595003, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:37.986693] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:15, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061857986592, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589543064}, next_to_submit_scn_:{val:1729061857639595004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589542952}, end_lsn:{lsn:207589543064}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061857639595003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061857639595003, v:0}, data_checksum:3676497381, flag:1}}, init_mode_version:0, accumulate_checksum:1477473902, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061857639595003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598827, miss_cnt:9, cache_read_size:149777907505, hit_ratio:"9.999999345924699412e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589543008}, scn_:{val:1729061857639595003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061857878293, first_handle_ts_:1729061857986692, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f38a65ce0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:37.986890] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:37.986981] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060050192549000) [2024-10-16 14:57:37.986994] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589543008}, scn_:{val:1729061857639595003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061857878293, first_handle_ts_:1729061857986968, replay_cost_:24, retry_cost_:-1, read_log_buf_:0x2b0f38a65cf0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061857986592, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589543064}, next_to_submit_scn_:{val:1729061857639595004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589542952}, end_lsn:{lsn:207589543064}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061857639595003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061857639595003, v:0}, data_checksum:3676497381, flag:1}}, init_mode_version:0, accumulate_checksum:1477473902, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061857639595003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598827, miss_cnt:9, cache_read_size:149777907505, hit_ratio:"9.999999345924699412e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_queue_idx=21, ret=0) [2024-10-16 14:57:37.987955] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D535F-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1454) [2024-10-16 14:57:37.996557] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061857996555, bytes: 4446812486491, bw: 0.483851 MB/s, add_ts: 1001168, add_bytes: 507947 [2024-10-16 14:57:37.996908] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=54] get number(ret=0, service_type_=0, range=1, base_id=1729061857996300000, start_id=1729061857996300000, end_id=1729061857996300001) [2024-10-16 14:57:37.998782] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=10] report RowHolderMapper summary info(count=1, bkt_cnt=248) [2024-10-16 14:57:38.000592] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=22] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:38.002364] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:38.002378] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:38.003280] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=268, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:38.003297] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=631, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:38.003325] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:38.005854] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:38.005872] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:38.005878] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.006785] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061858005900000, start_id=1729061858005900000, end_id=1729061858005900001) [2024-10-16 14:57:38.031218] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3534, avg_group_log_size=353) [2024-10-16 14:57:38.031503] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=208, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:38.031520] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=34, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:38.031535] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:38.031678] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C0FFC-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=743, avg_log_submit_to_slide_time=950) [2024-10-16 14:57:38.031842] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] no need to update min/max version(new_version={val:1729061857632198000, v:0}, min_version={val:1729061857632198000, v:0}, max_version={val:1729061857632198000, v:0}) [2024-10-16 14:57:38.033748] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=3] LogLoopThread round_cost_time(us)(round_cost_time=14) [2024-10-16 14:57:38.034894] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061858034892, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007613, add_bytes: 0 [2024-10-16 14:57:38.044335] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061858043985000, start_id=1729061858043985000, end_id=1729061858043985001) [2024-10-16 14:57:38.044357] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=595, get_gts_with_stc_cnt=32, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=29, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:38.044365] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061858044329], gts:1729061858043985000, latest_srr:[mts=1729061858044329]}) [2024-10-16 14:57:38.044382] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=544, get_gts_with_stc_cnt=8, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:38.057811] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.063556] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061858061994000, start_id=1729061858061994000, end_id=1729061858061994001) [2024-10-16 14:57:38.064227] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:38.072859] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=4] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:38.073485] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:38.073532] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061850192549000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:38.074955] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=339, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:38.074973] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=269, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:38.075012] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:38.076335] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:38.076350] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:38.076970] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1A6-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:38.076993] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE03-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:38.087256] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=50] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:38.087281] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598829, miss_cnt:9, cache_read_size:149777907749, hit_ratio:"9.999999345924709404e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:38.098749] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:38.103170] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15D8-0-0] [lt=11] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:38.105915] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.106659] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:38.106704] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:38.107499] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061858107428000, start_id=1729061858107428000, end_id=1729061858107428001) [2024-10-16 14:57:38.113508] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:38.113523] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:38.122645] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=11] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97915334656) [2024-10-16 14:57:38.126268] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=11] current monitor number(seq_=-1) [2024-10-16 14:57:38.127435] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87C-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:38.127461] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87C-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061857639595005, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061857639595005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:38.127611] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:38.128648] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA87C-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061858128273000, start_id=1729061858128273000, end_id=1729061858128273001) [2024-10-16 14:57:38.129785] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=1953, mem_used=4159488) [2024-10-16 14:57:38.130081] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601390, offset=9622364}, end_cursor:ObLogCursor{file_id=192, log_id=47601391, offset=9622648}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c7c050, len:3236, log_data_len:284, seq:47601390, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:38.131874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A0-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061858130389000, start_id=1729061858130389000, end_id=1729061858130389001) [2024-10-16 14:57:38.131947] INFO [LIB] stat (utility.h:1165) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A0-0-0] [lt=21] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:38.131959] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A0-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10944, avg_group_log_size=576) [2024-10-16 14:57:38.131969] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A0-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11530) [2024-10-16 14:57:38.131977] INFO [LIB] stat (utility.h:1165) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A0-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:38.132664] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA87C-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061858132000000, start_id=1729061858132000000, end_id=1729061858132000001) [2024-10-16 14:57:38.132866] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061858132000000, start_id=1729061858132000001, end_id=1729061858132000002) [2024-10-16 14:57:38.132966] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D57F0-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11530) [2024-10-16 14:57:38.133150] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=324, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:38.133163] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10944) [2024-10-16 14:57:38.133173] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=278, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:38.133235] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:38.133244] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=1384, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:38.133255] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=10, avg_log_gen_to_submit_time=39, avg_log_submit_to_flush_time=634, avg_log_submit_to_slide_time=1345) [2024-10-16 14:57:38.133263] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=11, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:38.133610] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109203040, max_lsn:17959690698, max_scn:{val:1729061858043985001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7385580775}, reuse_lsn:{lsn:17959690698}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17917747658}}, last_submit_lsn:{lsn:17959690576}, last_submit_end_lsn:{lsn:17959690698}, last_submit_log_id:109203040, last_submit_log_pid:246, max_flushed_lsn:{lsn:17959690576}, max_flushed_end_lsn:{lsn:17959690698}, max_flushed_log_pid:246, committed_end_lsn:{lsn:17959690698}, last_slide_log_id:109203040, last_slide_scn:{val:1729061858043985001, v:0}, last_slide_lsn:{lsn:17959690576}, last_slide_end_lsn:{lsn:17959690698}, last_slide_log_pid:246, last_slide_log_accum_checksum:3386021499, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1d4530}, [StateMgr]={this:0x2b0e0f1d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109203041, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261839590, palf_role_change_cb:0x2b0e0f1df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, reconfig_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, persistent_config_version:{proposal_id:246, config_seq:121}, ms_ack_list:0, resend_config_version:{proposal_id:246, config_seq:121}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258131745, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258131745, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262039736, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728994126835081001, v:0}, min_block_id:265, min_block_min_scn:{val:1728952113212346001, v:0}, base_lsn_for_block_gc:{lsn:17849868288}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, log_config_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, log_snapshot_meta:{version:1, base_lsn:{lsn:17849868288}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1798144}, readable_log_tail:{lsn:1798144}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17984077824}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/meta", dir_fd:167, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65306624, block_header_serialize_buf_:0x2b0e0f1da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17959690698}, readable_log_tail:{lsn:17959690698}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17916973056}, min_scn:{val:1729035023717936004, v:0}, curr_block_id:267, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/log", dir_fd:169, min_block_id:265, max_block_id:268, curr_writable_block_id:267}, logical_block_size_:67104768, curr_block_writable_size_:24387126, block_header_serialize_buf_:0x2b0e0f1dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261838799, this:0x2b0e0f1d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:246, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:245, majority_max_lsn:{lsn:7385763025}, saved_end_lsn:{lsn:7385763025}, last_submit_prepare_req_time_us:1722494261839744, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1d89b0}) [2024-10-16 14:57:38.133751] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=140] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858106770, last_flushed_end_lsn:{lsn:17959690698}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858107230, last_flushed_end_lsn:{lsn:17959690698}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858107026, last_flushed_end_lsn:{lsn:17959690698}}]) [2024-10-16 14:57:38.133762] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=109203041, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858106770, last_flushed_end_lsn:{lsn:17959690698}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858107230, last_flushed_end_lsn:{lsn:17959690698}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858107026, last_flushed_end_lsn:{lsn:17959690698}}]) [2024-10-16 14:57:38.133935] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601391, offset=9622648}, end_cursor:ObLogCursor{file_id=192, log_id=47601392, offset=9622922}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c28050, len:2952, log_data_len:274, seq:47601391, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:38.134014] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=12] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:38.136494] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601392, offset=9622922}, end_cursor:ObLogCursor{file_id=192, log_id=47601393, offset=9623196}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cd4050, len:2678, log_data_len:274, seq:47601392, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:38.137562] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=23] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601393, offset=9623196}, end_cursor:ObLogCursor{file_id=192, log_id=47601394, offset=9623470}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cb8050, len:2404, log_data_len:274, seq:47601393, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:38.137640] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561240, miss_cnt:1, cache_read_size:152590673408, hit_ratio:"9.999999928346867684e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:38.140656] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9A7-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061858140521000, start_id=1729061858140521000, end_id=1729061858140521001) [2024-10-16 14:57:38.140718] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9A9-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061858140521000, start_id=1729061858140521000, end_id=1729061858140521001) [2024-10-16 14:57:38.140820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9AB-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061858140521000, start_id=1729061858140521000, end_id=1729061858140521001) [2024-10-16 14:57:38.141035] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B9AA-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061858140521000, start_id=1729061858140521000, end_id=1729061858140521001) [2024-10-16 14:57:38.145327] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0FE7-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061858144800000, start_id=1729061858144800000, end_id=1729061858144800001) [2024-10-16 14:57:38.146792] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1A8-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:38.146804] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1A8-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=36146, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:38.146807] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:38.146814] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1A8-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=1462, avg_log_submit_to_slide_time=36140) [2024-10-16 14:57:38.146824] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656917, miss_cnt:17, cache_read_size:8050160557, hit_ratio:"9.999997286812661867e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:38.150658] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257738, offset=39026257}, end_cursor:ObLogCursor{file_id=197, log_id=49257739, offset=39026540}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e08050, len:431, log_data_len:283, seq:49257738, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:38.152690] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB36B-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061858152207000, start_id=1729061858152207000, end_id=1729061858152207001) [2024-10-16 14:57:38.153683] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257739, offset=39026540}, end_cursor:ObLogCursor{file_id=197, log_id=49257740, offset=39026822}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d48050, len:4244, log_data_len:282, seq:49257739, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:38.155681] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257740, offset=39026822}, end_cursor:ObLogCursor{file_id=197, log_id=49257741, offset=39027104}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d70050, len:3962, log_data_len:282, seq:49257740, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:38.155902] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257741, offset=39027104}, end_cursor:ObLogCursor{file_id=197, log_id=49257742, offset=39027386}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d30050, len:3680, log_data_len:282, seq:49257741, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:38.155953] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650177, miss_cnt:32, cache_read_size:8050412820, hit_ratio:"9.999994892275618774e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:38.157869] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.172455] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C09-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:38.177747] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD008-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061858168969, keepalive_ts:-1}) [2024-10-16 14:57:38.177773] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD008-0-0] [lt=24] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061858168969, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:38.183398] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5028-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:38.183418] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5028-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:38.187239] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:38.187877] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15D2-0-0] [lt=17] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:38.193319] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D76-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510326061712, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410592, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:38.193358] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15D4-0-0] [lt=16] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:38.197025] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061858196610000, start_id=1729061858196610000, end_id=1729061858196610001) [2024-10-16 14:57:38.205930] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.206955] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061858205198000, start_id=1729061858205198000, end_id=1729061858205198001) [2024-10-16 14:57:38.208164] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1729061857543807004, v:0}, new_max_version={val:1729061857543807004, v:0}, last_min_version={val:1729061857543807002, v:0}, last_max_version={val:1729061857543807002, v:0}, total_time_us=2086, delta_us=664355) [2024-10-16 14:57:38.208183] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1729061857543807004, v:0}, min_version={val:1729061857543807004, v:0}, max_version={val:1729061857543807004, v:0}, in_service=true, leader_epoch=17) [2024-10-16 14:57:38.231356] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:38.231380] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:38.236027] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=37] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:38.237793] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:213058233604}, scn={val:1729061858141579001, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:38.237893] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060050192549000) [2024-10-16 14:57:38.237904] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:213058233604}, scn_:{val:1729061858141579001, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061858227729, first_handle_ts_:1729061858237881, replay_cost_:22, retry_cost_:-1, read_log_buf_:0x2b0f38a679a0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061858237750, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058233660}, next_to_submit_scn_:{val:1729061858141579002, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058233548}, end_lsn:{lsn:213058233660}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061858141579001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061858141579001, v:0}, data_checksum:4067203453, flag:0}}, init_mode_version:0, accumulate_checksum:2841310546, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061858141579001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561242, miss_cnt:1, cache_read_size:152590673642, hit_ratio:"9.999999928346868794e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}, replay_queue_idx=17, ret=0) [2024-10-16 14:57:38.246775] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1293, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:38.246794] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:38.246806] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:38.246853] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:38.257934] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.262397] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE772D-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:38.262911] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5365-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:38.262921] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5365-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=10694, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:38.263485] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] succ to submit log task to replay service(to_submit_lsn={lsn:213058233716}, to_submit_scn={val:1729061858141579002, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061858262907, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058233782}, next_to_submit_scn_:{val:1729061858141579003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058233660}, end_lsn:{lsn:213058233782}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061858141579002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061858141579002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:1508997293, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061858141579002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561243, miss_cnt:1, cache_read_size:152590673764, hit_ratio:"9.999999928346869904e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:38.264341] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061858264334000, start_id=1729061858264334000, end_id=1729061858264334001) [2024-10-16 14:57:38.276514] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:38.276529] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:38.276555] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:38.276560] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:38.276916] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:38.287266] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=50] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=85, stat_interval=1000000, avg (count/cost)=57, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:38.287283] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=85, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:38.298825] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:38.305966] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.306576] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:38.306592] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:38.308276] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061858306717000, start_id=1729061858306717000, end_id=1729061858306717001) [2024-10-16 14:57:38.308675] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22468][T1003_MediumLoo][T1003][YB4A0A000407-00061E9973DF38E0-0-0] [lt=11] MediumCheckTask(cost_ts=16) [2024-10-16 14:57:38.311639] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:38.311656] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:38.311671] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373593641, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:38.315909] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6799, mem_used=12478464) [2024-10-16 14:57:38.328832] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:38.332840] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061858332782000, start_id=1729061858332782000, end_id=1729061858332782001) [2024-10-16 14:57:38.357986] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.362971] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=63] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=2050, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:38.366274] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=9] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:38.372488] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C0D-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:38.377255] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:38.377358] WDIAG read_multi_row (ob_table_access_helper.h:238) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:38.378280] INFO delete_row (ob_table_access_helper.h:312) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=12] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1002, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061858377371) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061858377371) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:38.378354] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=28] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:38.378434] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=9] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:38.378481] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=7] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:38.387543] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:38.393535] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:38.393550] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:38.393566] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:38.394245] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:38.397112] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061858397037000, start_id=1729061858397037000, end_id=1729061858397037001) [2024-10-16 14:57:38.397840] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:38.398888] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=18] ====== tenant freeze timer task ====== [2024-10-16 14:57:38.398903] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:38.398914] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:38.406026] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.407012] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061858406237000, start_id=1729061858406237000, end_id=1729061858406237001) [2024-10-16 14:57:38.407074] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=11573, avg_group_log_size=680) [2024-10-16 14:57:38.407084] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11573) [2024-10-16 14:57:38.407434] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=247, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:38.407445] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11573) [2024-10-16 14:57:38.407454] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=112, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:38.407467] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11573) [2024-10-16 14:57:38.407474] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:38.408164] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C12-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:38.408173] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C12-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=629, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:38.408177] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C12-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=381, avg_log_submit_to_slide_time=600) [2024-10-16 14:57:38.431466] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3423) [2024-10-16 14:57:38.431749] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3423) [2024-10-16 14:57:38.431795] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3423) [2024-10-16 14:57:38.458046] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.462641] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=289, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:38.462662] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:38.462932] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5369-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=2050) [2024-10-16 14:57:38.464510] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061858463522000, start_id=1729061858463522000, end_id=1729061858463522001) [2024-10-16 14:57:38.465075] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BCA-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:38.465085] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BCA-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=522, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:38.468953] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7C-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:38.468979] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7C-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:38.476850] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:38.477051] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1B3-0-0] [lt=19] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:38.477063] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1B3-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1169, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:38.477070] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1B3-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=1049, avg_log_submit_to_slide_time=1150) [2024-10-16 14:57:38.477944] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD009-0-0] [lt=13] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061858469141, keepalive_ts:-1}) [2024-10-16 14:57:38.477960] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD009-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061858469141, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:38.487297] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D536A-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:38.487314] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D536A-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=16181, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:38.488738] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8EE-0-0] [lt=10] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:38.488758] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8EE-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:38.488777] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8EE-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:38.498868] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:38.506090] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.506271] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] ====== checkpoint timer task ====== [2024-10-16 14:57:38.506642] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:38.506659] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:38.507053] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=15] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:38.507089] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=35] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:38.507118] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=23] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:38.507193] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=12] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061865590307000, latest_log_ts:{val:1729061848964072001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:38.507253] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=56] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:38.507279] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:38.507293] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061848964072001, v:0}) [2024-10-16 14:57:38.507301] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:38.507304] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:38.507325] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:38.507339] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:38.507354] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:38.507373] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:38.507797] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=6] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:38.507815] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=15] get_min_unreplayed_log_info(lsn={lsn:14538484345}, scn={val:1729061858140521005, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061858472963, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538484345}, next_to_submit_scn_:{val:1729061858140521005, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538484223}, end_lsn:{lsn:14538484345}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061858140521004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061858140521004, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:4215317806, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061858140521004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650182, miss_cnt:32, cache_read_size:8050413430, hit_ratio:"9.999994892276026226e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:38.507856] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=39] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:38.507897] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=343, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:38.507907] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=255, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:38.507923] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:38.507940] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:38.507973] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:38.507977] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=846, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:38.508149] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4285-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=618, avg_log_submit_to_slide_time=809) [2024-10-16 14:57:38.508559] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061858506775000, start_id=1729061858506775000, end_id=1729061858506775001) [2024-10-16 14:57:38.530020] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A22749A19-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061858529421000, start_id=1729061858529421000, end_id=1729061858529421001) [2024-10-16 14:57:38.532722] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:38.532746] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=18] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:38.532765] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444507571, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:38.533004] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061858532693000, start_id=1729061858532693000, end_id=1729061858532693001) [2024-10-16 14:57:38.534471] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:220938237, max_lsn:213058234026, max_scn:{val:1729061858141579004, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:60464363030}, reuse_lsn:{lsn:213058234026}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:213016290986}}, last_submit_lsn:{lsn:213058233904}, last_submit_end_lsn:{lsn:213058234026}, last_submit_log_id:220938237, last_submit_log_pid:88, max_flushed_lsn:{lsn:213058233904}, max_flushed_end_lsn:{lsn:213058234026}, max_flushed_log_pid:88, committed_end_lsn:{lsn:213058234026}, last_slide_log_id:220938237, last_slide_scn:{val:1729061858141579004, v:0}, last_slide_lsn:{lsn:213058233904}, last_slide_end_lsn:{lsn:213058234026}, last_slide_log_pid:88, last_slide_log_accum_checksum:591414721, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0dac0530}, [StateMgr]={this:0x2b0e0dac4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, leader:"10.0.4.8:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:60468525403}, scan_disk_log_finished:true, last_check_start_id:220938238, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261639483, palf_role_change_cb:0x2b0e0dacb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, reconfig_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, persistent_config_version:{proposal_id:88, config_seq:21}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061855928746, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061855928746, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0dac1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, accepted_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1729009096591759001, v:0}, min_block_id:3154, min_block_min_scn:{val:1729006455168279002, v:0}, base_lsn_for_block_gc:{lsn:212587905024}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, log_config_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, log_snapshot_meta:{version:1, base_lsn:{lsn:212587905024}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}}, log_meta_storage:{log_tail:{lsn:5767168}, readable_log_tail:{lsn:5767168}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213124743168}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/meta", dir_fd:175, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61337600, block_header_serialize_buf_:0x2b0e0dac69f8, flashback_version:0}, log_storage:{log_tail:{lsn:213058234026}, readable_log_tail:{lsn:213058234026}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213057638400}, min_scn:{val:1729061839666529002, v:0}, curr_block_id:3175, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/log", dir_fd:177, min_block_id:3154, max_block_id:3176, curr_writable_block_id:3175}, logical_block_size_:67104768, curr_block_writable_size_:66509142, block_header_serialize_buf_:0x2b0e0dac86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261637707, this:0x2b0e0dac4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:87, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:86, majority_max_lsn:{lsn:60466038076}, saved_end_lsn:{lsn:60466038076}, last_submit_prepare_req_time_us:1722494261639634, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0dac49b0}) [2024-10-16 14:57:38.534614] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=139] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:38.534631] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=220938238, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:38.534652] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:211147263, max_lsn:204023096803, max_scn:{val:1729061858144800003, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57922170954}, reuse_lsn:{lsn:204023096803}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:203981153763}}, last_submit_lsn:{lsn:204023096681}, last_submit_end_lsn:{lsn:204023096803}, last_submit_log_id:211147263, last_submit_log_pid:130, max_flushed_lsn:{lsn:204023096681}, max_flushed_end_lsn:{lsn:204023096803}, max_flushed_log_pid:130, committed_end_lsn:{lsn:204023096803}, last_slide_log_id:211147263, last_slide_scn:{val:1729061858144800003, v:0}, last_slide_lsn:{lsn:204023096681}, last_slide_end_lsn:{lsn:204023096803}, last_slide_log_pid:130, last_slide_log_accum_checksum:2812622859, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1ec530}, [StateMgr]={this:0x2b0e0f1f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:211147264, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261539368, palf_role_change_cb:0x2b0e0f1f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, reconfig_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, persistent_config_version:{proposal_id:130, config_seq:17}, ms_ack_list:0, resend_config_version:{proposal_id:130, config_seq:17}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258231773, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258231773, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, accepted_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261739861, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1729008439619722001, v:0}, min_block_id:3020, min_block_min_scn:{val:1729005678665693002, v:0}, base_lsn_for_block_gc:{lsn:203595866112}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, log_config_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, log_snapshot_meta:{version:1, base_lsn:{lsn:203595866112}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}}, log_meta_storage:{log_tail:{lsn:6397952}, readable_log_tail:{lsn:6397952}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:204065599488}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/meta", dir_fd:145, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:60706816, block_header_serialize_buf_:0x2b0e0f1f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:204023096803}, readable_log_tail:{lsn:204023096803}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:203998494720}, min_scn:{val:1729060878194301003, v:0}, curr_block_id:3040, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/log", dir_fd:173, min_block_id:3020, max_block_id:3041, curr_writable_block_id:3040}, logical_block_size_:67104768, curr_block_writable_size_:42502685, block_header_serialize_buf_:0x2b0e0f1f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261539352, this:0x2b0e0f1f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:130, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:129, majority_max_lsn:{lsn:57923794664}, saved_end_lsn:{lsn:57923794664}, last_submit_prepare_req_time_us:1722494261539574, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1f09b0}) [2024-10-16 14:57:38.534750] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=97] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858507966, last_flushed_end_lsn:{lsn:204023096803}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858507243, last_flushed_end_lsn:{lsn:204023096803}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858508952, last_flushed_end_lsn:{lsn:204023096803}}]) [2024-10-16 14:57:38.534758] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=211147264, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858507966, last_flushed_end_lsn:{lsn:204023096803}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858507243, last_flushed_end_lsn:{lsn:204023096803}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858508952, last_flushed_end_lsn:{lsn:204023096803}}]) [2024-10-16 14:57:38.534779] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:218497385, max_lsn:207589543796, max_scn:{val:1729061858140521003, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57807896114}, reuse_lsn:{lsn:207589543796}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:207547600756}}, last_submit_lsn:{lsn:207589543674}, last_submit_end_lsn:{lsn:207589543796}, last_submit_log_id:218497385, last_submit_log_pid:81, max_flushed_lsn:{lsn:207589543674}, max_flushed_end_lsn:{lsn:207589543796}, max_flushed_log_pid:81, committed_end_lsn:{lsn:207589543796}, last_slide_log_id:218497385, last_slide_scn:{val:1729061858140521003, v:0}, last_slide_lsn:{lsn:207589543674}, last_slide_end_lsn:{lsn:207589543796}, last_slide_log_pid:81, last_slide_log_accum_checksum:2293987252, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0db32530}, [StateMgr]={this:0x2b0e0db36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, leader:"10.0.4.9:2890", leader_epoch:7, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:57814812323}, scan_disk_log_finished:true, last_check_start_id:218497386, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433991656, palf_role_change_cb:0x2b0e0db3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, reconfig_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, persistent_config_version:{proposal_id:81, config_seq:20}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061856932384, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061856932384, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0db33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, accepted_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1729009518284639001, v:0}, min_block_id:3073, min_block_min_scn:{val:1729006819670466001, v:0}, base_lsn_for_block_gc:{lsn:207152418816}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, log_config_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, log_snapshot_meta:{version:1, base_lsn:{lsn:207152418816}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}}, log_meta_storage:{log_tail:{lsn:5660672}, readable_log_tail:{lsn:5660672}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207622152192}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/meta", dir_fd:180, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61444096, block_header_serialize_buf_:0x2b0e0db389f8, flashback_version:0}, log_storage:{log_tail:{lsn:207589543796}, readable_log_tail:{lsn:207589543796}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207555047424}, min_scn:{val:1729060516440786001, v:0}, curr_block_id:3093, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/log", dir_fd:182, min_block_id:3073, max_block_id:3094, curr_writable_block_id:3093}, logical_block_size_:67104768, curr_block_writable_size_:32608396, block_header_serialize_buf_:0x2b0e0db3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433990608, this:0x2b0e0db36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:80, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:79, majority_max_lsn:{lsn:57811903042}, saved_end_lsn:{lsn:57811903042}, last_submit_prepare_req_time_us:1722494433991756, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0db369b0}) [2024-10-16 14:57:38.534879] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=100] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:38.534887] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=218497386, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:38.534924] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115178424, max_lsn:14538484345, max_scn:{val:1729061858140521004, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6487749074}, reuse_lsn:{lsn:14538484345}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14496541305}}, last_submit_lsn:{lsn:14538484223}, last_submit_end_lsn:{lsn:14538484345}, last_submit_log_id:115178424, last_submit_log_pid:23, max_flushed_lsn:{lsn:14538484223}, max_flushed_end_lsn:{lsn:14538484345}, max_flushed_log_pid:23, committed_end_lsn:{lsn:14538484345}, last_slide_log_id:115178424, last_slide_scn:{val:1729061858140521004, v:0}, last_slide_lsn:{lsn:14538484223}, last_slide_end_lsn:{lsn:14538484345}, last_slide_log_pid:23, last_slide_log_accum_checksum:4215317806, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43cc0530}, [StateMgr]={this:0x2b0e43cc4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, leader:"10.0.4.8:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6488080380}, scan_disk_log_finished:true, last_check_start_id:115178425, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261653045, palf_role_change_cb:0x2b0e43ccb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, reconfig_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, persistent_config_version:{proposal_id:23, config_seq:34}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061857833647, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061857833647, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43cc1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, accepted_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1728259580444569001, v:0}, min_block_id:201, min_block_min_scn:{val:1728204893605525001, v:0}, base_lsn_for_block_gc:{lsn:14427525120}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, log_config_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, log_snapshot_meta:{version:1, base_lsn:{lsn:14427525120}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}}, log_meta_storage:{log_tail:{lsn:1134592}, readable_log_tail:{lsn:1134592}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14561734656}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/meta", dir_fd:166, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65970176, block_header_serialize_buf_:0x2b0e43cc69f8, flashback_version:0}, log_storage:{log_tail:{lsn:14538484345}, readable_log_tail:{lsn:14538484345}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14494629888}, min_scn:{val:1729026040007117003, v:0}, curr_block_id:216, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/log", dir_fd:201, min_block_id:201, max_block_id:217, curr_writable_block_id:216}, logical_block_size_:67104768, curr_block_writable_size_:23250311, block_header_serialize_buf_:0x2b0e43cc86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261652788, this:0x2b0e43cc4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:22, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:21, majority_max_lsn:{lsn:6487893593}, saved_end_lsn:{lsn:6487893593}, last_submit_prepare_req_time_us:1722494261653224, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43cc49b0}) [2024-10-16 14:57:38.535018] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=94] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:38.535026] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=115178425, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:38.545488] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061858543578000, start_id=1729061858543578000, end_id=1729061858543578001) [2024-10-16 14:57:38.545502] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] ts response task statistics(alloc_count=261364310, free_count=130682154) [2024-10-16 14:57:38.546781] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21680][T1_L0_G0][T1][YB4A0A000409-00061E998CE5BF84-0-0] [lt=3][errcode=-4008] query response time size is 0(MTL_ID()=1, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:38.546798] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21680][T1_L0_G0][T1001][YB4A0A000409-00061E998CE5BF84-0-0] [lt=16][errcode=-4008] query response time size is 0(MTL_ID()=1001, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:38.546803] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21680][T1_L0_G0][T1002][YB4A0A000409-00061E998CE5BF84-0-0] [lt=4][errcode=-4008] query response time size is 0(MTL_ID()=1002, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:38.546807] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21680][T1_L0_G0][T1003][YB4A0A000409-00061E998CE5BF84-0-0] [lt=3][errcode=-4008] query response time size is 0(MTL_ID()=1003, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:38.546810] WDIAG [SERVER] process_curr_tenant (ob_information_query_response_time.cpp:131) [21680][T1_L0_G0][T1004][YB4A0A000409-00061E998CE5BF84-0-0] [lt=2][errcode=-4008] query response time size is 0(MTL_ID()=1004, time=0x7fff4ed5ee10, ret=-4008) [2024-10-16 14:57:38.546817] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:105) [21680][T1_L0_G0][T1][YB4A0A000409-00061E998CE5BF84-0-0] [lt=5][errcode=-4008] fail to execute(ret=-4008) [2024-10-16 14:57:38.557189] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C0-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:38.557216] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C0-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061858140521003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061858140521003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:38.558102] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.558233] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9C0-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061858558136000, start_id=1729061858558136000, end_id=1729061858558136001) [2024-10-16 14:57:38.560626] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C0-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061858560243000, start_id=1729061858560243000, end_id=1729061858560243001) [2024-10-16 14:57:38.561249] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:38.562730] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:38.568445] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.568741] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.568763] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.569333] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.576942] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=374, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:38.576958] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=222, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:38.576982] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:38.577129] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE13-0-0] [lt=24] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:38.577135] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE13-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=909, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:38.584700] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1001}, {id:1003}]) [2024-10-16 14:57:38.587503] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:38.587520] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:38.587854] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:38.597157] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061858596345000, start_id=1729061858596345000, end_id=1729061858596345001) [2024-10-16 14:57:38.600844] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4D-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:38.600871] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4D-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061858141579005, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061858141579005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:38.601918] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C4D-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061858601565000, start_id=1729061858601565000, end_id=1729061858601565001) [2024-10-16 14:57:38.604350] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C4D-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061858604006000, start_id=1729061858604006000, end_id=1729061858604006001) [2024-10-16 14:57:38.606131] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.608686] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061858608256000, start_id=1729061858608256000, end_id=1729061858608256001) [2024-10-16 14:57:38.608789] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=4937, mem_used=10398720) [2024-10-16 14:57:38.634987] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109593693, max_lsn:13645415341, max_scn:{val:1729061858543578000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6198843846}, reuse_lsn:{lsn:13645415341}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:13603472301}}, last_submit_lsn:{lsn:13645415219}, last_submit_end_lsn:{lsn:13645415341}, last_submit_log_id:109593693, last_submit_log_pid:17, max_flushed_lsn:{lsn:13645415219}, max_flushed_end_lsn:{lsn:13645415341}, max_flushed_log_pid:17, committed_end_lsn:{lsn:13645415341}, last_slide_log_id:109593693, last_slide_scn:{val:1729061858543578000, v:0}, last_slide_lsn:{lsn:13645415219}, last_slide_end_lsn:{lsn:13645415341}, last_slide_log_pid:17, last_slide_log_accum_checksum:318626426, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431ec530}, [StateMgr]={this:0x2b0e431f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109593694, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261552988, palf_role_change_cb:0x2b0e431f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:14, lock_type:0, added_list:0, removed_list:0, type:"UNLOCK_CONFIG_CHANGE"}, state:0, checking_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:30}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:30}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494257649102, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494257649102, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261753456, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1728296971089337004, v:0}, min_block_id:189, min_block_min_scn:{val:1728240898542283004, v:0}, base_lsn_for_block_gc:{lsn:13555163136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:13555163136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}}, log_meta_storage:{log_tail:{lsn:1011712}, readable_log_tail:{lsn:1011712}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13689372672}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/meta", dir_fd:198, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66093056, block_header_serialize_buf_:0x2b0e431f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:13645415341}, readable_log_tail:{lsn:13645415341}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13622267904}, min_scn:{val:1729042537979987002, v:0}, curr_block_id:203, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/log", dir_fd:200, min_block_id:189, max_block_id:204, curr_writable_block_id:203}, logical_block_size_:67104768, curr_block_writable_size_:43957331, block_header_serialize_buf_:0x2b0e431f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261552391, this:0x2b0e431f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:6198988121}, saved_end_lsn:{lsn:6198988121}, last_submit_prepare_req_time_us:1722494261553147, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431f09b0}) [2024-10-16 14:57:38.635133] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=143] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858576974, last_flushed_end_lsn:{lsn:13645415341}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858577202, last_flushed_end_lsn:{lsn:13645415341}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858577236, last_flushed_end_lsn:{lsn:13645415341}}]) [2024-10-16 14:57:38.635145] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=109593694, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858576974, last_flushed_end_lsn:{lsn:13645415341}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858577202, last_flushed_end_lsn:{lsn:13645415341}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061858577236, last_flushed_end_lsn:{lsn:13645415341}}]) [2024-10-16 14:57:38.637639] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=41] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:38.641114] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9AC-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061858640407000, start_id=1729061858640407000, end_id=1729061858640407001) [2024-10-16 14:57:38.641121] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9AE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061858640407000, start_id=1729061858640407000, end_id=1729061858640407001) [2024-10-16 14:57:38.641331] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9B0-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061858640407000, start_id=1729061858640407000, end_id=1729061858640407001) [2024-10-16 14:57:38.646623] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488C-0-0] [lt=10] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061858614351, keepalive_ts:-1}) [2024-10-16 14:57:38.646649] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488C-0-0] [lt=25] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061858614351, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:38.646759] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D94-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:38.646780] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D94-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:38.646790] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53D94-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:38.658173] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.658421] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:38.658435] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=11] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:38.658441] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:38.658448] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=7] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:38.660654] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=104, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}]) [2024-10-16 14:57:38.660791] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=135] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:38.664491] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3ACFF9-0-0] [lt=22] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.738636016845703125e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=869, Throttle Trigger(MB)=521, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.115848223054147326e+01) [2024-10-16 14:57:38.664656] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1729061858663855000, start_id=1729061858663855000, end_id=1729061858663855001) [2024-10-16 14:57:38.666989] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:38.672635] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C13-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:38.676610] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:38.676623] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:57:38.676626] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:38.676645] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=2] get_max_applied_scn(scn={val:1729061858543578000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493463347}, palf_committed_end_scn_:{val:1729061858543578000, v:0}, last_check_scn_:{val:1729061858543578000, v:0}, max_applied_cb_scn_:{val:1729061858543578000, v:0}}) [2024-10-16 14:57:38.676661] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:38.677281] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=42] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061856042513000, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870721265000, latest_log_ts:{val:1729061854137257001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:38.677391] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=37] get_max_applied_scn(scn={val:1729061858543578000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645415341}, palf_committed_end_scn_:{val:1729061858543578000, v:0}, last_check_scn_:{val:1729061858543578000, v:0}, max_applied_cb_scn_:{val:1729061858543578000, v:0}}) [2024-10-16 14:57:38.677406] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:38.677797] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=23] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:38.678088] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061856042513000, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:38.678571] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061858677565000, start_id=1729061858677565000, end_id=1729061858677565001) [2024-10-16 14:57:38.678735] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=33] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061856042513000, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:38.678767] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] get_min_unreplayed_log_info(lsn={lsn:14352370937}, scn={val:1729061858640407001, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061858647371, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352370937}, next_to_submit_scn_:{val:1729061858640407001, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352370815}, end_lsn:{lsn:14352370937}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061858640407000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061858640407000, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:389307338, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061858640407000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656922, miss_cnt:17, cache_read_size:8050161158, hit_ratio:"9.999997286812878361e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:38.679389] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=31] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061856042513000, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:38.681233] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D78-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510326549626, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410593, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:38.682309] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D7C-0-0] [lt=14][errcode=-4201] hashtable not init [2024-10-16 14:57:38.682330] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D7C-0-0] [lt=20][errcode=-4016] get cursor info failed(cursor_id=1113, get_sessid()=3221565766) [2024-10-16 14:57:38.682531] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061858682169000, start_id=1729061858682169000, end_id=1729061858682169001) [2024-10-16 14:57:38.686995] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F67-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497386, log_task={header:{begin_lsn:{lsn:207589543796}, end_lsn:{lsn:207589543918}, log_id:218497386, min_scn:{val:1729061858140521004, v:0}, max_scn:{val:1729061858140521004, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589543674}, prev_proposal_id:81, committed_end_lsn:{lsn:207589543796}, data_checksum:940009244, accum_checksum:2521477716, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061858586863, freeze_ts:1729061858586863, submit_ts:1729061858586866, flushed_ts:1729061858587904, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:1038}, fs_cb_begin_ts=1729061858686989, log_life_time=100126) [2024-10-16 14:57:38.687178] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:38.690095] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D3E-0-0] [lt=13] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061858681318, keepalive_ts:-1}) [2024-10-16 14:57:38.690142] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D3E-0-0] [lt=45] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061858681318, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:38.690161] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D3F-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061858681360, keepalive_ts:-1}) [2024-10-16 14:57:38.690167] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D3F-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061858681360, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:38.698526] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:38.698980] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:38.706197] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.707139] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15DC-0-0] [lt=17] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:38.708374] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:38.708661] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061858708486000, start_id=1729061858708486000, end_id=1729061858708486001) [2024-10-16 14:57:38.710573] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=12] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:38.724207] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=22] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:38.726891] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=3548098, this={dio_aligned_buf:{buf_write_offset:1780, buf_padding_size:2316, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80962687, truncate_used_ts:29163496}, log_block_size:67108864, dir_fd:148, io_fd:171}, aligned_buf_len=4096, aligned_block_offset=3547136, buf_len=818, total_write_size_=102521, total_write_size_after_dio_=811008, ob_pwrite_used_ts=109258, count=173, each_pwrite_cost=631) [2024-10-16 14:57:38.733103] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061858732836000, start_id=1729061858732836000, end_id=1729061858732836001) [2024-10-16 14:57:38.741338] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=26] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:38.756281] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=2] gc stale ls task succ [2024-10-16 14:57:38.756305] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:38.756313] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:38.763499] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:38.763829] INFO [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:194) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] get wrs ts(ls_id={id:1}, delta=99446, log_service_ts=1729061858663855, min_tx_service_ts=4611686018427387, timestamp={val:1729061858663855000, v:0}) [2024-10-16 14:57:38.763846] INFO [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:214) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=0, Near_To_GTS_Cnt=464, Other_Error_Cnt=0, Submit_Succ_Cnt=336, last_scn="{val:1729061855555231001, v:0}", last_lsn={lsn:66390100146}, last_gts={val:1729061858682169000, v:0}, min_start_scn="{val:0, v:0}", min_start_status=1, sys_ls_end_scn={val:0, v:0}) [2024-10-16 14:57:38.765226] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.771037] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:38.781515] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:38.788161] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:38.788561] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.788771] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.788804] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.788818] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.788936] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.788948] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.788970] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.788983] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.788999] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.789013] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.789394] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:38.797299] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061858797229000, start_id=1729061858797229000, end_id=1729061858797229001) [2024-10-16 14:57:38.801111] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6A-0-0] [lt=102] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:38.801139] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6A-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061858641478001, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061858641478001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:38.802102] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D6A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061858801459000, start_id=1729061858801459000, end_id=1729061858801459001) [2024-10-16 14:57:38.804702] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6A-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061858804243000, start_id=1729061858804243000, end_id=1729061858804243001) [2024-10-16 14:57:38.804773] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6A-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:38.804794] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6A-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:38.806256] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.807632] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C428E-0-0] [lt=25] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:38.807645] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C428E-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1000, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:38.808803] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061858807430000, start_id=1729061858807430000, end_id=1729061858807430001) [2024-10-16 14:57:38.825086] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1001}, {id:1002}]) [2024-10-16 14:57:38.832025] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=18] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:38.836975] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=13] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:38.837538] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:38.837588] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=17] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061854425066000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:38.847255] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=46] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=30, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:38.847270] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:38.852715] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4314, mem_used=8318976) [2024-10-16 14:57:38.864713] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061858864166000, start_id=1729061858864166000, end_id=1729061858864166001) [2024-10-16 14:57:38.865303] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.872988] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:38.876805] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:38.876819] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:38.883019] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=43] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:38.883623] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:38.883670] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061851830492000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:38.887063] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F6B-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:38.899063] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:38.901198] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=14] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:38.906256] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:38.906273] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27208900, tenant_id=1004) [2024-10-16 14:57:38.906282] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:38.906320] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.907165] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:38.907178] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:38.908965] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061858908464000, start_id=1729061858908464000, end_id=1729061858908464001) [2024-10-16 14:57:38.909347] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=900, avg_log_submit_to_slide_time=1112) [2024-10-16 14:57:38.919110] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBDC-0-0] [lt=13] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061858886895, keepalive_ts:-1}) [2024-10-16 14:57:38.919129] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBDC-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061858886895, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:38.921906] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:38.933304] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] submit log success(service_type_=0, last_id=1729061875590307000, limited_id=1729061875590307000) [2024-10-16 14:57:38.933318] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] submit log with lock success(service_type_=0, last_id=1729061875590307000, limited_id=1729061875590307000) [2024-10-16 14:57:38.933342] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061858932674000, start_id=1729061858932674000, end_id=1729061858932674001) [2024-10-16 14:57:38.933830] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] update id meta(ret=0, service_type=0, limited_id=1729061875590307000, latest_scn={val:1729061858932674001, v:0}, *this={tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061875590307000, latest_log_ts:{val:1729061858932674001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:38.933870] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=39] handle submit callback(service_type_=0, success=true, used_ts=567, log_ts={val:1729061858932674001, v:0}, limited_id=1729061875590307000) [2024-10-16 14:57:38.933877] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:38.933882] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:38.934008] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061858933708000, start_id=1729061858933708000, end_id=1729061858933708001) [2024-10-16 14:57:38.937751] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115173122, max_lsn:14352371181, max_scn:{val:1729061858640407002, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6301879685}, reuse_lsn:{lsn:14352371181}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14310428141}}, last_submit_lsn:{lsn:14352371059}, last_submit_end_lsn:{lsn:14352371181}, last_submit_log_id:115173122, last_submit_log_pid:17, max_flushed_lsn:{lsn:14352371059}, max_flushed_end_lsn:{lsn:14352371181}, max_flushed_log_pid:17, committed_end_lsn:{lsn:14352371181}, last_slide_log_id:115173122, last_slide_scn:{val:1729061858640407002, v:0}, last_slide_lsn:{lsn:14352371059}, last_slide_end_lsn:{lsn:14352371181}, last_slide_log_pid:17, last_slide_log_accum_checksum:2877333325, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43d32530}, [StateMgr]={this:0x2b0e43d36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.9:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6302436797}, scan_disk_log_finished:true, last_check_start_id:115173123, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433837169, palf_role_change_cb:0x2b0e43d3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, reconfig_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, persistent_config_version:{proposal_id:17, config_seq:26}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061856832346, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061856832346, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43d33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1728301961314447004, v:0}, min_block_id:199, min_block_min_scn:{val:1728247158357854003, v:0}, base_lsn_for_block_gc:{lsn:14293315584}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:14293315584}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}}, log_meta_storage:{log_tail:{lsn:1064960}, readable_log_tail:{lsn:1064960}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14360420352}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/meta", dir_fd:192, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66039808, block_header_serialize_buf_:0x2b0e43d389f8, flashback_version:0}, log_storage:{log_tail:{lsn:14352371181}, readable_log_tail:{lsn:14352371181}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14293315584}, min_scn:{val:1729013741169122001, v:0}, curr_block_id:213, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/log", dir_fd:207, min_block_id:199, max_block_id:214, curr_writable_block_id:213}, logical_block_size_:67104768, curr_block_writable_size_:8049171, block_header_serialize_buf_:0x2b0e43d3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433836510, this:0x2b0e43d36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:16, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:15, majority_max_lsn:{lsn:6302238429}, saved_end_lsn:{lsn:6302238429}, last_submit_prepare_req_time_us:1722494433837290, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43d369b0}) [2024-10-16 14:57:38.937892] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=138] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:38.937903] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=115173123, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:38.941850] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2773, mem_used=6239232) [2024-10-16 14:57:38.945708] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=9] ====== tenant freeze timer task ====== [2024-10-16 14:57:38.949399] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:38.962723] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE773B-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:38.965364] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:38.973330] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90EB-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:38.973343] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90EB-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20240, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:38.973348] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90EB-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=390, avg_log_submit_to_slide_time=20234) [2024-10-16 14:57:38.973343] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:38.977054] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=966, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:38.977065] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=314, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:38.977231] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1C0-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=110, avg_log_gen_to_submit_time=140, avg_log_submit_to_flush_time=648, avg_log_submit_to_slide_time=1062) [2024-10-16 14:57:38.983831] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 447.50; ) [2024-10-16 14:57:38.983848] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:38.983890] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 172.25; ) [2024-10-16 14:57:38.983896] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:38.983904] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:38.987193] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F6D-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:38.987388] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=200, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:38.987406] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=89, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:38.987443] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:38.988458] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:38.991216] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Cache wash clean map node details(ret=0, clean_node_count=47, clean_time=0, clean_start_pos=943710, clean_num=62914) [2024-10-16 14:57:38.992783] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=8] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:38.992804] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=211) [2024-10-16 14:57:38.997436] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061858996928000, start_id=1729061858996928000, end_id=1729061858996928001) [2024-10-16 14:57:38.997531] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061858997518, bytes: 4446812902959, bw: 0.396793 MB/s, add_ts: 1000963, add_bytes: 416468 [2024-10-16 14:57:39.006381] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.007217] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:39.007230] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:39.007544] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=341, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:39.007554] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=478, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:39.007570] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:39.008913] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061859007622000, start_id=1729061859007622000, end_id=1729061859007622001) [2024-10-16 14:57:39.021248] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=2] ====== checkpoint timer task ====== [2024-10-16 14:57:39.021992] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=13] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.022021] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.022043] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:39.022181] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061855638334005, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870098662000, latest_log_ts:{val:1729061853539570001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:39.022239] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=54] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061855638334005, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870098662000, latest_log_ts:{val:1729061853539570001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:39.022271] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:39.022281] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061853539570001, v:0}) [2024-10-16 14:57:39.022287] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:39.022301] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:39.022314] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:39.022325] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:39.023089] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.023115] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.023133] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:39.023177] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061855638334005, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:39.023230] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=50] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061855638334005, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:39.023277] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:39.023288] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:39.023298] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:39.023305] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:39.024032] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.024054] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=22] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.024069] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:39.024107] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061855638334005, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:39.024151] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=42] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061855638334005, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:39.024178] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:39.024188] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:39.024197] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:39.024207] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:39.024976] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.025004] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.025018] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:39.025054] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061855638334005, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:39.025087] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061855638334005, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:39.025115] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:39.025126] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:39.025139] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:39.025149] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:39.025153] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:39.027698] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=10] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:39.032161] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=27] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3978, avg_group_log_size=397) [2024-10-16 14:57:39.032395] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=196, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:39.032412] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=33, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:39.032425] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:39.032669] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1006-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=251, avg_log_submit_to_slide_time=485) [2024-10-16 14:57:39.042121] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:39.042145] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=22] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:39.042157] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291915146, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:39.042506] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061859042505, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007613, add_bytes: 0 [2024-10-16 14:57:39.045874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061859044902000, start_id=1729061859044902000, end_id=1729061859044902001) [2024-10-16 14:57:39.045896] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:39.045925] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061859045920], gts:1729061859044902000, latest_srr:[mts=1729061859045920]}) [2024-10-16 14:57:39.046146] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:39.064793] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061859064140000, start_id=1729061859064140000, end_id=1729061859064140001) [2024-10-16 14:57:39.065433] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.076921] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:39.076940] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:39.077148] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1758) [2024-10-16 14:57:39.077233] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:57:39.077244] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1636) [2024-10-16 14:57:39.078661] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=20, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:57:39.078673] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=1833, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:57:39.078677] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:57:39.078681] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=-1278, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:57:39.078684] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=576, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:57:39.087185] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F6F-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:39.087761] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=35] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:39.087791] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598839, miss_cnt:9, cache_read_size:149777908969, hit_ratio:"9.999999345924756033e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:39.088158] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:39.088170] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151993940, tenant_id=1) [2024-10-16 14:57:39.088185] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:39.099137] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:39.106444] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.109027] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061859106212000, start_id=1729061859106212000, end_id=1729061859106212001) [2024-10-16 14:57:39.112618] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F40B4-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:39.113491] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:39.113502] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:39.127671] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=25] gc stale ls task succ [2024-10-16 14:57:39.129880] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=2010, mem_used=4159488) [2024-10-16 14:57:39.133320] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061859133086000, start_id=1729061859133086000, end_id=1729061859133086001) [2024-10-16 14:57:39.133385] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=25, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:39.133398] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=10185, avg_group_log_size=636) [2024-10-16 14:57:39.133406] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10734) [2024-10-16 14:57:39.133423] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:39.133628] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=232, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:39.133642] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10185) [2024-10-16 14:57:39.133652] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=28, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:39.133675] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10734) [2024-10-16 14:57:39.133687] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:39.133906] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94A4-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:39.133913] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94A4-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=711, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:39.133916] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94A4-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=44, avg_log_gen_to_submit_time=77, avg_log_submit_to_flush_time=285, avg_log_submit_to_slide_time=634) [2024-10-16 14:57:39.135598] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87D-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:39.135625] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87D-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061858640407004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061858640407004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:39.136634] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA87D-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061859136259000, start_id=1729061859136259000, end_id=1729061859136259001) [2024-10-16 14:57:39.139350] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601394, offset=9623470}, end_cursor:ObLogCursor{file_id=192, log_id=47601395, offset=9623754}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cc0050, len:2130, log_data_len:284, seq:47601394, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:39.139533] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA87D-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061859139431000, start_id=1729061859139431000, end_id=1729061859139431001) [2024-10-16 14:57:39.141134] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A1-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061859140517000, start_id=1729061859140517000, end_id=1729061859140517001) [2024-10-16 14:57:39.141411] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:39.141580] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9B1-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061859140517000, start_id=1729061859140517000, end_id=1729061859140517001) [2024-10-16 14:57:39.141802] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9B3-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061859141747000, start_id=1729061859141747000, end_id=1729061859141747001) [2024-10-16 14:57:39.141935] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9B5-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061859141747000, start_id=1729061859141747000, end_id=1729061859141747001) [2024-10-16 14:57:39.142002] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B9B4-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061859141747000, start_id=1729061859141747000, end_id=1729061859141747001) [2024-10-16 14:57:39.142081] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601395, offset=9623754}, end_cursor:ObLogCursor{file_id=192, log_id=47601396, offset=9624028}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d74050, len:1846, log_data_len:274, seq:47601395, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:39.144125] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601396, offset=9624028}, end_cursor:ObLogCursor{file_id=192, log_id=47601397, offset=9624302}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c20050, len:1572, log_data_len:274, seq:47601396, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:39.144296] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601397, offset=9624302}, end_cursor:ObLogCursor{file_id=192, log_id=47601398, offset=9624576}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e78050, len:1298, log_data_len:274, seq:47601397, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:39.144334] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561251, miss_cnt:1, cache_read_size:152590674740, hit_ratio:"9.999999928346874345e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:39.147363] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90EE-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:39.147376] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90EE-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10160, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:39.147375] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:39.147380] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B90EE-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=304, avg_log_submit_to_slide_time=10154) [2024-10-16 14:57:39.147389] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656927, miss_cnt:17, cache_read_size:8050161768, hit_ratio:"9.999997286813094854e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:39.152081] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C0FF8-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061859152007000, start_id=1729061859152007000, end_id=1729061859152007001) [2024-10-16 14:57:39.156766] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257742, offset=39027386}, end_cursor:ObLogCursor{file_id=197, log_id=49257743, offset=39027669}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ce0050, len:3398, log_data_len:283, seq:49257742, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:39.158856] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB36C-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061859158543000, start_id=1729061859158543000, end_id=1729061859158543001) [2024-10-16 14:57:39.159730] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257743, offset=39027669}, end_cursor:ObLogCursor{file_id=197, log_id=49257744, offset=39027951}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d34050, len:3115, log_data_len:282, seq:49257743, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:39.161732] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257744, offset=39027951}, end_cursor:ObLogCursor{file_id=197, log_id=49257745, offset=39028233}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72a6050, len:2833, log_data_len:282, seq:49257744, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:39.161996] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257745, offset=39028233}, end_cursor:ObLogCursor{file_id=197, log_id=49257746, offset=39028515}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2db4050, len:2551, log_data_len:282, seq:49257745, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:39.162046] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650188, miss_cnt:32, cache_read_size:8050414162, hit_ratio:"9.999994892276515834e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:39.165491] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.172843] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C1D-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:39.173014] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=185, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:39.173031] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=73, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:39.173056] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:39.184238] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5029-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:39.184259] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5029-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:39.191466] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:39.197307] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D7E-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510327065701, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410594, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:39.197556] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061859196587000, start_id=1729061859196587000, end_id=1729061859196587001) [2024-10-16 14:57:39.206468] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.208109] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:39.232283] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=37] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:39.232303] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:39.236613] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:39.237320] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.237353] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=33] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.237375] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:39.237515] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061857043725000, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870721265000, latest_log_ts:{val:1729061854137257001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:39.237571] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=53] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:39.237592] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:39.237615] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=21] get rec log scn(service_type_=0, rec_log_ts={val:1729061854137257001, v:0}) [2024-10-16 14:57:39.237618] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.237626] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:39.237641] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:39.237651] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:39.238369] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.238386] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.238401] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:39.238474] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061857043725000, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:39.238510] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=36] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061857043725000, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:39.238538] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:39.238547] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:39.238557] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:39.238566] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:39.239334] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.239358] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=24] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.239376] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:39.239464] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061857043725000, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:39.239501] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=35] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061857043725000, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:39.239532] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:39.239539] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:39.239547] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:39.239553] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:39.240275] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.240291] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:39.240306] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:39.240390] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061857043725000, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:39.240421] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061857043725000, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:39.240448] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:39.240454] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:39.240464] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:39.240473] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:39.240476] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:39.244964] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E0B-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061859244922000, start_id=1729061859244922000, end_id=1729061859244922001) [2024-10-16 14:57:39.247185] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=190, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:39.247204] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:39.247214] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=85, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:39.247241] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:39.262859] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7741-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:39.263351] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5379-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:39.263362] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5379-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=619, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:39.264882] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061859263985000, start_id=1729061859263985000, end_id=1729061859263985001) [2024-10-16 14:57:39.265551] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.267498] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21657][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE889-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510327135892, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410595, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:39.267754] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061859064140000, v:0}, new_max_version={val:1729061859064140000, v:0}, last_min_version={val:1729061858864166000, v:0}, last_max_version={val:1729061858864166000, v:0}, total_time_us=3679, delta_us=203613) [2024-10-16 14:57:39.267773] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061859064140000, v:0}, min_version={val:1729061859064140000, v:0}, max_version={val:1729061859064140000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:39.267788] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:39.274963] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1E00-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061859274096000, start_id=1729061859274096000, end_id=1729061859274096001) [2024-10-16 14:57:39.276996] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1636, avg_group_log_size=163) [2024-10-16 14:57:39.277019] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1636) [2024-10-16 14:57:39.277051] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1089, avg_group_log_size=121) [2024-10-16 14:57:39.277061] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1089) [2024-10-16 14:57:39.277341] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:39.284951] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1035-0-0] [lt=9] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:39.285643] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=11] dump tenant info(tenant={id:1, tenant_meta:{unit:{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, super_block:{tenant_id:1, replay_start_point:ObLogCursor{file_id=12, log_id=1497048, offset=40358755}, ls_meta_entry:[2738](ver=0,mode=0,seq=7639021), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"3.000000000000000000e+00", unit_max_cpu:"3.000000000000000000e+00", total_worker_cnt:54, min_worker_cnt:32, max_worker_cnt:150, stopped:0, worker_us:1096709877516, recv_hp_rpc_cnt:5717684, recv_np_rpc_cnt:154801474, recv_lp_rpc_cnt:0, recv_mysql_cnt:62154010, recv_task_cnt:6664582, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:19623865714, workers:32, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=7 cnt[3]=0 cnt[4]=0 cnt[5]=228764685 cnt[6]=10623 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541001, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143148686}{group_id:10, queue_size:0, recv_req_cnt:19682065, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143048442}{group_id:20, queue_size:0, recv_req_cnt:401, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015260372185}{group_id:15, queue_size:0, recv_req_cnt:5821, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723554329}{group_id:5, queue_size:0, recv_req_cnt:35746111, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494262337428}{group_id:19, queue_size:0, recv_req_cnt:7245680, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061834968674}{group_id:9, queue_size:0, recv_req_cnt:15758078, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143222379}{group_id:1, queue_size:0, recv_req_cnt:153510935, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061854965187}, rpc_stat_info: pcode=0x51c:cnt=332 pcode=0x1502:cnt=208 pcode=0x717:cnt=186 pcode=0x150b:cnt=60 pcode=0x51f:cnt=60, token_change_ts:1722494237022007, tenant_role:1}) [2024-10-16 14:57:39.286151] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=507] dump throttled time info(id_=1, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 10, group: OBCG_LOC_CACHE, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 9, group: OBCG_DETECT_RS, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1, tenant_throttled_time: 0;) [2024-10-16 14:57:39.286166] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] dump tenant info(tenant={id:508, tenant_meta:{unit:{tenant_id:508, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140910089, is_removed:false}, super_block:{tenant_id:508, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:15573117056, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:51691750, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6536480822, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:15, queue_size:0, recv_req_cnt:53884, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060722522359}, rpc_stat_info: pcode=0x5c0:cnt=220, token_change_ts:1722494140914497, tenant_role:0}) [2024-10-16 14:57:39.286586] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=420] dump throttled time info(id_=508, throttled_time_log=group_id: 15, group: OBCG_WR, throttled_time: 0;tenant_id: 508, tenant_throttled_time: 0;) [2024-10-16 14:57:39.286591] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=6] dump tenant info(tenant={id:509, tenant_meta:{unit:{tenant_id:509, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140901337, is_removed:false}, super_block:{tenant_id:509, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:12840482490, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6441819353, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1722494140916657, tenant_role:0}) [2024-10-16 14:57:39.287015] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=422] dump throttled time info(id_=509, throttled_time_log=tenant_id: 509, tenant_throttled_time: 0;) [2024-10-16 14:57:39.287019] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=5] dump tenant info(tenant={id:1001, tenant_meta:{unit:{tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1001, replay_start_point:ObLogCursor{file_id=11, log_id=1059563, offset=59267180}, ls_meta_entry:[7452](ver=0,mode=0,seq=7431948), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:468365681426, recv_hp_rpc_cnt:13589274, recv_np_rpc_cnt:126529431, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2784371638, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=9 cnt[3]=0 cnt[4]=0 cnt[5]=167011395 cnt[6]=3244 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541000, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494145075365}{group_id:20, queue_size:0, recv_req_cnt:306, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015263681355}{group_id:5, queue_size:0, recv_req_cnt:58923522, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726724812431820}{group_id:19, queue_size:0, recv_req_cnt:7662190, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061844973518}{group_id:1, queue_size:0, recv_req_cnt:227493101, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061856295889}, rpc_stat_info: pcode=0x1502:cnt=344 pcode=0x51c:cnt=220 pcode=0x717:cnt=181 pcode=0x710:cnt=91 pcode=0x150b:cnt=60, token_change_ts:1722494144838773, tenant_role:1}) [2024-10-16 14:57:39.287504] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=484] dump throttled time info(id_=1001, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1001, tenant_throttled_time: 0;) [2024-10-16 14:57:39.287516] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] dump tenant info(tenant={id:1002, tenant_meta:{unit:{tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1002, replay_start_point:ObLogCursor{file_id=191, log_id=47529733, offset=57901112}, ls_meta_entry:[10485](ver=0,mode=0,seq=7657455), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:998834075827, recv_hp_rpc_cnt:198478849, recv_np_rpc_cnt:379058613, recv_lp_rpc_cnt:0, recv_mysql_cnt:280947583, recv_task_cnt:146658, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:4103767198, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=673 cnt[3]=0 cnt[4]=0 cnt[5]=64428880 cnt[6]=226112 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131354853, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1728209719937557}{group_id:20, queue_size:0, recv_req_cnt:1673, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264869418}{group_id:15, queue_size:0, recv_req_cnt:5733, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723564477}{group_id:5, queue_size:0, recv_req_cnt:401894061, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1727167492226332}{group_id:19, queue_size:0, recv_req_cnt:1399885, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061856166239}{group_id:1, queue_size:0, recv_req_cnt:977159734, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061846446256}, rpc_stat_info: pcode=0x1502:cnt=382 pcode=0x1501:cnt=202 pcode=0x1516:cnt=197 pcode=0x717:cnt=181 pcode=0x150b:cnt=120, token_change_ts:1729004298741539, tenant_role:1}) [2024-10-16 14:57:39.287987] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=469] dump throttled time info(id_=1002, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1002, tenant_throttled_time: 0;) [2024-10-16 14:57:39.288000] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] dump tenant info(tenant={id:1003, tenant_meta:{unit:{tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1003, replay_start_point:ObLogCursor{file_id=11, log_id=1067656, offset=59494798}, ls_meta_entry:[3704](ver=0,mode=0,seq=7380334), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:503356597040, recv_hp_rpc_cnt:13609666, recv_np_rpc_cnt:126132142, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2783968422, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=4 cnt[3]=0 cnt[4]=0 cnt[5]=196317752 cnt[6]=6245 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52540965, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722517512781421}{group_id:20, queue_size:0, recv_req_cnt:308, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015261556381}{group_id:5, queue_size:0, recv_req_cnt:63971747, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726083164255294}{group_id:19, queue_size:0, recv_req_cnt:7660889, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061802906653}{group_id:1, queue_size:0, recv_req_cnt:228854150, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061850274854}, rpc_stat_info: pcode=0x1502:cnt=336 pcode=0x51c:cnt=256 pcode=0x717:cnt=187 pcode=0x710:cnt=100 pcode=0x150b:cnt=60, token_change_ts:1722494150252707, tenant_role:1}) [2024-10-16 14:57:39.288463] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=461] dump throttled time info(id_=1003, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1003, tenant_throttled_time: 0;) [2024-10-16 14:57:39.288470] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=7] dump tenant info(tenant={id:1004, tenant_meta:{unit:{tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1004, replay_start_point:ObLogCursor{file_id=197, log_id=49173916, offset=15367427}, ls_meta_entry:[12150](ver=0,mode=0,seq=7655944), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:603326291220, recv_hp_rpc_cnt:12341764, recv_np_rpc_cnt:113755426, recv_lp_rpc_cnt:0, recv_mysql_cnt:11227, recv_task_cnt:1052, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2741353030, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=99 cnt[3]=0 cnt[4]=0 cnt[5]=72426371 cnt[6]=155203 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131354711, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494154950936}{group_id:20, queue_size:0, recv_req_cnt:798, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264132685}{group_id:15, queue_size:0, recv_req_cnt:5423, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723614999}{group_id:21, queue_size:0, recv_req_cnt:15, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269726417109}{group_id:5, queue_size:0, recv_req_cnt:31029587, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1724705543331737}{group_id:19, queue_size:0, recv_req_cnt:1325445, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061845553743}{group_id:16, queue_size:0, recv_req_cnt:18, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269721623294}{group_id:1, queue_size:0, recv_req_cnt:509699432, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061855363770}, rpc_stat_info: pcode=0x1502:cnt=394 pcode=0x1501:cnt=201 pcode=0x1516:cnt=197 pcode=0x717:cnt=184 pcode=0x150b:cnt=120, token_change_ts:1725266206494273, tenant_role:1}) [2024-10-16 14:57:39.288954] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=482] dump throttled time info(id_=1004, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 21, group: OBCG_UNLOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 16, group: OBCG_TRANSFER, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1004, tenant_throttled_time: 0;) [2024-10-16 14:57:39.293100] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480D-0-0] [lt=10] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061859260851, keepalive_ts:-1}) [2024-10-16 14:57:39.293119] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480D-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061859260851, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:39.299215] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:39.302972] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:39.303824] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985D2-0-0] [lt=19] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=195) [2024-10-16 14:57:39.303954] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061859303874000, start_id=1729061859303874000, end_id=1729061859303874001) [2024-10-16 14:57:39.304723] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=42, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:57:39.304743] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=20] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=841, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:57:39.304747] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:57:39.304751] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=-524, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:57:39.304754] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=358, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:57:39.307410] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:39.307427] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:39.309528] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.310390] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061859308772000, start_id=1729061859308772000, end_id=1729061859308772001) [2024-10-16 14:57:39.312078] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061859311752000, start_id=1729061859311752000, end_id=1729061859311752001) [2024-10-16 14:57:39.317053] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=22] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6866, mem_used=12478464) [2024-10-16 14:57:39.319975] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B22-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061859318955000, start_id=1729061859318955000, end_id=1729061859318955001) [2024-10-16 14:57:39.328904] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:39.333569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061859333352000, start_id=1729061859333352000, end_id=1729061859333352001) [2024-10-16 14:57:39.363506] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:39.363556] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=26] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:39.363579] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:39.364702] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=15][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:39.365186] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789DB-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061859363570000, start_id=1729061859363570000, end_id=1729061859363570001) [2024-10-16 14:57:39.365593] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.367411] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAC-0-0] [lt=11] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:39.367441] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAC-0-0] [lt=24] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=30) [2024-10-16 14:57:39.373030] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C21-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:39.382591] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=14] blacklist_loop exec finished(cost_time=15, is_enabled=true, send_cnt=0) [2024-10-16 14:57:39.390052] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F75-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497393, log_task={header:{begin_lsn:{lsn:207589544650}, end_lsn:{lsn:207589544772}, log_id:218497393, min_scn:{val:1729061859141747001, v:0}, max_scn:{val:1729061859141747001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589544528}, prev_proposal_id:81, committed_end_lsn:{lsn:207589544650}, data_checksum:940009244, accum_checksum:4027873333, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061859287325, freeze_ts:1729061859287325, submit_ts:1729061859287331, flushed_ts:1729061859289203, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:1872}, fs_cb_begin_ts=1729061859390045, log_life_time=102720) [2024-10-16 14:57:39.391788] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:39.394306] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:39.396298] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=10] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:39.397823] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061859397117000, start_id=1729061859397117000, end_id=1729061859397117001) [2024-10-16 14:57:39.409567] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.432351] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4524) [2024-10-16 14:57:39.432964] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BD4-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4646) [2024-10-16 14:57:39.452171] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=25] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:39.457201] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061849454122, end_time=1729061859457199, dag_status={}) [2024-10-16 14:57:39.457217] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:39.457225] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:39.457229] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:39.457233] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:39.457236] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:39.457239] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:39.457243] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:39.457246] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:39.457249] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:39.457253] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457259] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457263] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457267] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457271] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457274] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457278] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457282] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457286] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457290] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457294] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457298] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457302] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457306] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457309] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457313] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457317] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457321] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457325] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457328] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457332] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457336] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457346] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457350] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457355] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457359] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457363] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457367] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457370] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457374] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457378] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457381] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457385] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457389] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457393] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457397] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457401] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457405] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457409] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457413] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457417] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457420] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457424] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457428] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457432] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457436] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457439] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457443] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457447] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457450] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457454] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457458] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457462] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457465] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457469] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457473] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457477] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457480] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:39.457485] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:39.457489] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:39.457492] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:39.457495] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:39.457498] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:39.457500] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:39.457502] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:39.457505] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:39.457507] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:39.457509] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:39.457513] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:39.463160] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=190, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:39.463180] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:39.463460] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D537D-0-0] [lt=30] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:39.465023] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061859464342000, start_id=1729061859464342000, end_id=1729061859464342001) [2024-10-16 14:57:39.465272] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=20] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=5111) [2024-10-16 14:57:39.465510] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C100C-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:39.465520] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C100C-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=737, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:39.465706] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.465967] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21810][T1001_TntShared][T1001][Y0-0000000000000000-0-0] [lt=11] clear dtl interm result cost(us)(clear_cost=15, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:39.469723] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7D-0-0] [lt=10] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:39.469745] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7D-0-0] [lt=21] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:39.473852] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=15] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:39.474441] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=12] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:39.474491] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=16] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061858725452000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:39.476287] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8C4-0-0] [lt=7] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:39.476307] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8C4-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:39.476326] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8C4-0-0] [lt=18] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:39.477299] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1089) [2024-10-16 14:57:39.477626] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1CF-0-0] [lt=24] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:39.477636] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1CF-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=912, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:39.477640] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1CF-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=568, avg_log_submit_to_slide_time=893) [2024-10-16 14:57:39.487219] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:39.487228] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:39.487238] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:39.487235] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED3-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:39.487243] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:39.487247] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:39.487250] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:39.487252] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:39.487248] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED3-0-0] [lt=11] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:39.487255] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:39.487257] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:39.487259] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:39.487263] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:39.487781] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D537E-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:39.487791] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D537E-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20687, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:39.489886] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:39.489899] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:39.489911] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=70658172, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:39.497856] INFO [STORAGE.TRANS] refresh_elr_tenant_config_ (ob_tx_elr_util.cpp:47) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C85-0-0] [lt=16] refresh tenant config success(tenant_id=1004, *this={last_refresh_ts:1729061859497517, can_tenant_elr:true}) [2024-10-16 14:57:39.498018] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C85-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061859497517000, start_id=1729061859497517000, end_id=1729061859497517001) [2024-10-16 14:57:39.499298] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:39.507522] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:39.507541] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:39.507823] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2194) [2024-10-16 14:57:39.508026] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42A4-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:39.508039] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42A4-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=875, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:39.508075] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42A3-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=1026, avg_log_submit_to_slide_time=1209) [2024-10-16 14:57:39.509622] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.509627] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=11319, avg_group_log_size=665) [2024-10-16 14:57:39.509638] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11790) [2024-10-16 14:57:39.510596] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061859510224000, start_id=1729061859510224000, end_id=1729061859510224001) [2024-10-16 14:57:39.510747] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=397, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:39.510759] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11319) [2024-10-16 14:57:39.510769] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=108, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:39.510782] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11790) [2024-10-16 14:57:39.510821] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:39.510874] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C23-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:39.510879] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C23-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1024, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:39.510883] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C23-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=26, avg_log_gen_to_submit_time=53, avg_log_submit_to_flush_time=616, avg_log_submit_to_slide_time=970) [2024-10-16 14:57:39.512239] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061859511456000, start_id=1729061859511456000, end_id=1729061859511456001) [2024-10-16 14:57:39.522861] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:7385763025}, base_lsn={lsn:7314419712}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:39.522925] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=64] get_replay_process success(id={id:1}, replayed_log_size=71343313, unreplayed_log_size=0) [2024-10-16 14:57:39.522934] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:57923794664}, base_lsn={lsn:57911414784}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:39.522956] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=22] get_replay_process success(id={id:1001}, replayed_log_size=12379880, unreplayed_log_size=0) [2024-10-16 14:57:39.522961] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=152596839278, unreplayed_log_size=0) [2024-10-16 14:57:39.522965] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1003}, replayed_log_size=149812339759, unreplayed_log_size=0) [2024-10-16 14:57:39.522980] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=2] dump tenant replay process(tenant_id=1002, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=288479, last_replayed_log_size(MB)=288479, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:39.533697] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061859533578000, start_id=1729061859533578000, end_id=1729061859533578001) [2024-10-16 14:57:39.541479] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=904810496, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=168931328, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:39.541535] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=26] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:39.541547] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=11] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:39.541581] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=11] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=967602176, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=106139648, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:39.541630] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:39.541637] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:39.541666] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=942559232, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=131182592, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:39.541703] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:39.541711] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011862, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:39.541742] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=961036288, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=112705536, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:39.541776] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:39.541783] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:39.546272] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061859546260000, start_id=1729061859546260000, end_id=1729061859546260001) [2024-10-16 14:57:39.546300] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=506, get_gts_with_stc_cnt=78, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=76, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:39.546311] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061859546296], gts:1729061859546260000, latest_srr:[mts=1729061859546296]}) [2024-10-16 14:57:39.546480] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=525, get_gts_with_stc_cnt=6, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:39.549347] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=21] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:39.563163] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C1-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:39.563201] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C1-0-0] [lt=25] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061859141747003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061859141747003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:39.563399] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:39.564401] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9C1-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061859564372000, start_id=1729061859564372000, end_id=1729061859564372001) [2024-10-16 14:57:39.565754] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.567047] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C1-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061859566454000, start_id=1729061859566454000, end_id=1729061859566454001) [2024-10-16 14:57:39.567701] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=11] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:39.568769] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.568784] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.569195] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.569462] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.584820] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1001}, {id:1}]) [2024-10-16 14:57:39.587527] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:39.587550] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:39.592072] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:39.594456] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B2C-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061859594010000, start_id=1729061859594010000, end_id=1729061859594010001) [2024-10-16 14:57:39.595057] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=48, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:39.595068] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=1328, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:39.595073] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:39.595076] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=-998, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:39.595080] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=378, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:39.597188] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1DFF-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061859596151000, start_id=1729061859596151000, end_id=1729061859596151001) [2024-10-16 14:57:39.597868] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061859597619000, start_id=1729061859597619000, end_id=1729061859597619001) [2024-10-16 14:57:39.604256] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0D-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061859595482, keepalive_ts:-1}) [2024-10-16 14:57:39.604273] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0D-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061859595482, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:39.606952] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4E-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:39.606977] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4E-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061859141747005, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061859141747005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:39.607785] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=300, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:39.607815] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=634, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:39.607846] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:39.607988] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C4E-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061859607096000, start_id=1729061859607096000, end_id=1729061859607096001) [2024-10-16 14:57:39.608892] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=27] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=4960, mem_used=10398720) [2024-10-16 14:57:39.609681] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.610508] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C4E-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061859610298000, start_id=1729061859610298000, end_id=1729061859610298001) [2024-10-16 14:57:39.611248] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=15] BlockGCTimerTask success(ret=0, cost_time_us=6, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:39.641991] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9B8-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061859641988000, start_id=1729061859641988000, end_id=1729061859641988001) [2024-10-16 14:57:39.642320] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9B6-0-0] [lt=46] get number(ret=0, service_type_=0, range=1, base_id=1729061859641988000, start_id=1729061859641988000, end_id=1729061859641988001) [2024-10-16 14:57:39.642343] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9BA-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061859641988000, start_id=1729061859641988000, end_id=1729061859641988001) [2024-10-16 14:57:39.647396] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DA8-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:39.647438] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DA8-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:39.647451] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DA8-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:39.647468] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DA8-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173129, log_task={header:{begin_lsn:{lsn:14352371913}, end_lsn:{lsn:14352372035}, log_id:115173129, min_scn:{val:1729061859141747004, v:0}, max_scn:{val:1729061859141747004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352371791}, prev_proposal_id:17, committed_end_lsn:{lsn:14352371913}, data_checksum:940009244, accum_checksum:3438378797, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061859547174, freeze_ts:1729061859547175, submit_ts:1729061859547180, flushed_ts:1729061859547837, gen_to_freeze cost time:1, gen_to_submit cost time:6, submit_to_flush cost time:657}, fs_cb_begin_ts=1729061859647461, log_life_time=100287) [2024-10-16 14:57:39.665210] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061859665150000, start_id=1729061859665150000, end_id=1729061859665150001) [2024-10-16 14:57:39.665813] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:39.665830] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:39.665837] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.667079] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:39.673172] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C27-0-0] [lt=21] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:39.677483] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=181, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:39.677497] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=212, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:39.677515] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:39.677771] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1D5-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:39.677777] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1D5-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=983, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:39.687743] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:39.698588] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:39.699623] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=10] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:39.700339] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21525][T1_ReqMemEvict][T1][YB4A0A000407-00061E9B3BA96484-0-0] [lt=12] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.736235961914062500e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=868, Throttle Trigger(MB)=520, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.133062993976673383e+01) [2024-10-16 14:57:39.700438] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061859698618000, start_id=1729061859698618000, end_id=1729061859698618001) [2024-10-16 14:57:39.700819] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2107619, replace_num=31457) [2024-10-16 14:57:39.700836] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:39.709700] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.710688] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061859710011000, start_id=1729061859710011000, end_id=1729061859710011001) [2024-10-16 14:57:39.712309] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061859712123000, start_id=1729061859712123000, end_id=1729061859712123001) [2024-10-16 14:57:39.714652] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B1F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061859713622000, start_id=1729061859713622000, end_id=1729061859713622001) [2024-10-16 14:57:39.715382] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2742) [2024-10-16 14:57:39.715861] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:39.715881] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=212) [2024-10-16 14:57:39.724308] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:39.733851] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061859733675000, start_id=1729061859733675000, end_id=1729061859733675001) [2024-10-16 14:57:39.741113] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=42] LogLoopThread round_cost_time(us)(round_cost_time=7) [2024-10-16 14:57:39.756373] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:39.756399] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:39.756407] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:39.765826] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.769083] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061859464342000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=304736, in_cluster_service=true, cluster_version={val:1729061859464342000, v:0}, min_cluster_version={val:1729061859464342000, v:0}, max_cluster_version={val:1729061859464342000, v:0}, get_cluster_version_err=0, cluster_version_delta=304736, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65248880, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=357041793, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:39.769121] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=27] no need to update min/max version(new_version={val:1729061859464342000, v:0}, min_version={val:1729061859464342000, v:0}, max_version={val:1729061859464342000, v:0}) [2024-10-16 14:57:39.769633] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:39.771048] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:39.781571] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:39.788537] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.788541] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.788802] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.788819] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.788921] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.788925] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.788941] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.788956] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.788971] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.788985] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.789374] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:39.792379] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=45] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:39.797873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061859797682000, start_id=1729061859797682000, end_id=1729061859797682001) [2024-10-16 14:57:39.808363] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6B-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:39.808388] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6B-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061859641988002, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061859641988002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:39.809359] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D6B-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061859809299000, start_id=1729061859809299000, end_id=1729061859809299001) [2024-10-16 14:57:39.810576] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.812431] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6B-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061859811683000, start_id=1729061859811683000, end_id=1729061859811683001) [2024-10-16 14:57:39.812493] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6B-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:39.812507] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6B-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:39.821750] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=23] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}) [2024-10-16 14:57:39.822057] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:39.825189] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1003}, {id:1002}]) [2024-10-16 14:57:39.828325] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:39.828341] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:39.828353] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52400459, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:39.832111] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:39.847911] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:39.847925] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:39.852826] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4378, mem_used=8318976) [2024-10-16 14:57:39.853394] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=34] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:39.866213] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.869958] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061859869849000, start_id=1729061859869849000, end_id=1729061859869849001) [2024-10-16 14:57:39.873447] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:39.877383] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:39.877399] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:39.887660] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F7F-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:39.887836] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=34502045, this={dio_aligned_buf:{buf_write_offset:1559, buf_padding_size:2537, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111299639, truncate_used_ts:40614817}, log_block_size:67108864, dir_fd:182, io_fd:127}, aligned_buf_len=4096, aligned_block_offset=34500608, buf_len=122, total_write_size_=12294, total_write_size_after_dio_=425984, ob_pwrite_used_ts=46349, count=101, each_pwrite_cost=458) [2024-10-16 14:57:39.900909] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:39.901250] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=20] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:39.907733] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2620, avg_group_log_size=238) [2024-10-16 14:57:39.907753] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2620) [2024-10-16 14:57:39.908480] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F40CC-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:39.908492] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F40CC-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1146, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:39.910637] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.911383] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061859911097000, start_id=1729061859911097000, end_id=1729061859911097001) [2024-10-16 14:57:39.912168] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=21] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1004, min_version={val:1729061859141747004, v:0}, base_version={val:1729061859141747002, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061859546260000, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061859912159, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061859141747004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061859787615, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061859141747005, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061859814886, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:39.912931] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061859912156000, start_id=1729061859912156000, end_id=1729061859912156001) [2024-10-16 14:57:39.921987] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:39.933876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061859933324000, start_id=1729061859933324000, end_id=1729061859933324001) [2024-10-16 14:57:39.935141] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061855928746, last_first_register_time_us=1722494150998907, register_time_us=1729061855928746, curr_time_us=1729061859935139) [2024-10-16 14:57:39.935181] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061859935160, last_submit_register_req_time_us=1729061859935160, register_parent_reason=3) [2024-10-16 14:57:39.935190] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:39.941954] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2804, mem_used=6239232) [2024-10-16 14:57:39.943963] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26142][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB25CC6B9-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:57:39.963532] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE774F-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:39.966241] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:39.974743] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=7][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:39.977614] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=164, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:39.977633] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=29, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:39.977931] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE3C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=236, avg_log_submit_to_slide_time=564) [2024-10-16 14:57:39.980507] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15D8-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:39.985617] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC734-0-0] [lt=17] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:39.985637] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC734-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:39.986219] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 142.00; ) [2024-10-16 14:57:39.986236] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:39.986274] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 160.25; ) [2024-10-16 14:57:39.986282] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:39.986290] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:39.987687] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F81-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:39.987943] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=179, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:39.987959] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=201, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:39.987993] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:39.992667] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:39.997958] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061859997542000, start_id=1729061859997542000, end_id=1729061859997542001) [2024-10-16 14:57:39.998053] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061859998052, bytes: 4446813321342, bw: 0.398788 MB/s, add_ts: 1000534, add_bytes: 418383 [2024-10-16 14:57:40.007804] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:40.007819] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:40.008063] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=208, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:40.008076] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=174, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:40.008091] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:40.008237] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42B2-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=400, avg_log_submit_to_slide_time=603) [2024-10-16 14:57:40.010701] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.016190] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:40.016298] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAB-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061859933324000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061875590307000, latest_log_ts:{val:1729061858932674001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:40.016359] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAB-0-0] [lt=59] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:40.032659] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=5061, avg_group_log_size=421) [2024-10-16 14:57:40.032908] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=208, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:40.032924] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=30, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:40.032939] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:40.033194] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1012-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=471, avg_log_submit_to_slide_time=702) [2024-10-16 14:57:40.046623] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061860046134000, start_id=1729061860046134000, end_id=1729061860046134001) [2024-10-16 14:57:40.046650] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=19] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:40.046665] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061860046662], gts:1729061860046134000, latest_srr:[mts=1729061860046662]}) [2024-10-16 14:57:40.046686] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=537, get_gts_with_stc_cnt=86, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=80, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:40.050129] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061860050127, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007622, add_bytes: 0 [2024-10-16 14:57:40.066307] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.070115] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061860069392000, start_id=1729061860069392000, end_id=1729061860069392001) [2024-10-16 14:57:40.073400] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C2F-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:40.073413] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C2F-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10636, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:40.073418] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C2F-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=217, avg_log_submit_to_slide_time=10629) [2024-10-16 14:57:40.073427] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C2F-0-0] [lt=8][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178439, log_task={header:{begin_lsn:{lsn:14538486053}, end_lsn:{lsn:14538486175}, log_id:115178439, min_scn:{val:1729061859641988004, v:0}, max_scn:{val:1729061859641988004, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538485931}, prev_proposal_id:23, committed_end_lsn:{lsn:14538486053}, data_checksum:1715074492, accum_checksum:318792550, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061859973344, freeze_ts:1729061859973344, submit_ts:1729061859973348, flushed_ts:1729061859973556, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:208}, fs_cb_begin_ts=1729061860073393, log_life_time=100049) [2024-10-16 14:57:40.073445] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:40.077446] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:40.077462] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:40.085013] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] worker thread created(tenant_->id()=1004, group_id_=1, token=1) [2024-10-16 14:57:40.084876] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107474][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=193) [2024-10-16 14:57:40.085061] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107474][][T1004][Y0-0000000000000000-0-0] [lt=178] Init thread local success [2024-10-16 14:57:40.085087] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1E1-0-0] [lt=4] init point thread id with(&point=0x55e3435ca300, idx_=978, point=[thread id=107474, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107474) [2024-10-16 14:57:40.085103] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1E1-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2192) [2024-10-16 14:57:40.085153] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1E2-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:40.085009] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107475][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=193) [2024-10-16 14:57:40.085277] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107475][][T1004][Y0-0000000000000000-0-0] [lt=266] Init thread local success [2024-10-16 14:57:40.085686] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D5-0-0] [lt=5] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061860074655, keepalive_ts:-1}) [2024-10-16 14:57:40.085702] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D5-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061860074655, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:40.087758] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F83-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:40.088096] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] ====== checkpoint timer task ====== [2024-10-16 14:57:40.088853] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=21] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:40.088880] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:40.088901] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:40.089007] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061867008855000, latest_log_ts:{val:1729061850421032001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:40.089052] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=43] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:40.089065] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:57:40.089075] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061850421032001, v:0}) [2024-10-16 14:57:40.089082] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:40.089085] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:40.089096] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:40.089109] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:57:40.089119] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:40.089125] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:40.089209] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:40.089224] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598849, miss_cnt:9, cache_read_size:149777910180, hit_ratio:"9.999999345924803773e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:40.100976] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:40.111537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061860110525000, start_id=1729061860110525000, end_id=1729061860110525001) [2024-10-16 14:57:40.111638] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.113118] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061860110525000, start_id=1729061860110525001, end_id=1729061860110525002) [2024-10-16 14:57:40.114982] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:40.114995] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:40.126414] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=13] current monitor number(seq_=-1) [2024-10-16 14:57:40.127728] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:40.129969] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=2072, mem_used=4159488) [2024-10-16 14:57:40.133983] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061860133530000, start_id=1729061860133530000, end_id=1729061860133530001) [2024-10-16 14:57:40.134058] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:40.134072] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9875, avg_group_log_size=658) [2024-10-16 14:57:40.134080] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9875) [2024-10-16 14:57:40.134102] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:40.134403] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=349, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:40.134418] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9875) [2024-10-16 14:57:40.134429] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=38, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:40.134457] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9875) [2024-10-16 14:57:40.134469] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:40.134612] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94B3-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:40.134623] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94B3-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=713, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:40.134628] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94B3-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=12, avg_log_gen_to_submit_time=42, avg_log_submit_to_flush_time=444, avg_log_submit_to_slide_time=671) [2024-10-16 14:57:40.134746] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get_max_applied_scn(scn={val:1729061859997542000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618502046}, palf_committed_end_scn_:{val:1729061860133530000, v:0}, last_check_scn_:{val:1729061860133530000, v:0}, max_applied_cb_scn_:{val:1729061859997542000, v:0}}) [2024-10-16 14:57:40.134766] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:40.142313] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9BB-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061860141971000, start_id=1729061860141971000, end_id=1729061860141971001) [2024-10-16 14:57:40.142329] INFO [STORAGE.TRANS] handle_request (ob_timestamp_service.cpp:175) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9BB-0-0] [lt=14] handle gts request statistics(total_rt=2831, total_cnt=448, avg_rt=6.305122494432071178e+00, avg_cnt=4.479999999999999716e+01) [2024-10-16 14:57:40.142514] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9BF-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061860141971000, start_id=1729061860141971000, end_id=1729061860141971001) [2024-10-16 14:57:40.142595] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B9BE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061860141971000, start_id=1729061860141971000, end_id=1729061860141971001) [2024-10-16 14:57:40.142634] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B9BC-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061860141971000, start_id=1729061860141971000, end_id=1729061860141971001) [2024-10-16 14:57:40.142678] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9BD-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061860141971000, start_id=1729061860141971000, end_id=1729061860141971001) [2024-10-16 14:57:40.143144] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87E-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:40.143166] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87E-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061859641988004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061859641988004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:40.145373] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601398, offset=9624576}, end_cursor:ObLogCursor{file_id=192, log_id=47601399, offset=9624860}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c54050, len:1024, log_data_len:284, seq:47601398, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:40.146827] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21923][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA87E-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061860146197000, start_id=1729061860146197000, end_id=1729061860146197001) [2024-10-16 14:57:40.147786] INFO [LIB] stat (utility.h:1165) [107474][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9102-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:40.147797] INFO [LIB] stat (utility.h:1165) [107474][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9102-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20407, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:40.147801] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107474][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9102-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=326, avg_log_submit_to_slide_time=20400) [2024-10-16 14:57:40.147807] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22067][T1002_TeRec][T1002][YB4A0A000407-00061E9B225FA4A2-0-0] [lt=22] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=501323996, MAX_GAP=900000000000, new_sync_scn={val:1729061859044902000, v:0}, new_replayable_scn={val:1729061859044902000, v:0}, new_readable_scn={val:1729061858543578004, v:0}, old_tenant_info={tenant_id:1002, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061858043985000, v:0}, replayable_scn:{val:1729061858043985000, v:0}, standby_scn:{val:1729061858043985000, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"NOARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:57:40.147827] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:40.147838] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656937, miss_cnt:17, cache_read_size:8050162988, hit_ratio:"9.999997286813527841e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:40.148005] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A2-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061860147282000, start_id=1729061860147282000, end_id=1729061860147282001) [2024-10-16 14:57:40.148517] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:40.148954] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601399, offset=9624860}, end_cursor:ObLogCursor{file_id=192, log_id=47601400, offset=9625134}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c60050, len:740, log_data_len:274, seq:47601399, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:40.150829] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=29] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601400, offset=9625134}, end_cursor:ObLogCursor{file_id=192, log_id=47601401, offset=9625408}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c50050, len:466, log_data_len:274, seq:47601400, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:40.151095] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601401, offset=9625408}, end_cursor:ObLogCursor{file_id=192, log_id=47601402, offset=9625682}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d40050, len:4288, log_data_len:274, seq:47601401, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:40.151138] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561261, miss_cnt:1, cache_read_size:152590675960, hit_ratio:"9.999999928346878786e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:40.158633] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1009-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061860157876000, start_id=1729061860157876000, end_id=1729061860157876001) [2024-10-16 14:57:40.163495] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257746, offset=39028515}, end_cursor:ObLogCursor{file_id=197, log_id=49257747, offset=39028798}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e50050, len:2269, log_data_len:283, seq:49257746, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:40.166353] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.166677] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB36D-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061860166298000, start_id=1729061860166298000, end_id=1729061860166298001) [2024-10-16 14:57:40.167637] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257747, offset=39028798}, end_cursor:ObLogCursor{file_id=197, log_id=49257748, offset=39029080}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e30050, len:1986, log_data_len:282, seq:49257747, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:40.169627] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257748, offset=39029080}, end_cursor:ObLogCursor{file_id=197, log_id=49257749, offset=39029362}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db731e050, len:1704, log_data_len:282, seq:49257748, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:40.169814] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257749, offset=39029362}, end_cursor:ObLogCursor{file_id=197, log_id=49257750, offset=39029644}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c74050, len:1422, log_data_len:282, seq:49257749, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:40.169853] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650198, miss_cnt:32, cache_read_size:8050415382, hit_ratio:"9.999994892277330738e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:40.175924] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C31-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:40.176107] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=185, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:40.176125] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:40.176157] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:40.177679] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2192) [2024-10-16 14:57:40.185007] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502A-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:40.185024] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502A-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:40.192942] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:40.198137] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061860198021000, start_id=1729061860198021000, end_id=1729061860198021001) [2024-10-16 14:57:40.198732] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=48] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:57:40.198741] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=8] try gc retain ctx [2024-10-16 14:57:40.198745] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:40.199368] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061875590307000, latest_log_ts:{val:1729061858932674001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:40.202126] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D80-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510328070520, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410596, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:40.208900] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:40.209007] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:40.210723] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061859912156000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=298561, in_cluster_service=true, cluster_version={val:1729061859811683000, v:0}, min_cluster_version={val:1729061859811683000, v:0}, max_cluster_version={val:1729061859811683000, v:0}, get_cluster_version_err=0, cluster_version_delta=399034, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244195, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=469280520, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:40.211294] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5879-0-0] [lt=13] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:40.211396] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5879-0-0] [lt=18][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:40.211710] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.218258] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=10][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:40.232775] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:40.232800] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:40.246988] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD0A-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:40.247805] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=240, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:40.247822] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=63, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:40.247854] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:40.255510] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:40.263396] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7755-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:40.263424] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7755-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:40.263428] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7755-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=21773, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:40.266376] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.270214] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061860270037000, start_id=1729061860270037000, end_id=1729061860270037001) [2024-10-16 14:57:40.270910] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061860069392000, v:0}, new_max_version={val:1729061860069392000, v:0}, last_min_version={val:1729061859869849000, v:0}, last_max_version={val:1729061859869849000, v:0}, total_time_us=1501, delta_us=201517) [2024-10-16 14:57:40.270928] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061860069392000, v:0}, min_version={val:1729061860069392000, v:0}, max_version={val:1729061860069392000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:40.270938] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:40.277967] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB98-0-0] [lt=7] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12239, add_timestamp:1729061860277957}) [2024-10-16 14:57:40.277994] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E40-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12239, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061860277981, sql_renew_time:1729061860277981, is_mark_fail:false}]) [2024-10-16 14:57:40.278021] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E40-0-0] [lt=27] success to process renew task(task={tenant_id:1, table_id:12239, add_timestamp:1729061860277957}, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061860277981, sql_renew_time:1729061860277981, is_mark_fail:false}]) [2024-10-16 14:57:40.281798] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2070, avg_group_log_size=207) [2024-10-16 14:57:40.281813] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2070) [2024-10-16 14:57:40.281838] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:40.281843] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:40.282514] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB98-0-0] [lt=21][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:40.282885] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:40.283328] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=43, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:57:40.283339] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=1709, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:57:40.283344] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:57:40.283349] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=-668, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:57:40.283353] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB TOTAL TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=1084, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:57:40.284719] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=13] ls blacklist refresh finish(cost_time=7397) [2024-10-16 14:57:40.301003] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:40.302274] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:40.302297] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53173860, tenant_id=1003) [2024-10-16 14:57:40.302312] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=11] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:40.308791] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:40.308818] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:40.311738] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:57:40.311755] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:40.311772] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373604640, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:40.313700] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061860311815000, start_id=1729061860311815000, end_id=1729061860311815001) [2024-10-16 14:57:40.314144] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.315270] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061860314935000, start_id=1729061860314935000, end_id=1729061860314935001) [2024-10-16 14:57:40.317160] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=6929, mem_used=14558208) [2024-10-16 14:57:40.328981] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:40.333279] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=65] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061860198021000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=135252, in_cluster_service=true, cluster_version={val:1729061859933324000, v:0}, min_cluster_version={val:1729061859933324000, v:0}, max_cluster_version={val:1729061859933324000, v:0}, get_cluster_version_err=0, cluster_version_delta=399949, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244444, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=234729444, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:40.334138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061860333802000, start_id=1729061860333802000, end_id=1729061860333802001) [2024-10-16 14:57:40.347867] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:40.364066] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:40.364119] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:40.364130] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:40.366435] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.373569] INFO [LIB] stat (utility.h:1165) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C35-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:40.380854] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=10] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:40.380880] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=16] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=214) [2024-10-16 14:57:40.393227] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:40.394360] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:40.395366] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:40.395380] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:40.395397] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:40.398332] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061860397101000, start_id=1729061860397101000, end_id=1729061860397101001) [2024-10-16 14:57:40.400704] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22443][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=19][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:40.400763] WDIAG read_multi_row (ob_table_access_helper.h:238) [22443][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=15][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:40.401540] INFO delete_row (ob_table_access_helper.h:312) [22443][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1003, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061860400773) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061860400773) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:40.414246] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.432919] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4515) [2024-10-16 14:57:40.433607] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BDF-0-0] [lt=17] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4515) [2024-10-16 14:57:40.463743] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=230, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:40.463767] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=175, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:40.466498] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.469527] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] no need to update min/max version(new_version={val:1729061860069392000, v:0}, min_version={val:1729061860069392000, v:0}, max_version={val:1729061860069392000, v:0}) [2024-10-16 14:57:40.469554] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get_max_applied_scn(scn={val:1729061860270037000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390118105}, palf_committed_end_scn_:{val:1729061860270037001, v:0}, last_check_scn_:{val:1729061860270037001, v:0}, max_applied_cb_scn_:{val:1729061860270037000, v:0}}) [2024-10-16 14:57:40.469573] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:40.470515] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7E-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:40.470530] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7E-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:40.482248] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:40.482509] INFO [LIB] stat (utility.h:1165) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1ED-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:40.482519] INFO [LIB] stat (utility.h:1165) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1ED-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1333, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:40.482524] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1ED-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=318, avg_log_submit_to_slide_time=1317) [2024-10-16 14:57:40.488483] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5392-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:40.488493] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5392-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=579, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:40.501085] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:40.508849] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:40.508873] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:40.509098] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2339) [2024-10-16 14:57:40.509360] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F40DE-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=588, avg_log_submit_to_slide_time=886) [2024-10-16 14:57:40.509397] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42C1-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:40.509404] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42C1-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=618, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:40.511661] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10795, avg_group_log_size=568) [2024-10-16 14:57:40.511676] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10795) [2024-10-16 14:57:40.511838] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=249, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:40.511849] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10795) [2024-10-16 14:57:40.511858] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=93, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:40.511870] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10795) [2024-10-16 14:57:40.511882] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:40.512058] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD122-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:40.512064] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD122-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=1181, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:40.512068] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD122-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=31, avg_log_gen_to_submit_time=57, avg_log_submit_to_flush_time=362, avg_log_submit_to_slide_time=1123) [2024-10-16 14:57:40.512946] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061860046134002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=466807, in_cluster_service=true, cluster_version={val:1729061859641988003, v:0}, min_cluster_version={val:1729061859641988003, v:0}, max_cluster_version={val:1729061859641988003, v:0}, get_cluster_version_err=0, cluster_version_delta=870953, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228223, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=405904462, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:40.513898] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061860513738000, start_id=1729061860513738000, end_id=1729061860513738001) [2024-10-16 14:57:40.514329] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=34][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.515400] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061860514795000, start_id=1729061860514795000, end_id=1729061860514795001) [2024-10-16 14:57:40.532836] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:40.532855] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:40.532873] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444519406, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:40.533176] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4050) [2024-10-16 14:57:40.533405] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1017-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:40.533419] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1017-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=551, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:40.534364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061860533786000, start_id=1729061860533786000, end_id=1729061860533786001) [2024-10-16 14:57:40.547040] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] submit log success(service_type_=0, last_id=1729061877008855000, limited_id=1729061877008855000) [2024-10-16 14:57:40.547052] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] submit log with lock success(service_type_=0, last_id=1729061877008855000, limited_id=1729061877008855000) [2024-10-16 14:57:40.547056] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061860546474000, start_id=1729061860546474000, end_id=1729061860546474001) [2024-10-16 14:57:40.547237] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061860547228], gts:1729061860546474000, latest_srr:[mts=1729061860547228]}) [2024-10-16 14:57:40.547637] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061877008855000, latest_scn={val:1729061860546474001, v:0}, *this={tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061877008855000, latest_log_ts:{val:1729061860546474001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:40.547672] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=35] handle submit callback(service_type_=0, success=true, used_ts=633, log_ts={val:1729061860546474001, v:0}, limited_id=1729061877008855000) [2024-10-16 14:57:40.547679] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:40.547690] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D7D-0-0] [lt=11] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:40.563595] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE775B-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:40.563814] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:40.566560] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.568744] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.568764] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.569235] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.569258] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C2-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:40.569281] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C2-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061860141971004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061860141971004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:40.569486] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.570388] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=48] get number(ret=0, service_type_=0, range=1, base_id=1729061860569961000, start_id=1729061860569961000, end_id=1729061860569961001) [2024-10-16 14:57:40.570521] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9C2-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061860569961000, start_id=1729061860569961000, end_id=1729061860569961001) [2024-10-16 14:57:40.573358] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C2-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061860573166000, start_id=1729061860573166000, end_id=1729061860573166001) [2024-10-16 14:57:40.574050] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=11] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:40.584883] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=21] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1}, {id:1003}]) [2024-10-16 14:57:40.587501] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:40.587521] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:40.593527] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:40.597496] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061860141971003, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=455521, in_cluster_service=true, cluster_version={val:1729061859641988005, v:0}, min_cluster_version={val:1729061859641988005, v:0}, max_cluster_version={val:1729061859641988005, v:0}, get_cluster_version_err=0, cluster_version_delta=955504, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244145, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=531927331, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:40.598338] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061860597518000, start_id=1729061860597518000, end_id=1729061860597518001) [2024-10-16 14:57:40.604195] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=23] BlockGCTimerTask success(ret=0, cost_time_us=7, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:40.609000] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4973, mem_used=10398720) [2024-10-16 14:57:40.609128] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=515, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:40.609144] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=47, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:40.609178] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:40.612818] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4F-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:40.612841] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C4F-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061860141971005, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061860141971005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:40.613896] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C4F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061860613407000, start_id=1729061860613407000, end_id=1729061860613407001) [2024-10-16 14:57:40.614401] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.616345] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C4F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061860615510000, start_id=1729061860615510000, end_id=1729061860615510001) [2024-10-16 14:57:40.632956] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=26] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:57:40.632966] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=9] try gc retain ctx [2024-10-16 14:57:40.632969] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:40.633582] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061877008855000, latest_log_ts:{val:1729061860546474001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:40.642698] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9C0-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061860642319000, start_id=1729061860642319000, end_id=1729061860642319001) [2024-10-16 14:57:40.642763] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9C2-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061860642319000, start_id=1729061860642319000, end_id=1729061860642319001) [2024-10-16 14:57:40.642866] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9C4-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061860642319000, start_id=1729061860642319000, end_id=1729061860642319001) [2024-10-16 14:57:40.647990] INFO [LIB] stat (utility.h:1165) [107474][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DBC-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:40.648013] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107474][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DBC-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:40.648022] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107474][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DBC-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:40.656747] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:40.666614] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.667495] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:40.673690] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C3B-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:40.688345] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:40.698616] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:40.701163] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:40.707699] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22141][T1002_TntShared][T1002][Y0-0000000000000000-0-0] [lt=18] clear dtl interm result cost(us)(clear_cost=950, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:40.713891] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061860713616000, start_id=1729061860713616000, end_id=1729061860713616001) [2024-10-16 14:57:40.714445] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.714747] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] no need to update min/max version(new_version={val:1729061860314935000, v:0}, min_version={val:1729061860314935000, v:0}, max_version={val:1729061860314935000, v:0}) [2024-10-16 14:57:40.714772] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get_max_applied_scn(scn={val:1729061860573166000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264724588}, palf_committed_end_scn_:{val:1729061860713616000, v:0}, last_check_scn_:{val:1729061860713616000, v:0}, max_applied_cb_scn_:{val:1729061860573166000, v:0}}) [2024-10-16 14:57:40.714789] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:40.724407] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:40.734326] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF15F-0-0] [lt=10] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.155589294433593750e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=307, Throttle Trigger(MB)=184, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.942301999381066480e+03) [2024-10-16 14:57:40.734484] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061860732544000, start_id=1729061860732544000, end_id=1729061860732544001) [2024-10-16 14:57:40.735968] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:40.735983] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=10] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=214) [2024-10-16 14:57:40.748246] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107474][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DBE-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173140, log_task={header:{begin_lsn:{lsn:14352373255}, end_lsn:{lsn:14352373377}, log_id:115173140, min_scn:{val:1729061860642319000, v:0}, max_scn:{val:1729061860642319000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352373133}, prev_proposal_id:17, committed_end_lsn:{lsn:14352373255}, data_checksum:940009244, accum_checksum:3887619237, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061860647985, freeze_ts:1729061860647985, submit_ts:1729061860648009, flushed_ts:1729061860648278, gen_to_freeze cost time:0, gen_to_submit cost time:24, submit_to_flush cost time:269}, fs_cb_begin_ts=1729061860748239, log_life_time=100254) [2024-10-16 14:57:40.756405] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:40.756431] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:40.756438] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:40.766677] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.771164] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:40.775247] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061860774364000, start_id=1729061860774364000, end_id=1729061860774364001) [2024-10-16 14:57:40.781594] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=14] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:40.782336] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=34] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=228, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:40.782351] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=108, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:40.782367] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:40.782643] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1F5-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:40.782650] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1F5-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1381, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:40.786522] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15DC-0-0] [lt=17] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:40.788404] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.788532] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.788823] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.788839] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.788852] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.788868] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.788884] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.788894] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.788895] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.788908] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.789078] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:40.793835] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:40.798419] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061860798084000, start_id=1729061860798084000, end_id=1729061860798084001) [2024-10-16 14:57:40.809296] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1913) [2024-10-16 14:57:40.814497] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.814891] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6C-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:40.814911] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6C-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061860643376000, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061860643376000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:40.815640] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=41] get number(ret=0, service_type_=0, range=1, base_id=1729061860813855000, start_id=1729061860813855000, end_id=1729061860813855001) [2024-10-16 14:57:40.815701] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:40.815717] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:40.815966] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D6C-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061860815736000, start_id=1729061860815736000, end_id=1729061860815736001) [2024-10-16 14:57:40.818457] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22483][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6C-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061860818278000, start_id=1729061860818278000, end_id=1729061860818278001) [2024-10-16 14:57:40.825272] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1003}, {id:1001}, {id:1}]) [2024-10-16 14:57:40.838397] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=15] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:40.841744] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061856832346, last_first_register_time_us=1722494157071428, register_time_us=1729061856832346, curr_time_us=1729061860841742) [2024-10-16 14:57:40.841777] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061860841760, last_submit_register_req_time_us=1729061860841760, register_parent_reason=3) [2024-10-16 14:57:40.841786] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:40.848634] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=27] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:40.848653] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:40.852941] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4439, mem_used=8318976) [2024-10-16 14:57:40.864001] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1333) [2024-10-16 14:57:40.866735] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.873816] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107474][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C3F-0-0] [lt=27][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178447, log_task={header:{begin_lsn:{lsn:14538487020}, end_lsn:{lsn:14538487142}, log_id:115178447, min_scn:{val:1729061860643376001, v:0}, max_scn:{val:1729061860643376001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538486898}, prev_proposal_id:23, committed_end_lsn:{lsn:14538487020}, data_checksum:1715074492, accum_checksum:3522037427, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061860773723, freeze_ts:1729061860773723, submit_ts:1729061860773728, flushed_ts:1729061860775670, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:1942}, fs_cb_begin_ts=1729061860873809, log_life_time=100086) [2024-10-16 14:57:40.874004] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:40.882155] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:40.882175] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:40.882630] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107475][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1F8-0-0] [lt=3] init point thread id with(&point=0x55e3435ca340, idx_=979, point=[thread id=107475, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107475) [2024-10-16 14:57:40.888217] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F93-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:40.888394] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=70] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:40.888706] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:40.888885] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D47-0-0] [lt=10] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061860546474001, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061858043985000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870098662000, latest_log_ts:{val:1729061853539570001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:40.889030] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:40.889011] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D47-0-0] [lt=83] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061860546474001, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061858043985000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:40.889082] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D47-0-0] [lt=36] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061860643376002, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061858043985000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:40.889081] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061857241966000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:40.889148] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D47-0-0] [lt=36] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061860642319001, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061858043985000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:40.889179] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D47-0-0] [lt=29] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:40.901279] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:40.901325] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:40.909066] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1913, avg_group_log_size=191) [2024-10-16 14:57:40.909098] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1913) [2024-10-16 14:57:40.909564] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42CE-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:40.909574] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42CE-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=823, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:40.914048] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061860913846000, start_id=1729061860913846000, end_id=1729061860913846001) [2024-10-16 14:57:40.914539] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=9] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:57:40.914622] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.914776] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=15] [APPLY STAT CB APPEND COST TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=39, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:57:40.914786] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=1078, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:57:40.914790] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:57:40.914793] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=-513, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:57:40.914796] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=32, stat_interval=5000000, avg (count/cost)=604, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:57:40.916666] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:40.916679] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27208900, tenant_id=1004) [2024-10-16 14:57:40.916688] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:40.920375] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBDD-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061860888131, keepalive_ts:-1}) [2024-10-16 14:57:40.920392] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBDD-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061860888131, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:40.922078] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:40.934472] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061860933891000, start_id=1729061860933891000, end_id=1729061860933891001) [2024-10-16 14:57:40.936102] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061856932384, last_first_register_time_us=1722494150998907, register_time_us=1729061856932384, curr_time_us=1729061860936101) [2024-10-16 14:57:40.936135] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061860936121, last_submit_register_req_time_us=1729061860936121, register_parent_reason=3) [2024-10-16 14:57:40.936144] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:40.942049] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2841, mem_used=6239232) [2024-10-16 14:57:40.949029] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=11] ====== tenant freeze timer task ====== [2024-10-16 14:57:40.949502] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:40.949519] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:40.949529] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=8] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=149) [2024-10-16 14:57:40.963901] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7763-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:40.966795] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:40.975333] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061860975112000, start_id=1729061860975112000, end_id=1729061860975112001) [2024-10-16 14:57:40.982399] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=206, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:40.982419] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=110, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:40.983163] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD1FB-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=413, avg_log_submit_to_slide_time=1471) [2024-10-16 14:57:40.987897] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 175.00; ) [2024-10-16 14:57:40.987917] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:40.987953] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 218.75; ) [2024-10-16 14:57:40.987962] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:40.987973] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:40.989076] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F95-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:40.989318] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:40.989336] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=110, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:40.989374] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:40.994144] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:40.997024] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Cache wash clean map node details(ret=0, clean_node_count=39, clean_time=0, clean_start_pos=1006624, clean_num=62914) [2024-10-16 14:57:40.998540] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061860997855000, start_id=1729061860997855000, end_id=1729061860997855001) [2024-10-16 14:57:40.999250] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO [ratelimit] time: 1729061860999249, bytes: 4446813712958, bw: 0.373028 MB/s, add_ts: 1001197, add_bytes: 391616 [2024-10-16 14:57:41.009157] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:41.009172] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:41.009407] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=83] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=223, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:41.009423] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=48, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:41.009448] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:41.009717] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42D1-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=304, avg_log_submit_to_slide_time=587) [2024-10-16 14:57:41.014630] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:41.014649] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:41.014656] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.015808] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061861015683000, start_id=1729061861015683000, end_id=1729061861015683001) [2024-10-16 14:57:41.023680] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061861023109000, start_id=1729061861023109000, end_id=1729061861023109001) [2024-10-16 14:57:41.042336] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:41.042352] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:41.042367] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=291923992, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:41.047409] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061861046378000, start_id=1729061861046378000, end_id=1729061861046378001) [2024-10-16 14:57:41.047429] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061861047403], gts:1729061861046378000, latest_srr:[mts=1729061861047403]}) [2024-10-16 14:57:41.047439] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:41.057747] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=18] PNIO [ratelimit] time: 1729061861057745, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007618, add_bytes: 0 [2024-10-16 14:57:41.058383] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:41.066855] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.076400] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=40] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:41.077038] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:41.077088] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061860198021000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:41.077164] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] no need to update min/max version(new_version={val:1729061860569961002, v:0}, min_version={val:1729061860569961002, v:0}, max_version={val:1729061860569961002, v:0}) [2024-10-16 14:57:41.082249] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:41.082267] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:41.090161] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:57:41.090333] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D45-0-0] [lt=9] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061860629635002, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061858043985000, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870721265000, latest_log_ts:{val:1729061854137257001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:41.090492] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D45-0-0] [lt=83] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061860629635002, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061858043985000, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:41.090643] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D45-0-0] [lt=64] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061860643376003, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061858043985000, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:41.090792] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:14538487386}], scn_array=[cnt:1, {val:1729061860643376004, v:0}], curr_size=122, accum_size=244, time_cost=10014) [2024-10-16 14:57:41.090783] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D45-0-0] [lt=58] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061860642319004, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061858043985000, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:41.090853] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D45-0-0] [lt=69] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:41.091396] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=6] ====== tenant freeze timer task ====== [2024-10-16 14:57:41.091408] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151993940, tenant_id=1) [2024-10-16 14:57:41.091421] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:41.091892] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:41.092103] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107474][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=25] worker thread exit(tenant_->id()=1004, workers_.get_size()=3) [2024-10-16 14:57:41.092123] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107474][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=16] tenant thread end_run(id_=1004, ret=0, thread_count_=192) [2024-10-16 14:57:41.092484] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F97-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:41.092501] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F97-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497410, log_task={header:{begin_lsn:{lsn:207589546715}, end_lsn:{lsn:207589546837}, log_id:218497410, min_scn:{val:1729061860642319003, v:0}, max_scn:{val:1729061860642319003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589546593}, prev_proposal_id:81, committed_end_lsn:{lsn:207589546715}, data_checksum:940009244, accum_checksum:1854871780, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061860989071, freeze_ts:1729061860989071, submit_ts:1729061860989090, flushed_ts:1729061860989355, gen_to_freeze cost time:0, gen_to_submit cost time:19, submit_to_flush cost time:265}, fs_cb_begin_ts=1729061861092495, log_life_time=103424) [2024-10-16 14:57:41.092540] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:41.092553] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598858, miss_cnt:9, cache_read_size:149777911278, hit_ratio:"9.999999345924847072e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:41.092772] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107475][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE5B-0-0] [lt=17] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:41.105270] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15DA-0-0] [lt=12] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:41.105402] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA8-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061861071678, keepalive_ts:-1}) [2024-10-16 14:57:41.105413] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA8-0-0] [lt=11] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061861071678, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:41.105473] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA7-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061861071624, keepalive_ts:-1}) [2024-10-16 14:57:41.105478] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EA7-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061861071624, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:41.110791] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:41.116518] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:41.116531] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:41.119948] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:207589546837}], scn_array=[cnt:1, {val:1729061860642319004, v:0}], curr_size=122, accum_size=488, time_cost=18979) [2024-10-16 14:57:41.120089] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.121625] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061861120890000, start_id=1729061861120890000, end_id=1729061861120890001) [2024-10-16 14:57:41.127793] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:41.130625] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2172, mem_used=4159488) [2024-10-16 14:57:41.134569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061861133168000, start_id=1729061861133168000, end_id=1729061861133168001) [2024-10-16 14:57:41.134641] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:41.134657] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:41.135041] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=232, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:41.135052] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10212) [2024-10-16 14:57:41.135063] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=88, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:41.135078] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10212) [2024-10-16 14:57:41.135100] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:41.135386] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5821-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:41.135391] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5821-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=865, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:41.135406] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5821-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=34, avg_log_gen_to_submit_time=63, avg_log_submit_to_flush_time=476, avg_log_submit_to_slide_time=802) [2024-10-16 14:57:41.143109] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9C5-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061861142746000, start_id=1729061861142746000, end_id=1729061861142746001) [2024-10-16 14:57:41.143502] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9C7-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061861142746000, start_id=1729061861142746000, end_id=1729061861142746001) [2024-10-16 14:57:41.144033] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0C51-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061861143993000, start_id=1729061861143993000, end_id=1729061861143993001) [2024-10-16 14:57:41.144187] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0C53-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061861143993000, start_id=1729061861143993000, end_id=1729061861143993001) [2024-10-16 14:57:41.144354] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0C54-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061861143993000, start_id=1729061861143993000, end_id=1729061861143993001) [2024-10-16 14:57:41.151079] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87F-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:41.151104] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA87F-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061860642319003, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061860642319003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:41.151213] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=14] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:41.155545] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA87F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061861155112000, start_id=1729061861155112000, end_id=1729061861155112001) [2024-10-16 14:57:41.155618] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA87F-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10965, avg_group_log_size=577) [2024-10-16 14:57:41.155627] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA87F-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10804) [2024-10-16 14:57:41.156136] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:41.162392] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Slow write(duration=10011, write_len=4096) [2024-10-16 14:57:41.162404] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601402, offset=9625682}, end_cursor:ObLogCursor{file_id=192, log_id=47601403, offset=9625966}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bd0050, len:4014, log_data_len:284, seq:47601402, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:41.164415] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A3-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061861162525000, start_id=1729061861162525000, end_id=1729061861162525001) [2024-10-16 14:57:41.166605] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7767-0-0] [lt=16][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938263, log_task={header:{begin_lsn:{lsn:213058237067}, end_lsn:{lsn:213058237189}, log_id:220938263, min_scn:{val:1729061860643376004, v:0}, max_scn:{val:1729061860643376004, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058236945}, prev_proposal_id:88, committed_end_lsn:{lsn:213058237067}, data_checksum:1715074492, accum_checksum:3819771111, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061861063914, freeze_ts:1729061861063914, submit_ts:1729061861063919, flushed_ts:1729061861080725, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:16806}, fs_cb_begin_ts=1729061861166593, log_life_time=102679) [2024-10-16 14:57:41.166676] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561271, miss_cnt:1, cache_read_size:152590677171, hit_ratio:"9.999999928346884337e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:41.166929] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.168961] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C101B-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061861168694000, start_id=1729061861168694000, end_id=1729061861168694001) [2024-10-16 14:57:41.169674] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601403, offset=9625966}, end_cursor:ObLogCursor{file_id=192, log_id=47601404, offset=9626240}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e10050, len:3730, log_data_len:274, seq:47601403, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:41.170057] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:41.170070] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:41.170076] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:41.170080] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:41.170083] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:41.170087] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16725) [2024-10-16 14:57:41.171550] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257750, offset=39029644}, end_cursor:ObLogCursor{file_id=197, log_id=49257751, offset=39029927}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e54050, len:1140, log_data_len:283, seq:49257750, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:41.173692] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB36E-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061861172210000, start_id=1729061861172210000, end_id=1729061861172210001) [2024-10-16 14:57:41.174037] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C45-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:41.174060] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C45-0-0] [lt=22] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18350, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:41.174064] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C45-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=1603, avg_log_submit_to_slide_time=18343) [2024-10-16 14:57:41.174090] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:41.174102] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650208, miss_cnt:32, cache_read_size:8050416593, hit_ratio:"9.999994892278145642e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:41.178088] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061861176992000, start_id=1729061861176992000, end_id=1729061861176992001) [2024-10-16 14:57:41.178141] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=13, avg_log_batch_cnt=1, total_group_log_size=4371, avg_group_log_size=336) [2024-10-16 14:57:41.178373] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=405, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:41.178387] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=52, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:41.178402] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:41.179159] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C101F-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=13, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=488, avg_log_submit_to_slide_time=783) [2024-10-16 14:57:41.181686] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601404, offset=9626240}, end_cursor:ObLogCursor{file_id=192, log_id=47601405, offset=9626514}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72ae050, len:3456, log_data_len:274, seq:47601404, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:41.182026] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601405, offset=9626514}, end_cursor:ObLogCursor{file_id=192, log_id=47601406, offset=9626788}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2bcc050, len:3182, log_data_len:274, seq:47601405, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:41.182417] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=26] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257751, offset=39029927}, end_cursor:ObLogCursor{file_id=197, log_id=49257752, offset=39030209}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dfc050, len:857, log_data_len:282, seq:49257751, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:41.182567] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1295, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:41.182579] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1212, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:41.182605] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:41.182789] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD200-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:41.184875] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257752, offset=39030209}, end_cursor:ObLogCursor{file_id=197, log_id=49257753, offset=39030491}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72c2050, len:575, log_data_len:282, seq:49257752, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:41.185851] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502B-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:41.185863] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502B-0-0] [lt=11] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:41.186819] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257753, offset=39030491}, end_cursor:ObLogCursor{file_id=197, log_id=49257754, offset=39030773}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7326050, len:4389, log_data_len:282, seq:49257753, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:41.186904] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656946, miss_cnt:17, cache_read_size:8050164086, hit_ratio:"9.999997286813917530e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:41.187987] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E41-0-0] [lt=8] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12274, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:41.188125] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E41-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12274, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061861188113, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061861188113, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061861188113, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:41.188144] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E41-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:12274, add_timestamp:1729061861187961}, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061861188113, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061861188113, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061861188113, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:41.188327] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:41.189137] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB278-0-0] [lt=10][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:41.195946] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8] set tenant work area memory(tenant_id=1001, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:41.196012] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=12] set tenant work area memory(tenant_id=1002, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:41.196062] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=15] set tenant work area memory(tenant_id=1003, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:41.196082] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15D6-0-0] [lt=12] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:41.196109] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=3] set tenant work area memory(tenant_id=1004, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:41.196266] EDIAG [SHARE.CONFIG] check_500_tenant_hold (ob_server_config.cpp:438) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] the hold memory of tenant_500 is over the reserved memory(hold=1084416000, reserved=1073741824) BACKTRACE:0x1a5796b1 0x6847325 0x6a4dfba 0x6a4d9a4 0x6a4d8e0 0x6a4d707 0x1479497f 0x11dea49b 0x6809ea1 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d [2024-10-16 14:57:41.196296] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:32) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=30] === Run print tenant memory usage task === [2024-10-16 14:57:41.196330] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:102) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6] ====== tenants memory info ====== === TENANTS MEMORY INFO === divisive_memory_used= 220,811,264 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,084,416,000 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 19,554,304 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 17,457,152 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 1 now= 1,729,061,861,194,977 active_memstore_used= 247,463,936 total_memstore_used= 245,366,784 total_memstore_hold= 245,366,784 memstore_freeze_trigger_limit= 151,155,080 memstore_limit= 858,993,440 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 1,988,595,712 max_mem_memstore_can_get_now= 755,775,484 memstore_alloc_pos= 20,002,635,776 memstore_frozen_pos= 19,755,171,840 memstore_reclaimed_pos= 19,757,268,992 [TENANT_MEMORY] tenant_id= 1,001 now= 1,729,061,861,194,977 active_memstore_used= 50,331,648 total_memstore_used= 46,137,344 total_memstore_hold= 46,137,344 memstore_freeze_trigger_limit= 54,432,140 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 906,907,648 max_mem_memstore_can_get_now= 272,160,744 memstore_alloc_pos= 32,719,765,504 memstore_frozen_pos= 32,669,433,856 memstore_reclaimed_pos= 32,673,628,160 [TENANT_MEMORY] tenant_id= 1,002 now= 1,729,061,861,194,977 active_memstore_used= 44,040,192 total_memstore_used= 44,040,192 total_memstore_hold= 44,040,192 memstore_freeze_trigger_limit= 28,230,020 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 967,602,176 max_mem_memstore_can_get_now= 141,150,192 memstore_alloc_pos= 465,951,522,816 memstore_frozen_pos= 465,907,482,624 memstore_reclaimed_pos= 465,907,482,624 [TENANT_MEMORY] tenant_id= 1,003 now= 1,729,061,861,194,977 active_memstore_used= 60,817,408 total_memstore_used= 60,817,408 total_memstore_hold= 60,817,408 memstore_freeze_trigger_limit= 53,173,860 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 942,559,232 max_mem_memstore_can_get_now= 265,869,308 memstore_alloc_pos= 34,273,755,136 memstore_frozen_pos= 34,212,937,728 memstore_reclaimed_pos= 34,212,937,728 [TENANT_MEMORY] tenant_id= 1,004 now= 1,729,061,861,194,977 active_memstore_used= 39,845,888 total_memstore_used= 39,845,888 total_memstore_hold= 39,845,888 memstore_freeze_trigger_limit= 27,208,900 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 961,036,288 max_mem_memstore_can_get_now= 136,044,538 memstore_alloc_pos= 1,602,224,128 memstore_frozen_pos= 1,562,378,240 memstore_reclaimed_pos= 1,562,378,240 [2024-10-16 14:57:41.196434] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:114) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=21] [CHUNK_MGR] limit= 12,884,901,888 hold= 7,749,963,776 total_hold= 8,640,266,240 used= 6,888,128,512 freelists_hold= 861,835,264 total_maps= 388,959 total_unmaps= 385,793 large_maps= 301 large_unmaps= 0 huge_maps= 200 huge_unmaps= 187 memalign=0 resident_size= 8,395,571,200 virtual_memory_used= 9,722,109,952 [CHUNK_MGR] 2 MB_CACHE: hold= 121,634,816 free= 58 pushes= 6,196,528,208 pops= 6,196,528,150 maps= 388,458 unmaps= 385,606 [CHUNK_MGR] 4 MB_CACHE: hold= 167,227,392 free= 73 pushes= 6,821,134 pops= 6,821,061 maps= 172 unmaps= 0 [CHUNK_MGR] 6 MB_CACHE: hold= 193,880,064 free= 46 pushes= 6,920 pops= 6,874 maps= 59 unmaps= 0 [CHUNK_MGR] 8 MB_CACHE: hold= 25,247,744 free= 4 pushes= 16,248,833 pops= 16,248,829 maps= 11 unmaps= 0 [CHUNK_MGR] 10 MB_CACHE: hold= 109,318,144 free= 13 pushes= 123 pops= 110 maps= 30 unmaps= 0 [CHUNK_MGR] 12 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [CHUNK_MGR] 14 MB_CACHE: hold= 26,435,584 free= 2 pushes= 4 pops= 2 maps= 9 unmaps= 0 [CHUNK_MGR] 16 MB_CACHE: hold= 33,316,864 free= 2 pushes= 4 pops= 2 maps= 2 unmaps= 0 [CHUNK_MGR] 18 MB_CACHE: hold= 184,774,656 free= 11 pushes= 993 pops= 982 maps= 18 unmaps= 0 [CHUNK_MGR] 20 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [2024-10-16 14:57:41.196481] INFO print (ob_malloc_time_monitor.cpp:39) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8] [MALLOC_TIME_MONITOR] show the distribution of ob_malloc's cost_time [MALLOC_TIME_MONITOR] [ 0, 10): delta_total_cost_time= -1467119259, delta_count= -7243032643, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 10, 100): delta_total_cost_time= -62888346, delta_count= -4016312, avg_cost_time= 15 [MALLOC_TIME_MONITOR] [ 100, 1000): delta_total_cost_time= -714685, delta_count= -3753, avg_cost_time= 190 [MALLOC_TIME_MONITOR] [ 1000, 10000): delta_total_cost_time= -644928, delta_count= -236, avg_cost_time= 2732 [MALLOC_TIME_MONITOR] [ 10000, 100000): delta_total_cost_time= -668844, delta_count= -36, avg_cost_time= 18579 [MALLOC_TIME_MONITOR] [ 100000, 1000000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 1000000, 9223372036854775807): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [2024-10-16 14:57:41.197298] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1179615232, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:41.197495] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash time detail, (compute_wash_size_time=90, refresh_score_time=71, wash_time=0) [2024-10-16 14:57:41.200852] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061861200706000, start_id=1729061861200706000, end_id=1729061861200706001) [2024-10-16 14:57:41.202249] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21671][T1_L0_G0][T1][YB4A0A000407-00061E9B257FB278-0-0] [lt=3][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:41.205492] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D82-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510329073886, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410597, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:41.209480] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:41.209521] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:41.220151] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.230981] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:57:41.230996] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=83, cache_node_num=83) [2024-10-16 14:57:41.231003] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=83, cache_node_num=83) [2024-10-16 14:57:41.231008] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:41.233175] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=83, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}, {obj_id:78893917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}]) [2024-10-16 14:57:41.233293] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=115] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:41.243394] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:57:41.244089] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061861243803000, start_id=1729061861243803000, end_id=1729061861243803001) [2024-10-16 14:57:41.248282] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DC8-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:41.248294] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DC8-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20581, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:41.248298] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DC8-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=11, avg_log_submit_to_flush_time=402, avg_log_submit_to_slide_time=20570) [2024-10-16 14:57:41.248306] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DC8-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173145, log_task={header:{begin_lsn:{lsn:14352373865}, end_lsn:{lsn:14352373987}, log_id:115173145, min_scn:{val:1729061861143993001, v:0}, max_scn:{val:1729061861143993001, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352373743}, prev_proposal_id:17, committed_end_lsn:{lsn:14352373865}, data_checksum:940009244, accum_checksum:962855910, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061861148161, freeze_ts:1729061861148162, submit_ts:1729061861148170, flushed_ts:1729061861148828, gen_to_freeze cost time:1, gen_to_submit cost time:9, submit_to_flush cost time:658}, fs_cb_begin_ts=1729061861248276, log_life_time=100115) [2024-10-16 14:57:41.248358] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:41.248513] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=204, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:41.248529] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=179, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:41.248558] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:41.263935] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7769-0-0] [lt=22] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:41.263964] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7769-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:41.263969] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7769-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20534, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:41.266969] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.274013] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C47-0-0] [lt=17] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1333) [2024-10-16 14:57:41.274033] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C47-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178451, log_task={header:{begin_lsn:{lsn:14538487508}, end_lsn:{lsn:14538487630}, log_id:115178451, min_scn:{val:1729061861143993000, v:0}, max_scn:{val:1729061861143993000, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538487386}, prev_proposal_id:23, committed_end_lsn:{lsn:14538487508}, data_checksum:1715074492, accum_checksum:1556333682, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061861174021, freeze_ts:1729061861174021, submit_ts:1729061861174025, flushed_ts:1729061861182588, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:8563}, fs_cb_begin_ts=1729061861274028, log_life_time=100007) [2024-10-16 14:57:41.277462] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:41.281516] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21656][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3DF2-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510329149911, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410598, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:41.282376] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:41.282390] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:41.282416] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:41.282421] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:41.287534] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1036-0-0] [lt=25] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:41.288514] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=8] replay status is not follower(min_unreplayed_lsn={lsn:45338084276}, base_lsn={lsn:45161508864}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:41.288569] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=55] get_replay_process success(id={id:1}, replayed_log_size=176575412, unreplayed_log_size=0) [2024-10-16 14:57:41.288586] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1003, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=168, last_replayed_log_size(MB)=168, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:41.289667] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84F9B-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497412, log_task={header:{begin_lsn:{lsn:207589546959}, end_lsn:{lsn:207589547081}, log_id:218497412, min_scn:{val:1729061861142746000, v:0}, max_scn:{val:1729061861142746000, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589546837}, prev_proposal_id:81, committed_end_lsn:{lsn:207589546959}, data_checksum:940009244, accum_checksum:2541388288, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061861189338, freeze_ts:1729061861189338, submit_ts:1729061861189341, flushed_ts:1729061861189536, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:195}, fs_cb_begin_ts=1729061861289661, log_life_time=100323) [2024-10-16 14:57:41.300159] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get_max_applied_scn(scn={val:1729061861046378000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959696573}, palf_committed_end_scn_:{val:1729061861168694000, v:0}, last_check_scn_:{val:1729061861168694000, v:0}, max_applied_cb_scn_:{val:1729061861046378000, v:0}}) [2024-10-16 14:57:41.300180] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:41.300235] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=41] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:57:41.300246] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get_min_unreplayed_log_info(lsn={lsn:213058237433}, scn={val:1729061861143993002, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061861264447, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058237433}, next_to_submit_scn_:{val:1729061861143993002, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058237311}, end_lsn:{lsn:213058237433}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061861143993001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061861143993001, v:0}, data_checksum:3348837161, flag:1}}, init_mode_version:0, accumulate_checksum:4203442361, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061861143993001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561273, miss_cnt:1, cache_read_size:152590677415, hit_ratio:"9.999999928346885447e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:41.306127] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=21] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:41.309319] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:41.309336] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:41.310864] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:41.311947] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:41.312032] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAA-0-0] [lt=11] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061869029444000, latest_log_ts:{val:1729061852392712001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:41.312088] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAA-0-0] [lt=53] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:41.316418] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061861316167000, start_id=1729061861316167000, end_id=1729061861316167001) [2024-10-16 14:57:41.317100] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1729061860642319003, v:0}, new_max_version={val:1729061860642319003, v:0}, last_min_version={val:1729061860629635001, v:0}, last_max_version={val:1729061860629635001, v:0}, total_time_us=1465, delta_us=674780) [2024-10-16 14:57:41.317119] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1729061860642319003, v:0}, min_version={val:1729061860642319003, v:0}, max_version={val:1729061860642319003, v:0}, in_service=true, leader_epoch=17) [2024-10-16 14:57:41.317250] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7026, mem_used=14558208) [2024-10-16 14:57:41.320224] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.321264] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061861320427000, start_id=1729061861320427000, end_id=1729061861320427001) [2024-10-16 14:57:41.329021] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:41.333355] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=53] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:41.333375] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:41.334820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061861334204000, start_id=1729061861334204000, end_id=1729061861334204001) [2024-10-16 14:57:41.335351] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [WRS] update tenant weak read server version(tenant_id=1001, server_version={version:{val:1729061861200706000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061861335333}, version_delta=134644) [2024-10-16 14:57:41.354786] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E05-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061861354697000, start_id=1729061861354697000, end_id=1729061861354697001) [2024-10-16 14:57:41.364450] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=30] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:41.364491] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:41.364502] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:41.367022] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.367961] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAD-0-0] [lt=7] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:41.367977] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAD-0-0] [lt=12] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=16) [2024-10-16 14:57:41.374033] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C49-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:41.378360] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061861377961000, start_id=1729061861377961000, end_id=1729061861377961001) [2024-10-16 14:57:41.380815] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=13] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:57:41.382635] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:41.394417] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=20] gc stale ls task succ [2024-10-16 14:57:41.397784] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=5] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:41.401070] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061861400969000, start_id=1729061861400969000, end_id=1729061861400969001) [2024-10-16 14:57:41.401812] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] get_max_applied_scn(scn={val:1729061861168694000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023100219}, palf_committed_end_scn_:{val:1729061861168694001, v:0}, last_check_scn_:{val:1729061861168694001, v:0}, max_applied_cb_scn_:{val:1729061861168694000, v:0}}) [2024-10-16 14:57:41.401834] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:41.420190] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=6] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:57:41.420206] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=15] try gc retain ctx [2024-10-16 14:57:41.420210] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:41.420847] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061869029444000, latest_log_ts:{val:1729061852392712001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:41.421005] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.448571] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:41.462198] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:41.464261] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=210, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:41.464281] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=696, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:41.467069] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.471310] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7F-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:41.471336] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A7F-0-0] [lt=24] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:41.474003] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:57:41.474027] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=21] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:41.474035] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:41.474042] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:41.476359] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=8] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:57:41.476492] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=131] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:41.482730] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:41.482949] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD20A-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:41.482959] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD20A-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=2104, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:41.482963] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD20A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=1885, avg_log_submit_to_slide_time=2087) [2024-10-16 14:57:41.487573] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:41.487582] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:41.487593] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:41.487594] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED4-0-0] [lt=7] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:41.487598] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:41.487611] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:41.487614] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:41.487611] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED4-0-0] [lt=15] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:41.487617] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:41.487619] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:41.487622] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:41.487624] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:41.487627] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:41.488903] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53A6-0-0] [lt=25] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:41.488915] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53A6-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40284, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:41.489971] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:41.489988] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:41.490004] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] cannot_recycle_log_size statistics(cannot_recycle_log_size=70678806, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:41.509369] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:41.509400] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:41.509647] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:41.509880] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42DF-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:41.509890] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42DF-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1611, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:41.509941] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42E0-0-0] [lt=20] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=1358, avg_log_submit_to_slide_time=1598) [2024-10-16 14:57:41.510927] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:41.514056] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061851512642, end_time=1729061861514055, dag_status={}) [2024-10-16 14:57:41.514071] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=14] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:41.514086] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=14] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:41.514090] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:41.514097] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:41.514100] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:41.514106] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:41.514110] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:41.514122] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:41.514127] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:41.514131] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514137] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514141] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514145] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514152] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514156] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514163] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514167] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514177] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514184] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514188] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514192] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514196] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514200] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514205] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514209] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514213] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514218] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514223] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514227] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514232] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514238] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514242] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514248] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514252] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514265] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514275] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514279] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514288] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514296] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514304] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514308] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514312] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514319] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514323] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514330] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514334] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514340] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514344] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514351] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514355] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514362] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514366] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514370] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514377] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514380] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514387] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514391] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514397] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514401] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514407] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514411] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514415] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514422] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514426] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514435] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514442] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514446] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:41.514453] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:41.514457] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:41.514465] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:41.514469] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:41.514476] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:41.514480] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:41.514487] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:41.514490] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:41.514492] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:41.514494] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:41.514505] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:41.516686] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061861516036000, start_id=1729061861516036000, end_id=1729061861516036001) [2024-10-16 14:57:41.516755] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10677, avg_group_log_size=628) [2024-10-16 14:57:41.516765] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10206) [2024-10-16 14:57:41.517062] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=284, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:41.517072] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10677) [2024-10-16 14:57:41.517083] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=46, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:41.517123] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10206) [2024-10-16 14:57:41.517139] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:41.517177] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD133-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:41.517184] INFO [LIB] stat (utility.h:1165) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD133-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1172, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:41.517188] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD133-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=52, avg_log_gen_to_submit_time=78, avg_log_submit_to_flush_time=897, avg_log_submit_to_slide_time=1094) [2024-10-16 14:57:41.521064] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.521614] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061861521470000, start_id=1729061861521470000, end_id=1729061861521470001) [2024-10-16 14:57:41.533436] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3784) [2024-10-16 14:57:41.533657] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3662) [2024-10-16 14:57:41.533724] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3662) [2024-10-16 14:57:41.534001] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1022-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:41.534016] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1022-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=849, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:41.534922] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061861534164000, start_id=1729061861534164000, end_id=1729061861534164001) [2024-10-16 14:57:41.536360] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E99801B4F13-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061861536287000, start_id=1729061861536287000, end_id=1729061861536287001) [2024-10-16 14:57:41.544994] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E99801B4F13-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061861544747000, start_id=1729061861544747000, end_id=1729061861544747001) [2024-10-16 14:57:41.548041] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061861546850000, start_id=1729061861546850000, end_id=1729061861546850001) [2024-10-16 14:57:41.548056] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] ts response task statistics(alloc_count=261364430, free_count=130682214) [2024-10-16 14:57:41.554200] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E99801B4F13-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061861553569000, start_id=1729061861553569000, end_id=1729061861553569001) [2024-10-16 14:57:41.564125] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE776F-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:41.564384] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:41.567133] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.567494] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E99801B4F13-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061861567105000, start_id=1729061861567105000, end_id=1729061861567105001) [2024-10-16 14:57:41.568624] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.568781] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.569225] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.569244] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.575647] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E99801B4F13-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061861574679000, start_id=1729061861574679000, end_id=1729061861574679001) [2024-10-16 14:57:41.576469] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C3-0-0] [lt=25] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:41.576493] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C3-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061861143993005, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061861143993005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:41.578503] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061861577830000, start_id=1729061861577830000, end_id=1729061861577830001) [2024-10-16 14:57:41.582125] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C3-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061861581478000, start_id=1729061861581478000, end_id=1729061861581478001) [2024-10-16 14:57:41.583279] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:41.584983] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1003}, {id:1}, {id:1002}]) [2024-10-16 14:57:41.586868] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D5-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061861578097, keepalive_ts:-1}) [2024-10-16 14:57:41.586881] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D5-0-0] [lt=12] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061861578097, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:41.587531] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:41.587548] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:41.598075] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:41.601121] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=55] get number(ret=0, service_type_=0, range=1, base_id=1729061861600496000, start_id=1729061861600496000, end_id=1729061861600496001) [2024-10-16 14:57:41.602518] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D6-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061861570251, keepalive_ts:-1}) [2024-10-16 14:57:41.602533] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D6-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061861570251, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:41.609130] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=40] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4995, mem_used=10398720) [2024-10-16 14:57:41.609674] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=255, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:41.609688] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=174, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:41.609712] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:41.618779] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C50-0-0] [lt=10] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:41.618805] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C50-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061861143993004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061861143993004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:41.621122] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.624096] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C50-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061861623801000, start_id=1729061861623801000, end_id=1729061861623801001) [2024-10-16 14:57:41.643511] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9CA-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061861642794000, start_id=1729061861642794000, end_id=1729061861642794001) [2024-10-16 14:57:41.648553] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DD0-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:41.648577] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DD0-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:41.648587] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DD0-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:41.663511] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26147][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB18CC6EC-0-0] [lt=33] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:41.667194] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.667593] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:41.674708] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107475][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C4F-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:41.674729] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107475][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C4F-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178455, log_task={header:{begin_lsn:{lsn:14538487996}, end_lsn:{lsn:14538488118}, log_id:115178455, min_scn:{val:1729061861143993004, v:0}, max_scn:{val:1729061861143993004, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538487874}, prev_proposal_id:23, committed_end_lsn:{lsn:14538487996}, data_checksum:1715074492, accum_checksum:1058008160, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061861574152, freeze_ts:1729061861574152, submit_ts:1729061861574158, flushed_ts:1729061861574876, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:718}, fs_cb_begin_ts=1729061861674723, log_life_time=100571) [2024-10-16 14:57:41.698681] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:41.709457] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:57:41.709476] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=17] try gc retain ctx [2024-10-16 14:57:41.709481] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] try to retry start_working [2024-10-16 14:57:41.710283] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061859044902000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870098662000, latest_log_ts:{val:1729061853539570001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:41.711212] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=68] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061859044902000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:41.711938] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=57] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061859044902000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:41.712697] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=40] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061859044902000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:41.712810] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:41.716858] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061861715797000, start_id=1729061861715797000, end_id=1729061861715797001) [2024-10-16 14:57:41.721322] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15DE-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:41.721442] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15E0-0-0] [lt=17] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:41.722376] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061861722152000, start_id=1729061861722152000, end_id=1729061861722152001) [2024-10-16 14:57:41.722547] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.724508] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:41.734208] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1001, min_version={val:1729061861400969000, v:0}, base_version={val:1729061861334204000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061861534164000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061861734190, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061861400969001, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061861626436, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061861400969000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061861590847, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:41.735094] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF16E-0-0] [lt=35] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.155740642547607422e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=307, Throttle Trigger(MB)=184, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.942112515423355262e+03) [2024-10-16 14:57:41.735266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061861735019000, start_id=1729061861735019000, end_id=1729061861735019001) [2024-10-16 14:57:41.756456] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:41.756501] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=32] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:41.756513] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=10] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:41.759573] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=20] BlockGCTimerTask success(ret=0, cost_time_us=10, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:41.767255] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.771226] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:41.778664] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061861777991000, start_id=1729061861777991000, end_id=1729061861777991001) [2024-10-16 14:57:41.781700] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=19] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:41.782811] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=892, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:41.782829] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=962, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:41.782843] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:41.783048] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD213-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:41.783054] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD213-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2162, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:41.788220] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.788524] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.788712] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.788727] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.788873] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.788874] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.788889] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.788895] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:41.788903] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.788921] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.788934] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.789020] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:41.798351] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:41.801325] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061861801196000, start_id=1729061861801196000, end_id=1729061861801196001) [2024-10-16 14:57:41.809792] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:41.820756] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6D-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:41.820784] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6D-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061861643829001, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061861643829001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:41.821890] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D6D-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061861821291000, start_id=1729061861821291000, end_id=1729061861821291001) [2024-10-16 14:57:41.822617] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.823422] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=23] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):113, sequence:1}) [2024-10-16 14:57:41.825373] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1}, {id:1003}]) [2024-10-16 14:57:41.825979] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6D-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061861825531000, start_id=1729061861825531000, end_id=1729061861825531001) [2024-10-16 14:57:41.826045] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6D-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:41.826059] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6D-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:41.828409] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:41.828423] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=433) [2024-10-16 14:57:41.828435] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52420242, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:41.840158] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=53] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:41.840737] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:41.840789] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=17] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061854425066000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:41.843166] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061857833647, last_first_register_time_us=1722494157071428, register_time_us=1729061857833647, curr_time_us=1729061861843164) [2024-10-16 14:57:41.843203] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061861843185, last_submit_register_req_time_us=1729061861843185, register_parent_reason=3) [2024-10-16 14:57:41.843211] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:41.844494] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:41.848996] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=44, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:41.849010] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:41.853051] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4512, mem_used=8318976) [2024-10-16 14:57:41.864477] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:41.864480] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:41.867362] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.874518] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26143][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB26CC6B9-0-0] [lt=22] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:57:41.874627] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:41.882649] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:41.882670] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:41.886899] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:41.887597] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:41.887658] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=20] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061861837115000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:41.888721] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FA5-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=9, avg_log_batch_cnt=0, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:41.901395] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=8] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:41.909587] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=50] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:41.909619] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=30] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:41.910185] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4108-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:41.910203] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4108-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1631, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:41.914245] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2139076, replace_num=31457) [2024-10-16 14:57:41.914265] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:41.915948] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=28] no need to update min/max version(new_version={val:1729061861046378004, v:0}, min_version={val:1729061861046378004, v:0}, max_version={val:1729061861046378004, v:0}) [2024-10-16 14:57:41.920365] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=8] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:41.920385] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=14] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=213) [2024-10-16 14:57:41.921197] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DD9-0-0] [lt=10] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:57:41.921863] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061861921724000, start_id=1729061861921724000, end_id=1729061861921724001) [2024-10-16 14:57:41.922175] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:41.922677] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.935118] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061861934409000, start_id=1729061861934409000, end_id=1729061861934409001) [2024-10-16 14:57:41.942107] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2881, mem_used=6239232) [2024-10-16 14:57:41.967462] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:41.967512] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7777-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:41.979029] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061861978803000, start_id=1729061861978803000, end_id=1729061861978803001) [2024-10-16 14:57:41.983032] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=816, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:41.983053] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=943, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:41.983699] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD219-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=1891, avg_log_submit_to_slide_time=2147) [2024-10-16 14:57:41.993210] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 753.25; ) [2024-10-16 14:57:41.993231] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:41.993274] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.98, rt: 693.50; ) [2024-10-16 14:57:41.993282] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:41.993293] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:41.993433] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D38-0-0] [lt=14] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:41.993449] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D38-0-0] [lt=13] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:41.997699] INFO [CLOG] do_thread_task_ (ob_remote_fetch_log_worker.cpp:249) [22412][T1003_RFLWorker][T1003][YB4A0A000407-00061E996FF960C1-0-0] [lt=16] ObRemoteFetchWorker is running(thread_index=0) [2024-10-16 14:57:41.998664] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:42.000806] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=18] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:42.001321] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061862000994000, start_id=1729061862000994000, end_id=1729061862000994001) [2024-10-16 14:57:42.001388] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061862001386, bytes: 4446814200987, bw: 0.464428 MB/s, add_ts: 1002137, add_bytes: 488029 [2024-10-16 14:57:42.009668] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:42.009683] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:42.009915] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=243, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:42.009929] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=307, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:42.009941] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:42.010081] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42ED-0-0] [lt=20] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=20, avg_log_submit_to_flush_time=1482, avg_log_submit_to_slide_time=1697) [2024-10-16 14:57:42.016860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061862016696000, start_id=1729061862016696000, end_id=1729061862016696001) [2024-10-16 14:57:42.022741] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.048382] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061862047401000, start_id=1729061862047401000, end_id=1729061862047401001) [2024-10-16 14:57:42.048406] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:42.048423] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061862048420], gts:1729061862047401000, latest_srr:[mts=1729061862048420]}) [2024-10-16 14:57:42.065855] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=16] PNIO [ratelimit] time: 1729061862065854, bytes: 0, bw: 0.000000 MB/s, add_ts: 1008109, add_bytes: 0 [2024-10-16 14:57:42.067809] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.082727] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:42.082744] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:42.088816] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FA9-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:42.089031] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2055, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:42.089042] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=864, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:42.089066] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:42.093084] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:42.093351] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107475][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=18] worker thread exit(tenant_->id()=1004, workers_.get_size()=2) [2024-10-16 14:57:42.093362] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107475][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=8] tenant thread end_run(id_=1004, ret=0, thread_count_=191) [2024-10-16 14:57:42.114319] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:42.116838] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:42.116855] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:42.122801] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.125882] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061862125800000, start_id=1729061862125800000, end_id=1729061862125800001) [2024-10-16 14:57:42.126589] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=14] current monitor number(seq_=-1) [2024-10-16 14:57:42.127853] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:42.130708] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2303, mem_used=6239232) [2024-10-16 14:57:42.135156] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061862134254000, start_id=1729061862134254000, end_id=1729061862134254001) [2024-10-16 14:57:42.135220] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:42.135234] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:42.135468] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=273, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:42.135478] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10666) [2024-10-16 14:57:42.135489] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=109, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:42.135503] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10667) [2024-10-16 14:57:42.135517] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:42.135799] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5833-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:42.135805] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5833-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=741, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:42.135809] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5833-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=440, avg_log_submit_to_slide_time=710) [2024-10-16 14:57:42.143976] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9D3-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061862143780000, start_id=1729061862143780000, end_id=1729061862143780001) [2024-10-16 14:57:42.144183] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9D1-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061862143780000, start_id=1729061862143780000, end_id=1729061862143780001) [2024-10-16 14:57:42.144537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0C5B-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061862143780000, start_id=1729061862143780000, end_id=1729061862143780001) [2024-10-16 14:57:42.144662] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9CF-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061862143780000, start_id=1729061862143780000, end_id=1729061862143780001) [2024-10-16 14:57:42.144690] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B9D2-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061862143780000, start_id=1729061862143780000, end_id=1729061862143780001) [2024-10-16 14:57:42.159102] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA880-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:42.159136] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA880-0-0] [lt=20] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061861642794003, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061861642794003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:42.163130] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA880-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061862162906000, start_id=1729061862162906000, end_id=1729061862162906001) [2024-10-16 14:57:42.163213] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA880-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10668, avg_group_log_size=592) [2024-10-16 14:57:42.163224] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA880-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10666) [2024-10-16 14:57:42.163898] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:42.163913] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB APPEND COST TIME](cur_stat_count=52, stat_interval=5000000, avg (count/cost)=44, this=0x2b0e07718280, extra_info="") [2024-10-16 14:57:42.163920] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=52, stat_interval=5000000, avg (count/cost)=1010, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:57:42.163923] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=52, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:57:42.163926] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=52, stat_interval=5000000, avg (count/cost)=-688, this=0x2b0e07718910, extra_info="") [2024-10-16 14:57:42.163930] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=52, stat_interval=5000000, avg (count/cost)=367, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:57:42.167869] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.174909] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B912B-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:42.174921] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B912B-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=45790, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:42.174926] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B912B-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=1269, avg_log_submit_to_slide_time=45783) [2024-10-16 14:57:42.174920] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:42.174939] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650219, miss_cnt:32, cache_read_size:8050417935, hit_ratio:"9.999994892279042702e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:42.178813] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061862178738000, start_id=1729061862178738000, end_id=1729061862178738001) [2024-10-16 14:57:42.178879] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=3423, avg_group_log_size=380) [2024-10-16 14:57:42.179134] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=201, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:42.179149] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=35, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:42.179164] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=7, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:42.179373] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BF1-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=260, avg_log_submit_to_slide_time=731) [2024-10-16 14:57:42.181995] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C102D-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061862180954000, start_id=1729061862180954000, end_id=1729061862180954001) [2024-10-16 14:57:42.183059] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=51] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1914) [2024-10-16 14:57:42.183139] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601406, offset=9626788}, end_cursor:ObLogCursor{file_id=192, log_id=47601407, offset=9627072}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d38050, len:2908, log_data_len:284, seq:47601406, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:42.185077] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A4-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061862184436000, start_id=1729061862184436000, end_id=1729061862184436001) [2024-10-16 14:57:42.185339] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE7C-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:42.186622] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502C-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:42.186639] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502C-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:42.186813] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601407, offset=9627072}, end_cursor:ObLogCursor{file_id=192, log_id=47601408, offset=9627346}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7302050, len:2624, log_data_len:274, seq:47601407, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:42.188893] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FAB-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:42.189471] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:42.189488] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598869, miss_cnt:9, cache_read_size:149777912620, hit_ratio:"9.999999345924899252e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:42.189962] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601408, offset=9627346}, end_cursor:ObLogCursor{file_id=192, log_id=47601409, offset=9627620}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e18050, len:2350, log_data_len:274, seq:47601408, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:42.189964] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257754, offset=39030773}, end_cursor:ObLogCursor{file_id=197, log_id=49257755, offset=39031056}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c4c050, len:4107, log_data_len:283, seq:49257754, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:42.190163] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601409, offset=9627620}, end_cursor:ObLogCursor{file_id=192, log_id=47601410, offset=9627894}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db728e050, len:2076, log_data_len:274, seq:47601409, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:42.190217] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561282, miss_cnt:1, cache_read_size:152590678513, hit_ratio:"9.999999928346889888e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:42.191760] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB36F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061862189834000, start_id=1729061862189834000, end_id=1729061862189834001) [2024-10-16 14:57:42.192510] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257755, offset=39031056}, end_cursor:ObLogCursor{file_id=197, log_id=49257756, offset=39031338}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7276050, len:3824, log_data_len:282, seq:49257755, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:42.194364] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257756, offset=39031338}, end_cursor:ObLogCursor{file_id=197, log_id=49257757, offset=39031620}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e3c050, len:3542, log_data_len:282, seq:49257756, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:42.194514] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257757, offset=39031620}, end_cursor:ObLogCursor{file_id=197, log_id=49257758, offset=39031902}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2da8050, len:3260, log_data_len:282, seq:49257757, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:42.194570] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656957, miss_cnt:17, cache_read_size:8050165428, hit_ratio:"9.999997286814393815e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:42.197425] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:57:42.197563] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D85-0-0] [lt=23] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061877008855000, latest_log_ts:{val:1729061860546474001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:42.197626] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D85-0-0] [lt=60] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:42.198944] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:42.201397] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061862200058000, start_id=1729061862200058000, end_id=1729061862200058001) [2024-10-16 14:57:42.209105] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D84-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510330077499, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410599, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:42.210046] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:42.210071] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:42.212959] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=4] Garbage Collector is running(seq_=656699, gc_interval=10000000) [2024-10-16 14:57:42.213033] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=17] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:42.213048] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=8] gc_check_member_list_ cost time(ret=0, time_us=73) [2024-10-16 14:57:42.213054] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:42.213804] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:42.213819] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=13] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:42.213825] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=7) [2024-10-16 14:57:42.213837] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:42.217502] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061862217083000, start_id=1729061862217083000, end_id=1729061862217083001) [2024-10-16 14:57:42.218444] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=19][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:42.222861] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.248990] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=213, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:42.249012] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:42.249048] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:42.249357] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B912C-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:42.249364] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B912C-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=502, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:42.249368] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B912C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=254, avg_log_submit_to_slide_time=495) [2024-10-16 14:57:42.249371] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:42.264589] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE777D-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:42.265196] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53B4-0-0] [lt=25] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:42.265208] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53B4-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9258, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:42.267403] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:42.267932] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.274599] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C5B-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:42.274811] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=187, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:42.274830] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=306, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:42.274869] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:42.280233] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD00B-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061862271335, keepalive_ts:-1}) [2024-10-16 14:57:42.280270] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD00B-0-0] [lt=36] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061862271335, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:42.282843] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1914, avg_group_log_size=191) [2024-10-16 14:57:42.282857] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1914) [2024-10-16 14:57:42.282880] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:42.282886] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:42.283121] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2036) [2024-10-16 14:57:42.292567] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106820][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D3B-0-0] [lt=15] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:42.292592] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106820][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D3B-0-0] [lt=20] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:42.305164] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:42.305186] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=15] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53173860, tenant_id=1003) [2024-10-16 14:57:42.305197] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:42.309799] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:42.309814] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:42.311841] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:42.311859] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:42.311876] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373614850, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:42.312481] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [21909][T1001_SSTableGC][T1001][YB4A0A000407-00061E99777CB787-0-0] [lt=12] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60476}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015249374835000}, old_min_reserved_snapshot=1729060058725452000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060048719447000) [2024-10-16 14:57:42.314398] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:42.316793] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:42.317354] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=8] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7084, mem_used=14558208) [2024-10-16 14:57:42.324190] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [21909][T1001_SSTableGC][T1001][YB4A0A000407-00061E99777CB787-0-0] [lt=29] SSTableGCTask(cost_ts=11877) [2024-10-16 14:57:42.324685] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.326433] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061862324239000, start_id=1729061862324239000, end_id=1729061862324239001) [2024-10-16 14:57:42.327345] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1729061862016696000, v:0}, new_max_version={val:1729061862016696000, v:0}, last_min_version={val:1729061861921723999, v:0}, last_max_version={val:1729061861921723999, v:0}, total_time_us=1660, delta_us=310648) [2024-10-16 14:57:42.327361] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1729061862016696000, v:0}, min_version={val:1729061862016696000, v:0}, max_version={val:1729061862016696000, v:0}, in_service=true, leader_epoch=33576) [2024-10-16 14:57:42.329106] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:42.333856] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:42.333874] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:42.335377] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061862334158000, start_id=1729061862334158000, end_id=1729061862334158001) [2024-10-16 14:57:42.351646] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:42.351667] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=16] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:42.351675] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:42.351679] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:42.351682] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:42.351685] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17115) [2024-10-16 14:57:42.352598] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E8-0-0] [lt=49] [LS_LOCATION]dump tenant ls location caches(tenant_id=1, tenant_ls_locations=[{cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061859988236, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:42.352637] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E8-0-0] [lt=38] [LS_LOCATION]dump tenant ls location caches(tenant_id=1001, tenant_ls_locations=[{cache_key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061859988981, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:42.352648] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E8-0-0] [lt=10] [LS_LOCATION]dump tenant ls location caches(tenant_id=1002, tenant_ls_locations=[{cache_key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061859989704, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061859989703, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061859989705, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}]}, {cache_key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061859989701, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:42.352678] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E8-0-0] [lt=29] [LS_LOCATION]dump tenant ls location caches(tenant_id=1003, tenant_ls_locations=[{cache_key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061859990342, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:42.352686] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E8-0-0] [lt=8] [LS_LOCATION]dump tenant ls location caches(tenant_id=1004, tenant_ls_locations=[{cache_key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061859991050, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061859991053, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061859991054, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}]}, {cache_key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061859991052, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:42.365236] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:42.365295] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:42.365311] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:42.367966] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.374622] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C5D-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:42.378947] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061862378674000, start_id=1729061862378674000, end_id=1729061862378674001) [2024-10-16 14:57:42.381916] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183786035, max_lsn:114618526881, max_scn:{val:1729061862334158000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45685691230}, reuse_lsn:{lsn:114618526881}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114576583841}}, last_submit_lsn:{lsn:114618526289}, last_submit_end_lsn:{lsn:114618526881}, last_submit_log_id:183786035, last_submit_log_pid:33705, max_flushed_lsn:{lsn:114618526289}, max_flushed_end_lsn:{lsn:114618526881}, max_flushed_log_pid:33705, committed_end_lsn:{lsn:114618526881}, last_slide_log_id:183786035, last_slide_scn:{val:1729061862334158000, v:0}, last_slide_lsn:{lsn:114618526289}, last_slide_end_lsn:{lsn:114618526881}, last_slide_log_pid:33705, last_slide_log_accum_checksum:2012944807, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0df99e2530}, [StateMgr]={this:0x2b0df99e6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, leader:"10.0.4.7:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183786036, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261592457, palf_role_change_cb:0x2b0df99ed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, reconfig_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, persistent_config_version:{proposal_id:33705, config_seq:14}, ms_ack_list:0, resend_config_version:{proposal_id:33705, config_seq:14}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258289572, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258289572, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0df99e3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261792660, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729029056668344001, v:0}, min_block_id:1702, min_block_min_scn:{val:1729022447183332001, v:0}, base_lsn_for_block_gc:{lsn:114547838976}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, log_config_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, log_snapshot_meta:{version:1, base_lsn:{lsn:114547838976}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:233218048}, readable_log_tail:{lsn:233218048}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114682048512}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/meta", dir_fd:146, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:35201024, block_header_serialize_buf_:0x2b0df99e89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114618526881}, readable_log_tail:{lsn:114618526881}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114614943744}, min_scn:{val:1729061506448191001, v:0}, curr_block_id:1708, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/log", dir_fd:148, min_block_id:1702, max_block_id:1709, curr_writable_block_id:1708}, logical_block_size_:67104768, curr_block_writable_size_:63521631, block_header_serialize_buf_:0x2b0df99ea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261592575, this:0x2b0df99e6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33705, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33704, majority_max_lsn:{lsn:45686897415}, saved_end_lsn:{lsn:45686897415}, last_submit_prepare_req_time_us:1722494261592686, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0df99e69b0}) [2024-10-16 14:57:42.382061] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=142] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061862336229, last_flushed_end_lsn:{lsn:114618526881}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061862337190, last_flushed_end_lsn:{lsn:114618526881}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061862336434, last_flushed_end_lsn:{lsn:114618526881}}]) [2024-10-16 14:57:42.382073] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183786036, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061862336229, last_flushed_end_lsn:{lsn:114618526881}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061862337190, last_flushed_end_lsn:{lsn:114618526881}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061862336434, last_flushed_end_lsn:{lsn:114618526881}}]) [2024-10-16 14:57:42.383135] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:42.394483] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:42.396379] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:42.396382] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=15] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:42.396394] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:42.396412] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:42.399236] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:42.401636] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061862400918000, start_id=1729061862400918000, end_id=1729061862400918001) [2024-10-16 14:57:42.417896] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] submit log success(service_type_=0, last_id=1729061879029444000, limited_id=1729061879029444000) [2024-10-16 14:57:42.417927] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=31] submit log with lock success(service_type_=0, last_id=1729061879029444000, limited_id=1729061879029444000) [2024-10-16 14:57:42.417934] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061862417342000, start_id=1729061862417342000, end_id=1729061862417342001) [2024-10-16 14:57:42.418546] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=14] update id meta(ret=0, service_type=0, limited_id=1729061879029444000, latest_scn={val:1729061862417342001, v:0}, *this={tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061879029444000, latest_log_ts:{val:1729061862417342001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:42.418582] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=35] handle submit callback(service_type_=0, success=true, used_ts=686, log_ts={val:1729061862417342001, v:0}, limited_id=1729061879029444000) [2024-10-16 14:57:42.418589] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:42.418593] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:42.418672] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061862418457000, start_id=1729061862418457000, end_id=1729061862418457001) [2024-10-16 14:57:42.424748] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.449248] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:42.464932] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=209, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:42.464953] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=27, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:42.468024] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.472068] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A80-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:42.472090] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A80-0-0] [lt=20] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:42.477302] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=40] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:42.477953] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=19] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:42.478011] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=15] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061858725452000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:42.478052] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=40] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061858725452000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1001, freeze_info:{latest_snapshot_gc_scn:{val:1729061858725452000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}}) [2024-10-16 14:57:42.480272] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD00C-0-0] [lt=19] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061862471463, keepalive_ts:-1}) [2024-10-16 14:57:42.480286] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD00C-0-0] [lt=12] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061862471463, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:42.484811] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:42.484824] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=733, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:42.489482] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53B9-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:42.489493] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53B9-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=470, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:42.509862] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:42.509891] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:42.510170] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:42.510308] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42FC-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=309, avg_log_submit_to_slide_time=487) [2024-10-16 14:57:42.510399] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42FD-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:42.510406] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C42FD-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=665, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:42.514470] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:42.524269] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=13] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a837f0, tx_id:{txid:199664216}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061862523777, active_ts:1729061862523777, commit_ts:-1, finish_ts:1729061862523777, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061892523772, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061862523776, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:42.525149] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=55] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a85790, tx_id:{txid:199664217}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061862523777, active_ts:1729061862523777, commit_ts:-1, finish_ts:1729061862523777, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061892523772, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061862523776, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:42.528478] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.528591] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=57] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9631, avg_group_log_size=566) [2024-10-16 14:57:42.528609] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10102) [2024-10-16 14:57:42.529150] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106864][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD144-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10102) [2024-10-16 14:57:42.529499] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061862529013000, start_id=1729061862529013000, end_id=1729061862529013001) [2024-10-16 14:57:42.529986] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=258, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:42.529999] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9631) [2024-10-16 14:57:42.530010] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=133, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:42.530059] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:42.530068] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=779, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:42.530072] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=11, avg_log_gen_to_submit_time=42, avg_log_submit_to_flush_time=426, avg_log_submit_to_slide_time=737) [2024-10-16 14:57:42.530086] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:42.532947] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:42.532966] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:42.532981] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444529026, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:42.533975] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3422) [2024-10-16 14:57:42.534203] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3422) [2024-10-16 14:57:42.534228] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3422) [2024-10-16 14:57:42.534381] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C102B-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:42.534389] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C102B-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=778, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:42.535395] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061862535375000, start_id=1729061862535375000, end_id=1729061862535375001) [2024-10-16 14:57:42.537796] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=16) [2024-10-16 14:57:42.541888] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=906907648, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=166834176, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:42.541942] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=22] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:42.541954] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=11] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:42.541986] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=967602176, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=106139648, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:42.542023] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:42.542031] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:42.542063] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=12] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=942559232, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=131182592, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:42.542106] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:42.542113] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011863, drift_size_=0, pre_drift_size=1498290, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:42.542146] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=961036288, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=112705536, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:42.542183] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:42.542191] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:42.548768] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061862548018000, start_id=1729061862548018000, end_id=1729061862548018001) [2024-10-16 14:57:42.548794] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061862548790], gts:1729061862548018000, latest_srr:[mts=1729061862548790]}) [2024-10-16 14:57:42.564713] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7783-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:42.564988] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:42.568036] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.568747] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.569251] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.569306] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.569504] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.572128] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCADE-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061862571515000, start_id=1729061862571515000, end_id=1729061862571515001) [2024-10-16 14:57:42.579022] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061862578949000, start_id=1729061862578949000, end_id=1729061862578949001) [2024-10-16 14:57:42.583286] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:42.583460] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD22A-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=446, avg_log_submit_to_slide_time=689) [2024-10-16 14:57:42.584049] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:14538489394}, scn={val:1729061862144854005, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:42.584055] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=25] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:25, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061862584015, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538489450}, next_to_submit_scn_:{val:1729061862144854006, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538489338}, end_lsn:{lsn:14538489450}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061862144854005, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061862144854005, v:0}, data_checksum:757116148, flag:1}}, init_mode_version:0, accumulate_checksum:1620652196, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061862144854005, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650224, miss_cnt:32, cache_read_size:8050418535, hit_ratio:"9.999994892279450154e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}, replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14538489394}, scn_:{val:1729061862144854005, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061862574827, first_handle_ts_:1729061862584054, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f77c0e890, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:42.584213] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060061837115000) [2024-10-16 14:57:42.584224] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14538489394}, scn_:{val:1729061862144854005, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061862574827, first_handle_ts_:1729061862584201, replay_cost_:21, retry_cost_:-1, read_log_buf_:0x2b0f77c0e8a0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061862584015, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538489450}, next_to_submit_scn_:{val:1729061862144854006, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538489338}, end_lsn:{lsn:14538489450}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061862144854005, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061862144854005, v:0}, data_checksum:757116148, flag:1}}, init_mode_version:0, accumulate_checksum:1620652196, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061862144854005, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650224, miss_cnt:32, cache_read_size:8050418535, hit_ratio:"9.999994892279450154e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}, replay_queue_idx=11, ret=0) [2024-10-16 14:57:42.585080] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1002}, {id:1003}]) [2024-10-16 14:57:42.585514] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C4-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:42.585534] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C4-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061862143780004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061862143780004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:42.587716] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:42.587733] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:42.587786] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9C4-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061862587386000, start_id=1729061862587386000, end_id=1729061862587386001) [2024-10-16 14:57:42.590290] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22483][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C4-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061862589504000, start_id=1729061862589504000, end_id=1729061862589504001) [2024-10-16 14:57:42.590750] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=5] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:42.599543] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:42.601725] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061862601249000, start_id=1729061862601249000, end_id=1729061862601249001) [2024-10-16 14:57:42.609237] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5019, mem_used=10398720) [2024-10-16 14:57:42.610174] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=223, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:42.610232] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=53] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=61, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:42.610240] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:42.617781] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061862617099000, start_id=1729061862617099000, end_id=1729061862617099001) [2024-10-16 14:57:42.626937] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C51-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:42.626964] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C51-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061862571515000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061862571515000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:42.628061] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C51-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061862627628000, start_id=1729061862627628000, end_id=1729061862627628001) [2024-10-16 14:57:42.629367] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.630723] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C51-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061862629749000, start_id=1729061862629749000, end_id=1729061862629749001) [2024-10-16 14:57:42.644356] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9D4-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061862643564000, start_id=1729061862643564000, end_id=1729061862643564001) [2024-10-16 14:57:42.644526] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9D8-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061862643564000, start_id=1729061862643564000, end_id=1729061862643564001) [2024-10-16 14:57:42.649046] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DE4-0-0] [lt=21] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:42.649064] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DE4-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:42.649072] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DE4-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:42.649722] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=15] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:57:42.660420] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:57:42.660434] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=12] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:42.660441] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=111, cache_node_num=111) [2024-10-16 14:57:42.660447] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:42.662612] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=111, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:57:42.662761] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=146] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:42.667693] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:42.668094] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:42.668111] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:42.668116] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.670673] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:42.673819] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=9] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:57:42.674628] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061862674401000, start_id=1729061862674401000, end_id=1729061862674401001) [2024-10-16 14:57:42.685772] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D85-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510330554165, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410600, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:42.686463] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D89-0-0] [lt=17][errcode=-4201] hashtable not init [2024-10-16 14:57:42.686473] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D89-0-0] [lt=9][errcode=-4016] get cursor info failed(cursor_id=1114, get_sessid()=3221565766) [2024-10-16 14:57:42.686559] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D8A-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061862686009000, start_id=1729061862686009000, end_id=1729061862686009001) [2024-10-16 14:57:42.689142] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FB5-0-0] [lt=11][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497425, log_task={header:{begin_lsn:{lsn:207589548545}, end_lsn:{lsn:207589548667}, log_id:218497425, min_scn:{val:1729061862143780005, v:0}, max_scn:{val:1729061862143780005, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589548423}, prev_proposal_id:81, committed_end_lsn:{lsn:207589548545}, data_checksum:940009244, accum_checksum:3723667452, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061862589053, freeze_ts:1729061862589053, submit_ts:1729061862589056, flushed_ts:1729061862590048, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:992}, fs_cb_begin_ts=1729061862689136, log_life_time=100083) [2024-10-16 14:57:42.692247] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D41-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061862683470, keepalive_ts:-1}) [2024-10-16 14:57:42.692277] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D41-0-0] [lt=30] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061862683470, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:42.692302] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D42-0-0] [lt=9] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061862683515, keepalive_ts:-1}) [2024-10-16 14:57:42.692311] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D42-0-0] [lt=9] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061862683515, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:42.698740] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:42.711404] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=23, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:57:42.711418] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=13] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=1760, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:57:42.711422] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:57:42.711425] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=-1191, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:57:42.711428] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=593, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:57:42.712929] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=10] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:42.714537] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:42.724613] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:42.729873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061862729419000, start_id=1729061862729419000, end_id=1729061862729419001) [2024-10-16 14:57:42.730026] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.735593] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061862734712000, start_id=1729061862734712000, end_id=1729061862734712001) [2024-10-16 14:57:42.749380] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488D-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061862717136, keepalive_ts:-1}) [2024-10-16 14:57:42.749403] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106864][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488D-0-0] [lt=22] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061862717136, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:42.756509] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=4] gc stale ls task succ [2024-10-16 14:57:42.756536] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:42.756544] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:42.768155] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.771285] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:42.774747] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C65-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=12, avg_log_batch_cnt=0, total_group_log_size=1454, avg_group_log_size=121) [2024-10-16 14:57:42.778343] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1, min_version={val:1729061862378674001, v:0}, base_version={val:1729061862378674000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061862578949000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061862778329, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061862578949000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061862715921, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061862378674001, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061862668715, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:42.779231] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061862776811000, start_id=1729061862776811000, end_id=1729061862776811001) [2024-10-16 14:57:42.779921] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:42.779948] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=21] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:42.779956] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=51) [2024-10-16 14:57:42.781722] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:42.785323] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=239, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:42.785335] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=890, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:42.785447] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:42.785463] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1394, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:42.785468] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:42.787881] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788530] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788683] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788684] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788693] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788708] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788885] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788901] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788916] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788929] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.788942] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:42.789409] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:42.799837] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:42.801812] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061862801608000, start_id=1729061862801608000, end_id=1729061862801608001) [2024-10-16 14:57:42.802587] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get_min_unreplayed_log_info(lsn={lsn:207589548911}, scn={val:1729061862643564002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061862789665, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589548911}, next_to_submit_scn_:{val:1729061862643564002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589548789}, end_lsn:{lsn:207589548911}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061862643564001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061862643564001, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:3554926553, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061862643564001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598875, miss_cnt:9, cache_read_size:149777913352, hit_ratio:"9.999999345924928118e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:57:42.810521] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:42.816348] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:42.816400] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1 and flag like "%DUPLICATE%", tenant_id=1, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:42.816476] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=11][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:42.816521] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=11][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1001 and flag like "%DUPLICATE%", tenant_id=1001, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:42.817706] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22795][T1004_TenantWea][T1003][YB4A0A000407-00061E9B3AFB964C-0-0] [lt=16] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.079256725311279297e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=303, Throttle Trigger(MB)=182, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.040887371745630389e+03) [2024-10-16 14:57:42.817858] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061862816804000, start_id=1729061862816804000, end_id=1729061862816804001) [2024-10-16 14:57:42.825482] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1001}, {id:1003}]) [2024-10-16 14:57:42.828657] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6E-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:42.828689] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6E-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061862644611001, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061862644611001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:42.829754] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D6E-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061862828862000, start_id=1729061862828862000, end_id=1729061862828862001) [2024-10-16 14:57:42.830049] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.832330] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6E-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061862832043000, start_id=1729061862832043000, end_id=1729061862832043001) [2024-10-16 14:57:42.832404] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6E-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:42.832429] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6E-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:42.845036] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:42.849608] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=54] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=33, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:42.849622] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:42.853111] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4568, mem_used=10398720) [2024-10-16 14:57:42.865076] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1816) [2024-10-16 14:57:42.868178] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.875040] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:42.883226] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:42.883241] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:42.889247] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FB9-0-0] [lt=23] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:42.901472] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=14] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:42.901943] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:42.902007] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1004 and flag like "%DUPLICATE%", tenant_id=1004, exec_tenant_id=1003, need_member_list=false) [2024-10-16 14:57:42.902615] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061852899162, end_time=1729061862902614, dag_status={}) [2024-10-16 14:57:42.902628] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:42.902635] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:42.902639] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:42.902645] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:42.902649] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:42.902655] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:42.902668] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:42.902671] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:42.902678] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:42.902681] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902690] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902694] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902701] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902705] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902712] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902716] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902720] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902724] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902731] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902735] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902746] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902750] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902754] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902763] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902767] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902771] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902778] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902782] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902788] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902798] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902802] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902806] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902810] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902817] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902821] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902828] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902832] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902839] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902843] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902849] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902854] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902861] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902865] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902872] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902876] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902883] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902886] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902890] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902895] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902899] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902904] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902908] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902911] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902915] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902919] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902923] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902926] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902930] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902933] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902937] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902941] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902944] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902948] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902952] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902955] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902959] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902967] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:42.902971] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:42.902977] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:42.902984] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:42.902986] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:42.902992] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:42.902994] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:42.902999] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:42.903002] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:42.903007] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:42.903010] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:42.903017] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:42.910111] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:42.910125] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:42.911693] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=11] worker thread began to shrink(tenant_->id()=1003, group_id_=19, token=0) [2024-10-16 14:57:42.911747] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:42.911755] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=707, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:42.913258] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=28] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:42.914565] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:42.915936] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106820][T1003_L0_G19][T1003][Y0-0000000000000000-0-0] [lt=26] worker thread exit(tenant_->id()=1003, workers_.get_size()=1) [2024-10-16 14:57:42.915952] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106820][T1003_L0_G19][T0][Y0-0000000000000000-0-0] [lt=12] tenant thread end_run(id_=1003, ret=0, thread_count_=171) [2024-10-16 14:57:42.922266] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:42.929115] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=36] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:42.929676] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061862928799000, start_id=1729061862928799000, end_id=1729061862928799001) [2024-10-16 14:57:42.929792] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:42.929805] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=26999180, tenant_id=1004) [2024-10-16 14:57:42.929814] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:42.929981] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBDF-0-0] [lt=15] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061862897714, keepalive_ts:-1}) [2024-10-16 14:57:42.929996] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBDF-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061862897714, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:42.930115] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.930700] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107484][][T1003][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1003, ret=0, thread_count_=172) [2024-10-16 14:57:42.930717] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107484][][T1003][Y0-0000000000000000-0-0] [lt=16] Init thread local success [2024-10-16 14:57:42.930742] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107484][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2015-0-0] [lt=4] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:42.930752] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107484][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2015-0-0] [lt=7] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:42.930851] INFO [SERVER.OMT] recv_group_request (ob_tenant.cpp:1419) [21191][pnio1][T0][YB4A0A000408-00061E9B44BE2015-0-0] [lt=4] worker thread created(id()=1003, group->group_id_=19) [2024-10-16 14:57:42.932055] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107485][][T1003][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1003, ret=0, thread_count_=173) [2024-10-16 14:57:42.932067] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107485][][T1003][Y0-0000000000000000-0-0] [lt=11] Init thread local success [2024-10-16 14:57:42.932086] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD14D-0-0] [lt=3] init point thread id with(&point=0x55e3435ca600, idx_=990, point=[thread id=107485, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107485) [2024-10-16 14:57:42.932268] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] worker thread created(tenant_->id()=1003, group_id_=1, token=1) [2024-10-16 14:57:42.933826] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=16] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:42.935717] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061862935632000, start_id=1729061862935632000, end_id=1729061862935632001) [2024-10-16 14:57:42.942236] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=2909, mem_used=6239232) [2024-10-16 14:57:42.946749] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:57:42.946762] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=10] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:42.946768] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:42.946773] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:42.947877] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=21] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:42.949021] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:57:42.949412] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=389] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:42.950344] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=11] ====== tenant freeze timer task ====== [2024-10-16 14:57:42.957545] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:42.957566] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=14] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:42.957573] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=159) [2024-10-16 14:57:42.968310] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:42.979216] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061862978287000, start_id=1729061862978287000, end_id=1729061862978287001) [2024-10-16 14:57:42.982840] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15DA-0-0] [lt=11] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:42.984579] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=200, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:42.984591] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=450, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:42.984685] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE92-0-0] [lt=15] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=1165, avg_log_submit_to_slide_time=1329) [2024-10-16 14:57:42.985036] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:42.985830] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:42.985858] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:42.985878] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=16] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:42.985943] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=14] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061879029444000, latest_log_ts:{val:1729061862417342001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:42.986002] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=56] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:42.986026] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=15] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:57:42.986034] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061862417342001, v:0}) [2024-10-16 14:57:42.986038] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:42.986041] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:42.986047] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:42.986057] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:57:42.986064] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:42.986073] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:42.994623] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=17] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:42.995080] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=39] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 323.00; ) [2024-10-16 14:57:42.995106] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:42.995141] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 133.50; ) [2024-10-16 14:57:42.995150] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:42.995162] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:43.000127] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:43.002000] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=89] get number(ret=0, service_type_=0, range=1, base_id=1729061863000594000, start_id=1729061863000594000, end_id=1729061863000594001) [2024-10-16 14:57:43.002698] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO [ratelimit] time: 1729061863002696, bytes: 4446814583053, bw: 0.363890 MB/s, add_ts: 1001310, add_bytes: 382066 [2024-10-16 14:57:43.003071] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Cache wash clean map node details(ret=0, clean_node_count=32, clean_time=0, clean_start_pos=1069538, clean_num=62914) [2024-10-16 14:57:43.004186] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=10] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:43.010185] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:43.010200] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:43.010539] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=213, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:43.010550] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=170, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:43.010559] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:43.010793] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F412A-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=408, avg_log_submit_to_slide_time=704) [2024-10-16 14:57:43.015316] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21574][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:43.015367] WDIAG read_multi_row (ob_table_access_helper.h:238) [21574][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:43.016057] INFO delete_row (ob_table_access_helper.h:312) [21574][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=8] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061863015377) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061863015377) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:43.017890] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061863017342000, start_id=1729061863017342000, end_id=1729061863017342001) [2024-10-16 14:57:43.030176] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.042436] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:43.042452] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:43.042465] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291931270, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:43.047982] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=15) [2024-10-16 14:57:43.049181] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061863049052000, start_id=1729061863049052000, end_id=1729061863049052001) [2024-10-16 14:57:43.049203] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=662, get_gts_with_stc_cnt=30, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=30, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:43.049211] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:43.049229] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=520, get_gts_with_stc_cnt=11, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:43.049233] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=4] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061863049226], gts:1729061863049052000, latest_srr:[mts=1729061863049226]}) [2024-10-16 14:57:43.064990] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE778B-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1938, avg_group_log_size=176) [2024-10-16 14:57:43.068373] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.071676] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:43.072973] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=39] PNIO [ratelimit] time: 1729061863072971, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007117, add_bytes: 0 [2024-10-16 14:57:43.083959] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:43.083989] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:43.089373] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FBD-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:43.089584] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=258, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:43.089605] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=38, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:43.089638] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:43.094783] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:43.094795] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151784220, tenant_id=1) [2024-10-16 14:57:43.094809] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:43.114637] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:43.116763] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:43.116782] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:43.122810] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=15] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97915322368) [2024-10-16 14:57:43.129545] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:43.129881] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061863129765000, start_id=1729061863129765000, end_id=1729061863129765001) [2024-10-16 14:57:43.130742] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.130798] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2405, mem_used=6239232) [2024-10-16 14:57:43.135636] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061863135057000, start_id=1729061863135057000, end_id=1729061863135057001) [2024-10-16 14:57:43.135694] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:43.135708] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:43.135920] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=474, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:43.135937] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=30, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:43.135952] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:43.136168] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94E8-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10282) [2024-10-16 14:57:43.136222] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94E8-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:43.136234] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94E8-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=894, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:43.136238] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94E8-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=583, avg_log_submit_to_slide_time=863) [2024-10-16 14:57:43.144550] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9D9-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061863144520000, start_id=1729061863144520000, end_id=1729061863144520001) [2024-10-16 14:57:43.144720] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9DD-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061863144520000, start_id=1729061863144520000, end_id=1729061863144520001) [2024-10-16 14:57:43.144940] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9DB-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061863144520000, start_id=1729061863144520000, end_id=1729061863144520001) [2024-10-16 14:57:43.144946] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B9DC-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061863144520000, start_id=1729061863144520000, end_id=1729061863144520001) [2024-10-16 14:57:43.144968] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B9DA-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061863144520000, start_id=1729061863144520000, end_id=1729061863144520001) [2024-10-16 14:57:43.166561] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA881-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:43.166595] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA881-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061862643564004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061862643564004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:43.168433] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.170194] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA881-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061863169816000, start_id=1729061863169816000, end_id=1729061863169816001) [2024-10-16 14:57:43.170269] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA881-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9691, avg_group_log_size=570) [2024-10-16 14:57:43.170286] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA881-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9691) [2024-10-16 14:57:43.170527] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10446) [2024-10-16 14:57:43.170824] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:43.175408] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE97-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:43.175420] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE97-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9988, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:43.175425] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FE97-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=548, avg_log_submit_to_slide_time=9982) [2024-10-16 14:57:43.175424] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:43.175444] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650230, miss_cnt:32, cache_read_size:8050419267, hit_ratio:"9.999994892279939762e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:43.179373] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061863179351000, start_id=1729061863179351000, end_id=1729061863179351001) [2024-10-16 14:57:43.179432] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=3855, avg_group_log_size=428) [2024-10-16 14:57:43.179650] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=201, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:43.179671] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=60, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:43.179687] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:43.179902] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45BFA-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=279, avg_log_submit_to_slide_time=1113) [2024-10-16 14:57:43.184303] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:43.187386] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502D-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:43.187432] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502D-0-0] [lt=44] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:43.189443] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FBF-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:43.189913] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:43.189936] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598879, miss_cnt:9, cache_read_size:149777913840, hit_ratio:"9.999999345924946992e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:43.191352] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601410, offset=9627894}, end_cursor:ObLogCursor{file_id=192, log_id=47601411, offset=9628178}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e68050, len:1802, log_data_len:284, seq:47601410, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:43.193138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A5-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061863192647000, start_id=1729061863192647000, end_id=1729061863192647001) [2024-10-16 14:57:43.194414] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C103F-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061863193715000, start_id=1729061863193715000, end_id=1729061863193715001) [2024-10-16 14:57:43.194958] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601411, offset=9628178}, end_cursor:ObLogCursor{file_id=192, log_id=47601412, offset=9628452}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d28050, len:1518, log_data_len:274, seq:47601411, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:43.196720] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257758, offset=39031902}, end_cursor:ObLogCursor{file_id=197, log_id=49257759, offset=39032185}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c30050, len:2978, log_data_len:283, seq:49257758, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:43.198870] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB370-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061863197951000, start_id=1729061863197951000, end_id=1729061863197951001) [2024-10-16 14:57:43.199012] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601412, offset=9628452}, end_cursor:ObLogCursor{file_id=192, log_id=47601413, offset=9628726}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e64050, len:1244, log_data_len:274, seq:47601412, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:43.200635] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257759, offset=39032185}, end_cursor:ObLogCursor{file_id=197, log_id=49257760, offset=39032467}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d6c050, len:2695, log_data_len:282, seq:49257759, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:43.200698] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601413, offset=9628726}, end_cursor:ObLogCursor{file_id=192, log_id=47601414, offset=9629000}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e1c050, len:970, log_data_len:274, seq:47601413, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:43.200899] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561292, miss_cnt:1, cache_read_size:152590680329, hit_ratio:"9.999999928346894329e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:43.202060] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061863200181000, start_id=1729061863200181000, end_id=1729061863200181001) [2024-10-16 14:57:43.203395] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1179615232, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:43.203826] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257760, offset=39032467}, end_cursor:ObLogCursor{file_id=197, log_id=49257761, offset=39032749}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7362050, len:2413, log_data_len:282, seq:49257760, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:43.204055] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257761, offset=39032749}, end_cursor:ObLogCursor{file_id=197, log_id=49257762, offset=39033031}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d7c050, len:2131, log_data_len:282, seq:49257761, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:43.204139] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656967, miss_cnt:17, cache_read_size:8050166648, hit_ratio:"9.999997286814826802e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:43.210865] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587A-0-0] [lt=16] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:43.211004] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587A-0-0] [lt=16][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:43.212554] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D8B-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510331080947, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410601, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:43.213002] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4130-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:43.218041] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061863217009000, start_id=1729061863217009000, end_id=1729061863217009001) [2024-10-16 14:57:43.230805] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.246539] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD0B-0-0] [lt=8][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:43.249581] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=228, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:43.249611] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=72, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:43.249647] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:43.249837] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9141-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:43.249845] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9141-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=615, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:43.249849] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9141-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=326, avg_log_submit_to_slide_time=608) [2024-10-16 14:57:43.249853] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:43.265067] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE778F-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:43.265598] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53C8-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:43.265614] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53C8-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10436, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:43.265626] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] succ to submit log task to replay service(to_submit_lsn={lsn:213058240403}, to_submit_scn={val:1729061863144520002, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061863265593, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058240469}, next_to_submit_scn_:{val:1729061863144520003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058240347}, end_lsn:{lsn:213058240469}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061863144520002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061863144520002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:778763089, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061863144520002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561293, miss_cnt:1, cache_read_size:152590680451, hit_ratio:"9.999999928346895439e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:43.268498] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.269649] INFO [MVCC] repeat_study (ob_multi_version_garbage_collector.cpp:196) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=8] skip repeat study(ret=0, this={this:0x2b0ddc5fcf30, last_study_timestamp:1729061503190166, last_refresh_timestamp:1729061803261227, last_reclaim_timestamp:1729061443173532, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061502727285000, v:0}, is_inited:true}, current_timestamp=1729061863269644, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:43.270382] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=23] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1729061503187285000, snapshot_type=1, create_time=1729061503187775, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:43.270399] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=12] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1729061747023878000, snapshot_type=1, create_time=1729061747026462, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:43.270409] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=8] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1729061611605251000, snapshot_type=1, create_time=1729061611584374, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:43.270418] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=6] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1729061502727285000, snapshot_type=2, create_time=1729061503187775, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:43.270426] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=7] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1729061746773113000, snapshot_type=2, create_time=1729061747026462, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:43.270434] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=6] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1729061611311218000, snapshot_type=2, create_time=1729061611584374, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:43.270440] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1729061503027460000, snapshot_type=3, create_time=1729061503187775, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:43.270448] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=7] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1729061747023878000, snapshot_type=3, create_time=1729061747026462, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:43.270453] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1729061611510858000, snapshot_type=3, create_time=1729061611584374, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:43.270461] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=6] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061503187775, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:43.270469] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=6] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061747026462, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:43.270477] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=6] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061611584374, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:43.270580] INFO [MVCC] decide_reserved_snapshot_version_ (ob_multi_version_garbage_collector.cpp:618) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=6] succeed to update global reserved snapshot(ret=0, this={this:0x2b0ddc5fcf30, last_study_timestamp:1729061503190166, last_refresh_timestamp:1729061803261227, last_reclaim_timestamp:1729061443173532, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061502727285000, v:0}, is_inited:true}, global_reserved_snapshot_={val:1729061502727285000, v:0}, reserved_snapshot={val:1729061502727285000, v:0}) [2024-10-16 14:57:43.270594] INFO [MVCC] refresh_ (ob_multi_version_garbage_collector.cpp:552) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=13] multi-version garbage collector refresh successfully(this={this:0x2b0ddc5fcf30, last_study_timestamp:1729061503190166, last_refresh_timestamp:1729061803261227, last_reclaim_timestamp:1729061443173532, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061502727285000, v:0}, is_inited:true}, collector={reserved_snapshot_version:{val:1729061502727285000, v:0}, reserved_snapshot_type:2, reserved_status:-1, reserved_create_time:1729061503187775, reserved_addr:"10.0.4.7:2890", is_this_server_disabled:false, status:0}) [2024-10-16 14:57:43.270616] INFO [MVCC] repeat_refresh (ob_multi_version_garbage_collector.cpp:231) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=22] repeat refresh successfully(ret=0, this={this:0x2b0ddc5fcf30, last_study_timestamp:1729061503190166, last_refresh_timestamp:1729061863270616, last_reclaim_timestamp:1729061443173532, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061502727285000, v:0}, is_inited:true}, current_timestamp=1729061863269673, GARBAGE_COLLECT_RETRY_INTERVAL=60000000) [2024-10-16 14:57:43.270625] INFO [MVCC] repeat_reclaim (ob_multi_version_garbage_collector.cpp:266) [21611][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=8] skip repeat reclaim(ret=0, this={this:0x2b0ddc5fcf30, last_study_timestamp:1729061503190166, last_refresh_timestamp:1729061863270616, last_reclaim_timestamp:1729061443173532, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061502727285000, v:0}, is_inited:true}, current_timestamp=1729061863270624, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:43.275078] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C6F-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:43.275268] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=193, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:43.275287] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=333, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:43.275324] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:43.278584] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB99-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12287, add_timestamp:1729061863278573}) [2024-10-16 14:57:43.278605] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E42-0-0] [lt=11] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12287, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:43.278764] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E42-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12287, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061863278751, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061863278751, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061863278751, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:43.278783] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E42-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:12287, add_timestamp:1729061863278573}, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061863278751, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061863278751, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061863278751, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:43.282188] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] no need to update min/max version(new_version={val:1729061862776811001, v:0}, min_version={val:1729061862776811001, v:0}, max_version={val:1729061862776811001, v:0}) [2024-10-16 14:57:43.283084] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB99-0-0] [lt=16][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:43.284081] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:43.284101] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:43.284225] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:43.284234] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:43.284479] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:43.284485] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:43.285273] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=24] start to print blacklist info [2024-10-16 14:57:43.285439] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=11] ls blacklist refresh finish(cost_time=7426) [2024-10-16 14:57:43.290246] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1037-0-0] [lt=8] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:43.311174] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:43.311190] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:43.311446] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:43.314715] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:43.317462] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7173, mem_used=14558208) [2024-10-16 14:57:43.329181] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:43.329924] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061863328952000, start_id=1729061863328952000, end_id=1729061863328952001) [2024-10-16 14:57:43.330543] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1729061863129765000, v:0}, new_max_version={val:1729061863129765000, v:0}, last_min_version={val:1729061862928799000, v:0}, last_max_version={val:1729061862928799000, v:0}, total_time_us=1373, delta_us=200777) [2024-10-16 14:57:43.330558] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1729061863129765000, v:0}, min_version={val:1729061863129765000, v:0}, max_version={val:1729061863129765000, v:0}, in_service=true, leader_epoch=33576) [2024-10-16 14:57:43.330865] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.332805] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [22251][T1002_SSTableGC][T1002][YB4A0A000407-00061E997A3CB6A5-0-0] [lt=11] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60379}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015250305282004}, old_min_reserved_snapshot=1729060060198021000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060050192549000) [2024-10-16 14:57:43.334324] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:43.334345] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:43.335792] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061863334399000, start_id=1729061863334399000, end_id=1729061863334399001) [2024-10-16 14:57:43.360094] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [22251][T1002_SSTableGC][T1002][YB4A0A000407-00061E997A3CB6A5-0-0] [lt=18] SSTableGCTask(cost_ts=27461) [2024-10-16 14:57:43.365556] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=51] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1816, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:43.365609] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:43.365624] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=25] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:43.368351] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAE-0-0] [lt=12] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:43.368368] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAE-0-0] [lt=13] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=18) [2024-10-16 14:57:43.368561] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.373358] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=15] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:43.375102] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C71-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:43.383161] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061863382436000, start_id=1729061863382436000, end_id=1729061863382436001) [2024-10-16 14:57:43.384350] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:43.389556] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FC3-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497432, log_task={header:{begin_lsn:{lsn:207589549399}, end_lsn:{lsn:207589549521}, log_id:218497432, min_scn:{val:1729061863144520001, v:0}, max_scn:{val:1729061863144520001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589549277}, prev_proposal_id:81, committed_end_lsn:{lsn:207589549399}, data_checksum:940009244, accum_checksum:2824223689, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061863289504, freeze_ts:1729061863289504, submit_ts:1729061863289508, flushed_ts:1729061863291149, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1641}, fs_cb_begin_ts=1729061863389549, log_life_time=100045) [2024-10-16 14:57:43.394546] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:43.396328] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=6] query dest_info_ succ(dest_info=[], archive_dest_changed=false) [2024-10-16 14:57:43.396352] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=16] finish to resize log external storage handler(current_ts=1729061863396344, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:43.402283] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061863401496000, start_id=1729061863401496000, end_id=1729061863401496001) [2024-10-16 14:57:43.403010] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:43.403693] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:43.418226] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061863418159000, start_id=1729061863418159000, end_id=1729061863418159001) [2024-10-16 14:57:43.418856] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:43.430929] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.449717] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:43.465686] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=207, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:43.465712] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:43.468629] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.473014] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A81-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:43.473032] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A81-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:43.473503] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:43.484861] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FEA1-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:43.484882] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FEA1-0-0] [lt=20] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=783, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:43.487945] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:43.487956] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=11] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:43.487966] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:43.487964] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED5-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:43.487975] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:43.487979] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:43.487982] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:43.487980] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED5-0-0] [lt=14] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:43.487985] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:43.487987] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:43.487990] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:43.487992] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:43.487995] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:43.490042] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:43.490054] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:43.490067] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=70698190, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:43.490077] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53CD-0-0] [lt=29] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:43.490088] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53CD-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20373, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:43.495321] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480E-0-0] [lt=9] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061863463080, keepalive_ts:-1}) [2024-10-16 14:57:43.495339] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480E-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061863463080, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:43.502711] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] Garbage Collector is running(seq_=656495, gc_interval=10000000) [2024-10-16 14:57:43.502743] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=14] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:43.502753] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:43.503019] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:43.503276] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:43.503289] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] gc_check_member_list_ cost time(ret=0, time_us=564) [2024-10-16 14:57:43.503302] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:43.504142] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=7] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:43.504811] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=14] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:43.505429] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=14] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:43.506036] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=16] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:43.506052] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=13] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:43.506060] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:43.506066] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:43.506080] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=14] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:43.506085] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=34) [2024-10-16 14:57:43.506090] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:43.507553] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=7] ====== checkpoint timer task ====== [2024-10-16 14:57:43.508355] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=17] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:43.508385] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=29] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:43.508407] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:43.508474] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=10] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061875590307000, latest_log_ts:{val:1729061858932674001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:43.508531] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=53] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:43.508554] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=19] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:43.508564] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061858932674001, v:0}) [2024-10-16 14:57:43.508568] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:43.508575] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:43.508581] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:43.508594] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:43.508613] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=18] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:43.508620] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:43.511249] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:43.511269] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:43.511536] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:43.511707] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C431A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=466, avg_log_submit_to_slide_time=780) [2024-10-16 14:57:43.511739] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C431B-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:43.511745] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C431B-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=918, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:43.514831] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:43.517495] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:43.517516] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:14538490548}, scn={val:1729061863144520005, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061863475716, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538490548}, next_to_submit_scn_:{val:1729061863144520005, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538490426}, end_lsn:{lsn:14538490548}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061863144520004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061863144520004, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:3520091674, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061863144520004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650233, miss_cnt:32, cache_read_size:8050419633, hit_ratio:"9.999994892280184011e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:43.517561] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=43] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:43.530116] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061863529255000, start_id=1729061863529255000, end_id=1729061863529255001) [2024-10-16 14:57:43.530187] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=11113, avg_group_log_size=653) [2024-10-16 14:57:43.530196] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10642) [2024-10-16 14:57:43.530490] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=411, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:43.530504] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11113) [2024-10-16 14:57:43.530515] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=66, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:43.530533] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10642) [2024-10-16 14:57:43.530567] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=31] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:43.530857] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C69-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:43.530865] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C69-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=858, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:43.530868] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C69-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=31, avg_log_gen_to_submit_time=60, avg_log_submit_to_flush_time=532, avg_log_submit_to_slide_time=797) [2024-10-16 14:57:43.530994] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.534385] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3856) [2024-10-16 14:57:43.534593] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3856) [2024-10-16 14:57:43.534621] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3856) [2024-10-16 14:57:43.534815] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1034-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:43.534822] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1034-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1166, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:43.535929] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061863535586000, start_id=1729061863535586000, end_id=1729061863535586001) [2024-10-16 14:57:43.549551] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061863549302000, start_id=1729061863549302000, end_id=1729061863549302001) [2024-10-16 14:57:43.549627] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=18] submit log success(service_type_=0, last_id=1729061880098662000, limited_id=1729061880098662000) [2024-10-16 14:57:43.549634] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=7] submit log with lock success(service_type_=0, last_id=1729061880098662000, limited_id=1729061880098662000) [2024-10-16 14:57:43.549792] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=3] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061863549782], gts:1729061863549302000, latest_srr:[mts=1729061863549782]}) [2024-10-16 14:57:43.550084] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061880098662000, latest_scn={val:1729061863549302001, v:0}, *this={tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880098662000, latest_log_ts:{val:1729061863549302001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:43.550134] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=49] handle submit callback(service_type_=0, success=true, used_ts=507, log_ts={val:1729061863549302001, v:0}, limited_id=1729061880098662000) [2024-10-16 14:57:43.550140] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:43.550147] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=7] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:43.568621] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.568687] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.569029] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.569042] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.569695] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.569743] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:43.581922] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26148][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB19CC6EC-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:43.583146] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061863582159000, start_id=1729061863582159000, end_id=1729061863582159001) [2024-10-16 14:57:43.584490] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:43.584782] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FEA4-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=513, avg_log_submit_to_slide_time=768) [2024-10-16 14:57:43.585183] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1003}, {id:1002}]) [2024-10-16 14:57:43.587839] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:43.587859] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:43.588076] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=10] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:57:43.592377] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C5-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:43.592400] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C5-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061863144520003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061863144520003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:43.593536] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9C5-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061863592721000, start_id=1729061863592721000, end_id=1729061863592721001) [2024-10-16 14:57:43.596207] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C5-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061863595905000, start_id=1729061863595905000, end_id=1729061863595905001) [2024-10-16 14:57:43.596773] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:43.602429] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061863602230000, start_id=1729061863602230000, end_id=1729061863602230001) [2024-10-16 14:57:43.603949] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:43.606494] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0E-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061863597724, keepalive_ts:-1}) [2024-10-16 14:57:43.606510] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0E-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061863597724, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:43.609351] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5034, mem_used=10398720) [2024-10-16 14:57:43.618381] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061863616934000, start_id=1729061863616934000, end_id=1729061863616934001) [2024-10-16 14:57:43.619027] INFO [STORAGE.COMPACTION] schedule_all_tablets_minor (ob_tenant_tablet_scheduler.cpp:553) [22807][T1004_MergeLoop][T1004][YB4A0A000407-00061E9980FA366C-0-0] [lt=13] start schedule all tablet minor merge(minor_ls_tablet_iter_={ls_idx:-1, ls_ids:[{id:1}, {id:1001}, {id:1002}, {id:1003}], tablet_idx:0, tablet_ids_.count():0, schedule_tablet_cnt:0, max_batch_tablet_cnt:50000}) [2024-10-16 14:57:43.631053] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.633860] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C52-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:43.633878] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C52-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061863144520004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061863144520004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:43.636635] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C52-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061863636514000, start_id=1729061863636514000, end_id=1729061863636514001) [2024-10-16 14:57:43.640323] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C52-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061863640068000, start_id=1729061863640068000, end_id=1729061863640068001) [2024-10-16 14:57:43.642225] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:33216819320}, base_lsn={lsn:33149755392}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:43.642281] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=56] get_replay_process success(id={id:1}, replayed_log_size=67063928, unreplayed_log_size=0) [2024-10-16 14:57:43.642299] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=63, last_replayed_log_size(MB)=63, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:43.645064] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9E0-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061863644987000, start_id=1729061863644987000, end_id=1729061863644987001) [2024-10-16 14:57:43.645633] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0C69-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061863644987000, start_id=1729061863644987000, end_id=1729061863644987001) [2024-10-16 14:57:43.645798] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0C6D-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061863644987000, start_id=1729061863644987000, end_id=1729061863644987001) [2024-10-16 14:57:43.650671] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DF8-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:43.650705] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DF8-0-0] [lt=27] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:43.650716] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DF8-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:43.650733] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53DF8-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173169, log_task={header:{begin_lsn:{lsn:14352376784}, end_lsn:{lsn:14352376906}, log_id:115173169, min_scn:{val:1729061863144520004, v:0}, max_scn:{val:1729061863144520004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352376662}, prev_proposal_id:17, committed_end_lsn:{lsn:14352376784}, data_checksum:940009244, accum_checksum:3120756601, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061863549476, freeze_ts:1729061863549476, submit_ts:1729061863549481, flushed_ts:1729061863550175, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:694}, fs_cb_begin_ts=1729061863650725, log_life_time=101249) [2024-10-16 14:57:43.653208] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:85) [22807][T1004_MergeLoop][T1004][YB4A0A000407-00061E9980FA366C-0-0] [lt=14] MergeLoopTask(cost_ts=34216) [2024-10-16 14:57:43.660870] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:43.660885] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=12] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:43.660891] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=104, cache_node_num=104) [2024-10-16 14:57:43.660896] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:43.663013] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=7] Dumping All Cache Objs(alloc_obj_list.count()=104, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}]) [2024-10-16 14:57:43.663147] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=132] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:43.665383] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7797-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1938) [2024-10-16 14:57:43.665403] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7797-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938288, log_task={header:{begin_lsn:{lsn:213058240713}, end_lsn:{lsn:213058240835}, log_id:220938288, min_scn:{val:1729061863144520005, v:0}, max_scn:{val:1729061863144520005, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058240591}, prev_proposal_id:88, committed_end_lsn:{lsn:213058240713}, data_checksum:1715074492, accum_checksum:3143251291, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061863565220, freeze_ts:1729061863565220, submit_ts:1729061863565225, flushed_ts:1729061863569721, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:4496}, fs_cb_begin_ts=1729061863665398, log_life_time=100178) [2024-10-16 14:57:43.667787] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:43.668747] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.682321] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=9] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:43.683108] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061863682118000, start_id=1729061863682118000, end_id=1729061863682118001) [2024-10-16 14:57:43.684026] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] no need to update min/max version(new_version={val:1729061863382436000, v:0}, min_version={val:1729061863382436000, v:0}, max_version={val:1729061863382436000, v:0}) [2024-10-16 14:57:43.684221] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:43.684232] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:57:43.684235] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:43.684251] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] get_max_applied_scn(scn={val:1729061863549302000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493471163}, palf_committed_end_scn_:{val:1729061863549302000, v:0}, last_check_scn_:{val:1729061863682118000, v:0}, max_applied_cb_scn_:{val:1729061863549302000, v:0}}) [2024-10-16 14:57:43.684266] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:43.684922] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=47] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061870721265000, latest_log_ts:{val:1729061854137257001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:43.684986] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=44] get_max_applied_scn(scn={val:1729061863549302000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645421066}, palf_committed_end_scn_:{val:1729061863549302000, v:0}, last_check_scn_:{val:1729061863549302000, v:0}, max_applied_cb_scn_:{val:1729061863549302000, v:0}}) [2024-10-16 14:57:43.684995] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:43.685630] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:43.686293] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:43.686327] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] get_min_unreplayed_log_info(lsn={lsn:14352376906}, scn={val:1729061863144520005, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061863650729, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352376906}, next_to_submit_scn_:{val:1729061863144520005, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352376784}, end_lsn:{lsn:14352376906}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061863144520004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061863144520004, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:3120756601, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061863144520004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656971, miss_cnt:17, cache_read_size:8050167127, hit_ratio:"9.999997286814999997e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:43.686970] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:43.698799] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:43.701426] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15DE-0-0] [lt=12] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:43.701521] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15E0-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:43.702558] INFO [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:262) [22249][T1002_ArcSender][T1002][YB4A0A000407-00061E996F9960C1-0-0] [lt=16] ObArchiveSender is running(thread_index=1) [2024-10-16 14:57:43.710846] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:43.712340] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=259, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:43.712357] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:43.712372] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:43.714912] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:43.724710] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:43.730240] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061863729720000, start_id=1729061863729720000, end_id=1729061863729720001) [2024-10-16 14:57:43.731116] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.736093] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061863736086000, start_id=1729061863736086000, end_id=1729061863736086001) [2024-10-16 14:57:43.742750] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=24] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:43.756575] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:43.756606] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=14] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:43.756617] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=10] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:43.768810] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.771327] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:43.775319] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C79-0-0] [lt=28] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:43.781819] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=12] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:43.788653] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.788818] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.788833] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.788846] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.788857] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.788871] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.789632] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.789642] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.789650] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.789661] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.789669] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:43.790181] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061863789749000, start_id=1729061863789749000, end_id=1729061863789749001) [2024-10-16 14:57:43.790645] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:43.802486] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061863801693000, start_id=1729061863801693000, end_id=1729061863801693001) [2024-10-16 14:57:43.804254] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:43.808702] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [21908][T1001_MediumLoo][T1001][YB4A0A000407-00061E9973AF38E5-0-0] [lt=12] MediumCheckTask(cost_ts=7) [2024-10-16 14:57:43.812196] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1226) [2024-10-16 14:57:43.818413] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061863817409000, start_id=1729061863817409000, end_id=1729061863817409001) [2024-10-16 14:57:43.825534] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1002}, {id:1001}]) [2024-10-16 14:57:43.828507] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:43.828522] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:43.828534] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52440230, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:43.831128] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.832206] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=21] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}) [2024-10-16 14:57:43.834801] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6F-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:43.834825] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D6F-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061863644987001, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061863644987001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:43.835893] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D6F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061863835410000, start_id=1729061863835410000, end_id=1729061863835410001) [2024-10-16 14:57:43.836748] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B732E8D6F-0-0] [lt=18] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.055407524108886719e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=302, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.072965669169430839e+03) [2024-10-16 14:57:43.838236] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061863837527000, start_id=1729061863837527000, end_id=1729061863837527001) [2024-10-16 14:57:43.838306] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6F-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:43.838325] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D6F-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:43.850099] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:43.850112] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:43.850323] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=16] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:43.853220] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=4630, mem_used=10398720) [2024-10-16 14:57:43.868843] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.875653] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:43.877234] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:43.884533] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:43.884548] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:43.884821] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=469, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:43.884842] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=149, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:43.884858] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:43.884981] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD250-0-0] [lt=26] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:43.885022] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD250-0-0] [lt=40] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1120, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:43.891912] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:43.892510] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=25] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:43.892558] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061857241966000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:43.892851] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FCD-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:43.901510] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:43.911953] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1226, avg_group_log_size=122) [2024-10-16 14:57:43.911972] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1226) [2024-10-16 14:57:43.912552] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4144-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:43.912563] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4144-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=681, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:43.915543] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:43.922362] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:43.930173] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061863929954000, start_id=1729061863929954000, end_id=1729061863929954001) [2024-10-16 14:57:43.931215] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.936083] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061863935165000, start_id=1729061863935165000, end_id=1729061863935165001) [2024-10-16 14:57:43.938579] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061859935160, last_first_register_time_us=1722494150998907, register_time_us=1729061859935160, curr_time_us=1729061863938578) [2024-10-16 14:57:43.938620] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061863938597, last_submit_register_req_time_us=1729061863938597, register_parent_reason=3) [2024-10-16 14:57:43.938629] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:43.941282] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=7] worker thread began to shrink(tenant_->id()=1003, group_id_=1, token=0) [2024-10-16 14:57:43.942400] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=66] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2942, mem_used=6239232) [2024-10-16 14:57:43.950861] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106864][T1003_L0_G1][T1003][Y0-0000000000000000-0-0] [lt=8] worker thread exit(tenant_->id()=1003, workers_.get_size()=2) [2024-10-16 14:57:43.950879] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106864][T1003_L0_G1][T0][Y0-0000000000000000-0-0] [lt=14] tenant thread end_run(id_=1003, ret=0, thread_count_=172) [2024-10-16 14:57:43.965617] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:43.968865] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:43.985193] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FEB1-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=212, avg_log_gen_to_submit_time=242, avg_log_submit_to_flush_time=638, avg_log_submit_to_slide_time=918) [2024-10-16 14:57:43.985395] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=244, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:43.985409] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=146, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:43.990323] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061863990033000, start_id=1729061863990033000, end_id=1729061863990033001) [2024-10-16 14:57:43.992876] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:43.992897] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=212) [2024-10-16 14:57:43.996855] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=30] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 563.25; ) [2024-10-16 14:57:43.996875] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:43.996921] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 472.50; ) [2024-10-16 14:57:43.996929] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:43.996940] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:44.002683] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061864001745000, start_id=1729061864001745000, end_id=1729061864001745001) [2024-10-16 14:57:44.002782] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=16] PNIO [ratelimit] time: 1729061864002780, bytes: 4446814974494, bw: 0.373276 MB/s, add_ts: 1000084, add_bytes: 391441 [2024-10-16 14:57:44.004543] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:44.012011] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:44.012030] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:44.012239] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=242, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:44.012258] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=380, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:44.012274] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:44.012464] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C432A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=670, avg_log_submit_to_slide_time=1096) [2024-10-16 14:57:44.018574] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061864017553000, start_id=1729061864017553000, end_id=1729061864017553001) [2024-10-16 14:57:44.025220] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:44.025965] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.025993] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.026015] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=16] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:44.026157] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880098662000, latest_log_ts:{val:1729061863549302001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:44.026203] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880098662000, latest_log_ts:{val:1729061863549302001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:44.026240] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:44.026250] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=0, rec_log_ts={val:1729061863549302001, v:0}) [2024-10-16 14:57:44.026257] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:44.026265] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:44.026278] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:44.026289] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:44.027022] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.027037] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.027048] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:44.027084] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:44.027114] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:44.027138] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=23] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:44.027144] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:44.027160] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:44.027169] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:44.027873] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.027890] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.027902] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:44.027939] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:44.027970] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:44.027996] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=24] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:44.028005] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:44.028014] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:44.028022] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:44.028730] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.028745] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.028754] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:44.028791] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:44.028825] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=33] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:44.028851] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=24] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:44.028860] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:44.028869] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:44.028876] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:44.028880] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:44.031259] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:44.031279] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:44.031286] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.036508] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=14] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:44.049505] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:44.049993] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061864049233000, start_id=1729061864049233000, end_id=1729061864049233001) [2024-10-16 14:57:44.050014] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061864049988], gts:1729061864049233000, latest_srr:[mts=1729061864049988]}) [2024-10-16 14:57:44.050025] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:44.065537] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE779F-0-0] [lt=45] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:44.068922] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.071946] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] Garbage Collector is running(seq_=656492, gc_interval=10000000) [2024-10-16 14:57:44.071975] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=11] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:44.071982] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:44.072754] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:44.072980] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=13] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:44.072991] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] gc_check_member_list_ cost time(ret=0, time_us=1034) [2024-10-16 14:57:44.073008] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=12] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:44.073764] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:44.074426] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=14] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:44.075024] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=13] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:44.075913] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:44.075928] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=12] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:44.075996] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=67] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:44.076005] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:44.076013] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:44.076021] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] execute_gc cost time(ret=0, time_us=94) [2024-10-16 14:57:44.076025] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:44.076032] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:133) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-10-16 14:57:44.076043] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:140) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=4] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-10-16 14:57:44.079690] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=37] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:44.080251] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:44.080300] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=16] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061860198021000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:44.080597] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=16] PNIO [ratelimit] time: 1729061864080596, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007625, add_bytes: 0 [2024-10-16 14:57:44.084689] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:44.084706] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:44.085412] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D6-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061864076651, keepalive_ts:-1}) [2024-10-16 14:57:44.085428] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D6-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061864076651, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:44.089902] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FD1-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:44.090104] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=172, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:44.090122] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=157, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:44.090150] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:44.108168] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15DC-0-0] [lt=10] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:44.110190] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15D8-0-0] [lt=11] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:44.112335] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=24612880, this={dio_aligned_buf:{buf_write_offset:138, buf_padding_size:3958, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:93866978, truncate_used_ts:39944931}, log_block_size:67108864, dir_fd:173, io_fd:196}, aligned_buf_len=4096, aligned_block_offset=24612864, buf_len=122, total_write_size_=11684, total_write_size_after_dio_=405504, ob_pwrite_used_ts=22432, count=96, each_pwrite_cost=233) [2024-10-16 14:57:44.116771] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=2][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:44.116784] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:44.117027] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=8] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2170533, replace_num=31457) [2024-10-16 14:57:44.117045] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:44.126801] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=53] current monitor number(seq_=-1) [2024-10-16 14:57:44.130264] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061864128736000, start_id=1729061864128736000, end_id=1729061864128736001) [2024-10-16 14:57:44.131322] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.132638] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2455, mem_used=6239232) [2024-10-16 14:57:44.133849] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [22325][T1003_COMMONLSS][T1003][YB4A0A000407-00061E9B3A19CB16-0-0] [lt=16][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:44.135282] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:44.136079] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061864128736000, start_id=1729061864128736000, end_id=1729061864128736001) [2024-10-16 14:57:44.136144] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:44.136159] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:44.136394] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=242, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:44.136410] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=110, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:44.136429] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9099) [2024-10-16 14:57:44.136458] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=27] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=11, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:44.136651] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94F9-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:44.136660] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94F9-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=734, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:44.136664] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F94F9-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=538, avg_log_submit_to_slide_time=703) [2024-10-16 14:57:44.145355] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9E3-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061864144741000, start_id=1729061864144741000, end_id=1729061864144741001) [2024-10-16 14:57:44.145544] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9E7-0-0] [lt=8] submit log success(service_type_=0, last_id=1729061880721265000, limited_id=1729061880721265000) [2024-10-16 14:57:44.145553] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9E7-0-0] [lt=8] submit log with lock success(service_type_=0, last_id=1729061880721265000, limited_id=1729061880721265000) [2024-10-16 14:57:44.145556] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9E7-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061864144741000, start_id=1729061864144741000, end_id=1729061864144741001) [2024-10-16 14:57:44.145673] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B9E4-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061864144741000, start_id=1729061864144741000, end_id=1729061864144741001) [2024-10-16 14:57:44.145821] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9E5-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061864145815000, start_id=1729061864145815000, end_id=1729061864145815001) [2024-10-16 14:57:44.145902] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B9E6-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061864145815000, start_id=1729061864145815000, end_id=1729061864145815001) [2024-10-16 14:57:44.146016] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] update id meta(ret=0, service_type=0, limited_id=1729061880721265000, latest_scn={val:1729061864144741001, v:0}, *this={tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061861046378000, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880721265000, latest_log_ts:{val:1729061864144741001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:44.146049] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=33] handle submit callback(service_type_=0, success=true, used_ts=506, log_ts={val:1729061864144741001, v:0}, limited_id=1729061880721265000) [2024-10-16 14:57:44.146055] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:44.146059] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:44.168973] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.172415] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA882-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:44.172446] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA882-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061863644987004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061863644987004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:44.176177] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA882-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061864176166000, start_id=1729061864176166000, end_id=1729061864176166001) [2024-10-16 14:57:44.176249] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA882-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9689, avg_group_log_size=569) [2024-10-16 14:57:44.176264] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA882-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9690) [2024-10-16 14:57:44.176570] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9689) [2024-10-16 14:57:44.176793] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=13] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:44.185041] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1894) [2024-10-16 14:57:44.188256] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502E-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:44.188276] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502E-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:44.189953] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FD3-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:44.190388] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:44.190404] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598889, miss_cnt:9, cache_read_size:149777915060, hit_ratio:"9.999999345924994731e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:44.190512] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061864190087000, start_id=1729061864190087000, end_id=1729061864190087001) [2024-10-16 14:57:44.190578] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=3422, avg_group_log_size=380) [2024-10-16 14:57:44.190751] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=222, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:44.190763] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=25, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:44.190775] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:44.191052] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C103A-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=265, avg_log_submit_to_slide_time=505) [2024-10-16 14:57:44.197263] INFO [SHARE.LOCATION] renew_all_ls_locations_by_rpc (ob_ls_location_service.cpp:801) [21248][AutoLSLocRpc][T0][YB4A0A000407-00061E9B390A225A-0-0] [lt=45] [LS_LOCATION] Get ls leaders by RPC(ret=0, ret="OB_SUCCESS", dests=["10.0.4.7:2890", "10.0.4.8:2890", "10.0.4.9:2890"], leaders=[{key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}}, {key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}}, {key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}}, {key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}}, {key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}}, {key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}}, {key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}}, {key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}}, {key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}]) [2024-10-16 14:57:44.201656] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601414, offset=9629000}, end_cursor:ObLogCursor{file_id=192, log_id=47601415, offset=9629284}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d58050, len:696, log_data_len:284, seq:47601414, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:44.202795] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061864201840000, start_id=1729061864201840000, end_id=1729061864201840001) [2024-10-16 14:57:44.204460] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A6-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061864201840000, start_id=1729061864201840001, end_id=1729061864201840002) [2024-10-16 14:57:44.204995] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:44.206010] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257762, offset=39033031}, end_cursor:ObLogCursor{file_id=197, log_id=49257763, offset=39033314}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e40050, len:1849, log_data_len:283, seq:49257762, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:44.206429] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601415, offset=9629284}, end_cursor:ObLogCursor{file_id=192, log_id=47601416, offset=9629558}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ca8050, len:412, log_data_len:274, seq:47601415, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:44.208495] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB371-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061864207882000, start_id=1729061864207882000, end_id=1729061864207882001) [2024-10-16 14:57:44.209631] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1050-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061864208920000, start_id=1729061864208920000, end_id=1729061864208920001) [2024-10-16 14:57:44.210206] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257763, offset=39033314}, end_cursor:ObLogCursor{file_id=197, log_id=49257764, offset=39033596}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d00050, len:1566, log_data_len:282, seq:49257763, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:44.210727] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601416, offset=9629558}, end_cursor:ObLogCursor{file_id=192, log_id=47601417, offset=9629832}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72da050, len:4234, log_data_len:274, seq:47601416, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:44.211022] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601417, offset=9629832}, end_cursor:ObLogCursor{file_id=192, log_id=47601418, offset=9630106}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d8c050, len:3960, log_data_len:274, seq:47601417, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:44.211072] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561302, miss_cnt:1, cache_read_size:152590681549, hit_ratio:"9.999999928346899880e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:44.213596] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F414D-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:44.216283] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D8D-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510332084677, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410602, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:44.216722] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257764, offset=39033596}, end_cursor:ObLogCursor{file_id=197, log_id=49257765, offset=39033878}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e80050, len:1284, log_data_len:282, seq:49257764, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:44.217849] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257765, offset=39033878}, end_cursor:ObLogCursor{file_id=197, log_id=49257766, offset=39034160}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e24050, len:1002, log_data_len:282, seq:49257765, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:44.218881] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061864217358000, start_id=1729061864217358000, end_id=1729061864217358001) [2024-10-16 14:57:44.219175] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650239, miss_cnt:32, cache_read_size:8050420365, hit_ratio:"9.999994892280673620e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:44.219202] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656977, miss_cnt:17, cache_read_size:8050167859, hit_ratio:"9.999997286815259789e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:44.231386] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.240517] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:57:44.241265] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.241298] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=32] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.241322] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=19] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:44.241456] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=15] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061862047401001, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880721265000, latest_log_ts:{val:1729061864144741001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:44.241502] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:44.241524] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:44.241532] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061864144741001, v:0}) [2024-10-16 14:57:44.241535] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.241542] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:44.241553] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:44.241561] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:44.242246] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.242265] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=20] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.242280] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:44.242355] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061862047401001, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:44.242389] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=33] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061862047401001, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:44.242416] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:44.242422] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:44.242431] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:44.242440] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:44.243152] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.243165] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.243175] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:44.243241] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061862047401001, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:44.243269] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061862047401001, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:44.243292] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:44.243297] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:44.243303] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:44.243309] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:44.244024] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.244039] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:44.244048] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:44.244125] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061862047401001, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:44.244159] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=32] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061862047401001, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:44.244185] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:44.244190] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:44.244199] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:44.244208] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:44.244211] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:44.247422] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E0C-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061864246721000, start_id=1729061864246721000, end_id=1729061864246721001) [2024-10-16 14:57:44.250012] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=293, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:44.250031] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:44.250061] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:44.250256] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9157-0-0] [lt=30] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:44.250267] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9157-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20428, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:44.265630] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77A3-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:44.266037] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53DC-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:44.266045] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53DC-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10608, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:44.269012] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.275562] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C83-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:44.275584] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C83-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:44.275590] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C83-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10448, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:44.275594] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C83-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=278, avg_log_submit_to_slide_time=10440) [2024-10-16 14:57:44.275609] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C83-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178482, log_task={header:{begin_lsn:{lsn:14538491280}, end_lsn:{lsn:14538491402}, log_id:115178482, min_scn:{val:1729061864145815000, v:0}, max_scn:{val:1729061864145815000, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538491158}, prev_proposal_id:23, committed_end_lsn:{lsn:14538491280}, data_checksum:1715074492, accum_checksum:739605410, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061864175476, freeze_ts:1729061864175477, submit_ts:1729061864175481, flushed_ts:1729061864176078, gen_to_freeze cost time:1, gen_to_submit cost time:5, submit_to_flush cost time:597}, fs_cb_begin_ts=1729061864275578, log_life_time=100102) [2024-10-16 14:57:44.275617] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:44.275830] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=205, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:44.275843] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=58, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:44.275874] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:44.278163] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1E01-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061864277790000, start_id=1729061864277790000, end_id=1729061864277790001) [2024-10-16 14:57:44.279152] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:44.283239] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21656][T1_L0_G0][T1][YB4A0A000409-00061E9991036467-0-0] [lt=8][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510332151634, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410603, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:44.284726] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=1894, avg_group_log_size=157) [2024-10-16 14:57:44.284744] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1894) [2024-10-16 14:57:44.284770] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:44.284775] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:44.285005] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1894) [2024-10-16 14:57:44.285035] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:44.301644] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=6] ====== tenant freeze timer task ====== [2024-10-16 14:57:44.301660] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:44.301676] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:44.304869] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:44.305590] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985D5-0-0] [lt=15] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=187) [2024-10-16 14:57:44.305697] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061864305586000, start_id=1729061864305586000, end_id=1729061864305586001) [2024-10-16 14:57:44.306283] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=45, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:57:44.306299] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=16] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=708, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:57:44.306303] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:57:44.306307] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=-731, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:57:44.306310] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=21, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:57:44.306469] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:44.306478] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=6] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:44.306488] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:44.311951] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:44.311971] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:44.311993] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373625134, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:44.317117] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:44.317559] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7229, mem_used=14558208) [2024-10-16 14:57:44.322245] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=18] BlockGCTimerTask success(ret=0, cost_time_us=8, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):114, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:44.323870] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B23-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061864323534000, start_id=1729061864323534000, end_id=1729061864323534001) [2024-10-16 14:57:44.328296] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183801563, max_lsn:114264760385, max_scn:{val:1729061864217358000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45336899169}, reuse_lsn:{lsn:114264760385}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114222817345}}, last_submit_lsn:{lsn:114264759792}, last_submit_end_lsn:{lsn:114264760385}, last_submit_log_id:183801563, last_submit_log_pid:33576, max_flushed_lsn:{lsn:114264759792}, max_flushed_end_lsn:{lsn:114264760385}, max_flushed_log_pid:33576, committed_end_lsn:{lsn:114264760385}, last_slide_log_id:183801563, last_slide_scn:{val:1729061864217358000, v:0}, last_slide_lsn:{lsn:114264759792}, last_slide_end_lsn:{lsn:114264760385}, last_slide_log_pid:33576, last_slide_log_accum_checksum:1631470208, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e2efe2530}, [StateMgr]={this:0x2b0e2efe6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, leader:"10.0.4.7:2890", leader_epoch:2, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183801564, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261840845, palf_role_change_cb:0x2b0e2efed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, reconfig_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, persistent_config_version:{proposal_id:33576, config_seq:9}, ms_ack_list:0, resend_config_version:{proposal_id:33576, config_seq:9}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494259837171, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494259837171, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e2efe3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262041271, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729031774558563002, v:0}, min_block_id:1697, min_block_min_scn:{val:1729025370959302001, v:0}, base_lsn_for_block_gc:{lsn:114212315136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, log_config_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, log_snapshot_meta:{version:1, base_lsn:{lsn:114212315136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:232079360}, readable_log_tail:{lsn:232079360}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114279419904}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/meta", dir_fd:187, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:36339712, block_header_serialize_buf_:0x2b0e2efe89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114264760385}, readable_log_tail:{lsn:114264760385}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114212315136}, min_scn:{val:1729057087482360001, v:0}, curr_block_id:1702, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/log", dir_fd:190, min_block_id:1697, max_block_id:1703, curr_writable_block_id:1702}, logical_block_size_:67104768, curr_block_writable_size_:14659519, block_header_serialize_buf_:0x2b0e2efea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261839896, this:0x2b0e2efe6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33576, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33575, majority_max_lsn:{lsn:45338084276}, saved_end_lsn:{lsn:45338084276}, last_submit_prepare_req_time_us:1722494261840968, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e2efe69b0}) [2024-10-16 14:57:44.328431] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=133] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061864219273, last_flushed_end_lsn:{lsn:114264760385}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061864219531, last_flushed_end_lsn:{lsn:114264760385}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061864219788, last_flushed_end_lsn:{lsn:114264760385}}]) [2024-10-16 14:57:44.328442] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183801564, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061864219273, last_flushed_end_lsn:{lsn:114264760385}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061864219531, last_flushed_end_lsn:{lsn:114264760385}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061864219788, last_flushed_end_lsn:{lsn:114264760385}}]) [2024-10-16 14:57:44.329622] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=10] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:44.330427] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061864328667000, start_id=1729061864328667000, end_id=1729061864328667001) [2024-10-16 14:57:44.332667] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.333363] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1729061864128736000, v:0}, new_max_version={val:1729061864128736000, v:0}, last_min_version={val:1729061863929954000, v:0}, last_max_version={val:1729061863929954000, v:0}, total_time_us=3695, delta_us=204626) [2024-10-16 14:57:44.333379] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1729061864128736000, v:0}, min_version={val:1729061864128736000, v:0}, max_version={val:1729061864128736000, v:0}, in_service=true, leader_epoch=33576) [2024-10-16 14:57:44.336235] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061864334785000, start_id=1729061864334785000, end_id=1729061864334785001) [2024-10-16 14:57:44.350416] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9159-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=533, avg_log_submit_to_slide_time=18608) [2024-10-16 14:57:44.350430] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:44.366238] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:44.366284] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=46, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:44.366295] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:44.368115] INFO [SHARE] adaptive_update_limit (ob_share_throttle_define.cpp:112) [55867][T1004_L0_G0][T1004][YB4A0A000409-00061E9A11A789DC-0-0] [lt=14] adaptive update(Tenant ID=1004, Config Specify Resource Limit(MB)=511, TxShare Current Memory Limit(MB)=155, Holding Memory(MB)=46, Tenant Remain Memory(MB)=181, Usable Remain Memory(MB)=108, Last Update Limit Timestamp=1729061864368048, Is Updated=true) [2024-10-16 14:57:44.368221] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [55867][T1004_L0_G0][T1004][YB4A0A000409-00061E9A11A789DC-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061864368048000, start_id=1729061864368048000, end_id=1729061864368048001) [2024-10-16 14:57:44.368499] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=46237107, this={dio_aligned_buf:{buf_write_offset:2006, buf_padding_size:2090, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33201944, truncate_used_ts:11502123}, log_block_size:67108864, dir_fd:195, io_fd:131}, aligned_buf_len=4096, aligned_block_offset=46235648, buf_len=547, total_write_size_=15430, total_write_size_after_dio_=425984, ob_pwrite_used_ts=43870, count=100, each_pwrite_cost=438) [2024-10-16 14:57:44.369073] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.375613] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C85-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:44.382869] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=23] blacklist_loop exec finished(cost_time=14, is_enabled=true, send_cnt=0) [2024-10-16 14:57:44.385191] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:44.385404] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=32, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:57:44.385418] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=13] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=1429, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:57:44.385422] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:57:44.385425] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=-1585, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:57:44.385428] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=-123, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:57:44.390553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061864390147000, start_id=1729061864390147000, end_id=1729061864390147001) [2024-10-16 14:57:44.390629] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:44.390642] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:44.393050] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FD7-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497442, log_task={header:{begin_lsn:{lsn:207589550619}, end_lsn:{lsn:207589550741}, log_id:218497442, min_scn:{val:1729061864145815001, v:0}, max_scn:{val:1729061864145815001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589550497}, prev_proposal_id:81, committed_end_lsn:{lsn:207589550619}, data_checksum:940009244, accum_checksum:248442228, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061864291013, freeze_ts:1729061864291013, submit_ts:1729061864291017, flushed_ts:1729061864291227, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:210}, fs_cb_begin_ts=1729061864393045, log_life_time=102032) [2024-10-16 14:57:44.394609] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:44.400857] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] Garbage Collector is running(seq_=656749, gc_interval=10000000) [2024-10-16 14:57:44.400891] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=15] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:44.400905] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=9] gc_check_member_list_ cost time(ret=0, time_us=32) [2024-10-16 14:57:44.400913] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:44.400919] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:44.400928] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:44.400936] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=8] execute_gc cost time(ret=0, time_us=9) [2024-10-16 14:57:44.400940] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:44.403254] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061864402985000, start_id=1729061864402985000, end_id=1729061864402985001) [2024-10-16 14:57:44.405290] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:44.412482] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:44.412500] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:44.412818] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:44.419170] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061864418780000, start_id=1729061864418780000, end_id=1729061864418780001) [2024-10-16 14:57:44.432717] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.450150] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:44.465154] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=12][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:44.465937] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=211, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:44.465954] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=420, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:44.469112] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.473872] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A82-0-0] [lt=11] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:44.473888] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A82-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:44.485475] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FEC1-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:44.485491] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FEC1-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=668, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:44.490561] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53E1-0-0] [lt=29] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:44.490572] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D53E1-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20294, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:44.501048] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C86-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061864499709000, start_id=1729061864499709000, end_id=1729061864499709001) [2024-10-16 14:57:44.512460] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:44.512492] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:44.512789] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2200) [2024-10-16 14:57:44.512941] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4339-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:44.512957] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4339-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1030, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:44.512996] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4338-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=396, avg_log_submit_to_slide_time=654) [2024-10-16 14:57:44.517198] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:44.530683] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061864529651000, start_id=1729061864529651000, end_id=1729061864529651001) [2024-10-16 14:57:44.530755] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9331, avg_group_log_size=666) [2024-10-16 14:57:44.530765] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9331) [2024-10-16 14:57:44.531048] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=306, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:44.531068] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=20] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9331) [2024-10-16 14:57:44.531081] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=463, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:44.531092] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9331) [2024-10-16 14:57:44.531102] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:44.531239] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD163-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:44.531248] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD163-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=1147, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:44.531261] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD163-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=791, avg_log_submit_to_slide_time=1114) [2024-10-16 14:57:44.532756] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.533056] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:57:44.533072] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:44.533090] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444540231, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:44.536448] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061864536008000, start_id=1729061864536008000, end_id=1729061864536008001) [2024-10-16 14:57:44.550355] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061864549733000, start_id=1729061864549733000, end_id=1729061864549733001) [2024-10-16 14:57:44.550375] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=16] ts response task statistics(alloc_count=261364550, free_count=130682274) [2024-10-16 14:57:44.550393] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=492, get_gts_with_stc_cnt=74, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=74, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:44.550568] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=509, get_gts_with_stc_cnt=8, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:44.558549] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B5A0C6796-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061864558346000, start_id=1729061864558346000, end_id=1729061864558346001) [2024-10-16 14:57:44.568036] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B5A0C6796-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061864567868000, start_id=1729061864567868000, end_id=1729061864567868001) [2024-10-16 14:57:44.568945] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.568963] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.569171] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.569506] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.569622] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.573023] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [21736][T1001_COMMONLSS][T1001][YB4A0A000407-00061E9B3A29CAFD-0-0] [lt=21][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:44.582969] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B5A0C6796-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061864582778000, start_id=1729061864582778000, end_id=1729061864582778001) [2024-10-16 14:57:44.585305] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1}, {id:1003}]) [2024-10-16 14:57:44.585672] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD267-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=430, avg_log_submit_to_slide_time=674) [2024-10-16 14:57:44.587314] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=17] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:44.587686] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:44.587701] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:44.590673] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061864589369000, start_id=1729061864589369000, end_id=1729061864589369001) [2024-10-16 14:57:44.590738] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=5174) [2024-10-16 14:57:44.591453] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=5639) [2024-10-16 14:57:44.591481] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=5174) [2024-10-16 14:57:44.592163] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C08-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:44.592174] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C08-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=681, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:44.592312] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:44.596063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B5A0C6796-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061864595048000, start_id=1729061864595048000, end_id=1729061864595048001) [2024-10-16 14:57:44.596962] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B2D-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061864596107000, start_id=1729061864596107000, end_id=1729061864596107001) [2024-10-16 14:57:44.597798] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23523350, this={dio_aligned_buf:{buf_write_offset:568, buf_padding_size:3528, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:47004045, truncate_used_ts:19010292}, log_block_size:67108864, dir_fd:134, io_fd:183}, aligned_buf_len=4096, aligned_block_offset=23523328, buf_len=546, total_write_size_=39664, total_write_size_after_dio_=458752, ob_pwrite_used_ts=26847, count=102, each_pwrite_cost=263) [2024-10-16 14:57:44.598285] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D8A-0-0] [lt=8] [APPLY STAT CB APPEND COST TIME](cur_stat_count=38, stat_interval=5000000, avg (count/cost)=43, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:44.598296] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D8A-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=38, stat_interval=5000000, avg (count/cost)=895, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:44.598300] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D8A-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=38, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:44.598303] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D8A-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=38, stat_interval=5000000, avg (count/cost)=-846, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:44.598312] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D8A-0-0] [lt=9] [APPLY STAT CB TOTAL TIME](cur_stat_count=38, stat_interval=5000000, avg (count/cost)=92, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:44.598908] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C6-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:44.598933] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C6-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061864144741004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061864144741004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:44.599836] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1E00-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061864599340000, start_id=1729061864599340000, end_id=1729061864599340001) [2024-10-16 14:57:44.600147] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=52451124, this={dio_aligned_buf:{buf_write_offset:2393, buf_padding_size:1703, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80311002, truncate_used_ts:29009503}, log_block_size:67108864, dir_fd:190, io_fd:179}, aligned_buf_len=4096, aligned_block_offset=52449280, buf_len=549, total_write_size_=108069, total_write_size_after_dio_=802816, ob_pwrite_used_ts=49560, count=170, each_pwrite_cost=291) [2024-10-16 14:57:44.602996] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061864602932000, start_id=1729061864602932000, end_id=1729061864602932001) [2024-10-16 14:57:44.605595] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:44.605662] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22495][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C6-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061864605127000, start_id=1729061864605127000, end_id=1729061864605127001) [2024-10-16 14:57:44.606507] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:44.608625] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B5A0C6796-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061864608329000, start_id=1729061864608329000, end_id=1729061864608329001) [2024-10-16 14:57:44.609458] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5056, mem_used=10398720) [2024-10-16 14:57:44.619114] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061864617882000, start_id=1729061864617882000, end_id=1729061864617882001) [2024-10-16 14:57:44.632584] INFO [ARCHIVE] do_thread_task_ (ob_archive_fetcher.cpp:310) [22252][T1002_ArcFetche][T1002][YB4A0A000407-00061E996FB960C1-0-0] [lt=13] ObArchiveFetcher is running(thread_index=1) [2024-10-16 14:57:44.632813] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.645061] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C53-0-0] [lt=10] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:44.645090] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C53-0-0] [lt=19] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061864145815004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061864145815004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:44.651300] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C53-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061864651227000, start_id=1729061864651227000, end_id=1729061864651227001) [2024-10-16 14:57:44.666644] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:44.667865] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:44.669230] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.680377] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:44.690259] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FDD-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497445, log_task={header:{begin_lsn:{lsn:207589550976}, end_lsn:{lsn:207589551098}, log_id:218497445, min_scn:{val:1729061864145815004, v:0}, max_scn:{val:1729061864145815004, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589550854}, prev_proposal_id:81, committed_end_lsn:{lsn:207589550976}, data_checksum:940009244, accum_checksum:45578997, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061864590164, freeze_ts:1729061864590164, submit_ts:1729061864590168, flushed_ts:1729061864591256, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1088}, fs_cb_begin_ts=1729061864690254, log_life_time=100090) [2024-10-16 14:57:44.695165] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22398][T1003_TntShared][T1003][Y0-0000000000000000-0-0] [lt=14] clear dtl interm result cost(us)(clear_cost=37, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:44.698857] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:44.701317] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=7] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:44.702138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061864700717000, start_id=1729061864700717000, end_id=1729061864700717001) [2024-10-16 14:57:44.707422] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31896][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB1FCC6DB-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:44.713012] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=42732732, this={dio_aligned_buf:{buf_write_offset:3382, buf_padding_size:714, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35133258, truncate_used_ts:12254335}, log_block_size:67108864, dir_fd:169, io_fd:144}, aligned_buf_len=4096, aligned_block_offset=42729472, buf_len=122, total_write_size_=14879, total_write_size_after_dio_=417792, ob_pwrite_used_ts=27720, count=99, each_pwrite_cost=280) [2024-10-16 14:57:44.713034] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=212, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:44.713045] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=117, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:44.713074] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:44.715917] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:44.715934] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=9] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=213) [2024-10-16 14:57:44.717309] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:44.717382] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B20-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061864717087000, start_id=1729061864717087000, end_id=1729061864717087001) [2024-10-16 14:57:44.719327] INFO [STORAGE.COMPACTION] schedule_all_tablets_minor (ob_tenant_tablet_scheduler.cpp:553) [21602][T1_MergeLoop][T1][YB4A0A000407-00061E997F5A367D-0-0] [lt=13] start schedule all tablet minor merge(minor_ls_tablet_iter_={ls_idx:-1, ls_ids:[{id:1}], tablet_idx:0, tablet_ids_.count():0, schedule_tablet_cnt:0, max_batch_tablet_cnt:50000}) [2024-10-16 14:57:44.724809] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:44.726979] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15E2-0-0] [lt=5] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:44.728685] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=44] success to update min reserved snapshot(reserved_snapshot=1729060061837115000, duration=1800, snapshot_gc_ts=1729061861837115000) [2024-10-16 14:57:44.729971] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=11] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060061837115000) [2024-10-16 14:57:44.730280] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23158469, this={dio_aligned_buf:{buf_write_offset:3893, buf_padding_size:203, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33384974, truncate_used_ts:14052864}, log_block_size:67108864, dir_fd:200, io_fd:174}, aligned_buf_len=4096, aligned_block_offset=23154688, buf_len=112, total_write_size_=11684, total_write_size_after_dio_=401408, ob_pwrite_used_ts=42770, count=96, each_pwrite_cost=445) [2024-10-16 14:57:44.730306] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1323) [2024-10-16 14:57:44.730705] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=12] submit reserved snapshot log success(ls_id={id:1001}, new_snapshot_version=1729060061837115000) [2024-10-16 14:57:44.730741] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=34] update reserved snapshot finished(cost_ts=2058, reserved_snapshot=1729060061837115000) [2024-10-16 14:57:44.730805] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061864729928000, start_id=1729061864729928000, end_id=1729061864729928001) [2024-10-16 14:57:44.732873] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.738481] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061864737416000, start_id=1729061864737416000, end_id=1729061864737416001) [2024-10-16 14:57:44.741622] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=20][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:114264765252}], scn_array=[cnt:1, {val:1729061864729928000, v:0}], curr_size=593, accum_size=3681, time_cost=10736) [2024-10-16 14:57:44.743923] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:85) [21602][T1_MergeLoop][T1][YB4A0A000407-00061E997F5A367D-0-0] [lt=17] MergeLoopTask(cost_ts=24620) [2024-10-16 14:57:44.744067] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=14] LogLoopThread round_cost_time(us)(round_cost_time=7) [2024-10-16 14:57:44.750060] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E0E-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:44.750078] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E0E-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:44.750090] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E0E-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:44.756654] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:44.756680] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:44.756687] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:44.765874] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77AD-0-0] [lt=32] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:44.765900] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77AD-0-0] [lt=18][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938299, log_task={header:{begin_lsn:{lsn:213058242055}, end_lsn:{lsn:213058242177}, log_id:220938299, min_scn:{val:1729061864647253000, v:0}, max_scn:{val:1729061864647253000, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058241933}, prev_proposal_id:88, committed_end_lsn:{lsn:213058242055}, data_checksum:1715074492, accum_checksum:1082893576, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061864665776, freeze_ts:1729061864665776, submit_ts:1729061864665780, flushed_ts:1729061864666622, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:842}, fs_cb_begin_ts=1729061864765895, log_life_time=100119) [2024-10-16 14:57:44.769290] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.771383] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:44.781882] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=14] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:44.788620] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.789903] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061864589369000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=200527, in_cluster_service=true, cluster_version={val:1729061864390147000, v:0}, min_cluster_version={val:1729061864390147000, v:0}, max_cluster_version={val:1729061864390147000, v:0}, get_cluster_version_err=0, cluster_version_delta=399749, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65248930, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=362062611, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:44.791233] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061864786393000, start_id=1729061864786393000, end_id=1729061864786393001) [2024-10-16 14:57:44.793833] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.793851] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.793871] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.793882] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.793891] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=3] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.793905] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.793916] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.793927] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.793940] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.793955] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:44.803171] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061864802925000, start_id=1729061864802925000, end_id=1729061864802925001) [2024-10-16 14:57:44.805882] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:44.819277] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061864818842000, start_id=1729061864818842000, end_id=1729061864818842001) [2024-10-16 14:57:44.825638] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1002}, {id:1003}]) [2024-10-16 14:57:44.826572] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=51] success to update min reserved snapshot(reserved_snapshot=1729060057241966000, duration=1800, snapshot_gc_ts=1729061857241966000) [2024-10-16 14:57:44.827149] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=13] update reserved snapshot finished(cost_ts=579, reserved_snapshot=1729060057241966000) [2024-10-16 14:57:44.832931] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.840709] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D70-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:44.840740] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D70-0-0] [lt=21] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061864647253001, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061864647253001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:44.843368] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=39] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:44.843951] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:44.844009] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=14] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061864434291000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:44.844186] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D70-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061864843202000, start_id=1729061864843202000, end_id=1729061864843202001) [2024-10-16 14:57:44.846720] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D70-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061864846379000, start_id=1729061864846379000, end_id=1729061864846379001) [2024-10-16 14:57:44.846789] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D70-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:44.846803] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D70-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:44.849453] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=13] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061860841760, last_first_register_time_us=1722494157071428, register_time_us=1729061860841760, curr_time_us=1729061864849452) [2024-10-16 14:57:44.849491] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061864849472, last_submit_register_req_time_us=1729061864849472, register_parent_reason=3) [2024-10-16 14:57:44.849499] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:44.850273] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=59066882, this={dio_aligned_buf:{buf_write_offset:2684, buf_padding_size:1412, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:38334835, truncate_used_ts:14111002}, log_block_size:67108864, dir_fd:207, io_fd:202}, aligned_buf_len=4096, aligned_block_offset=59064320, buf_len=122, total_write_size_=12182, total_write_size_after_dio_=421888, ob_pwrite_used_ts=38263, count=100, each_pwrite_cost=382) [2024-10-16 14:57:44.850565] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:44.850577] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:44.853320] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4693, mem_used=10398720) [2024-10-16 14:57:44.854764] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:44.862682] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=18] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:44.869314] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.875933] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C8F-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:44.876095] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:44.885294] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:44.885309] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:44.885559] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=207, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:44.885570] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=51, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:44.885586] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:44.885736] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD270-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:44.885742] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD270-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=614, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:44.890564] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=5] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:44.891139] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:44.891194] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=22] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061861837115000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:44.891511] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1333) [2024-10-16 14:57:44.892228] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [21891][T1001_ObXAHbWor][T1001][Y0-0000000000000000-0-0] [lt=12] XA scheduler heartbeat task statistics(avg_time=1) [2024-10-16 14:57:44.901586] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:44.905030] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21882][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=21][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:44.905081] WDIAG read_multi_row (ob_table_access_helper.h:238) [21882][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=13][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:44.905826] INFO delete_row (ob_table_access_helper.h:312) [21882][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1001, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061864905087) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061864905087) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:44.912891] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2620, avg_group_log_size=238) [2024-10-16 14:57:44.912905] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2620) [2024-10-16 14:57:44.913127] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2742) [2024-10-16 14:57:44.913313] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4345-0-0] [lt=19] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:44.913322] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4345-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=605, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:44.914907] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:44.914954] WDIAG read_multi_row (ob_table_access_helper.h:238) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:44.915671] INFO delete_row (ob_table_access_helper.h:312) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1004, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061864914960) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061864914960) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:44.915726] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=11] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:44.915769] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=5] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:44.915811] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:44.919571] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:44.920917] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234A-0-0] [lt=36] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:57:44.921560] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234A-0-0] [lt=9] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1004) [2024-10-16 14:57:44.921570] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234A-0-0] [lt=8] no transfer partition job [2024-10-16 14:57:44.921577] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234A-0-0] [lt=5] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:57:44.922462] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:44.922837] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234A-0-0] [lt=6] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1004, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:57:44.922855] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234A-0-0] [lt=12] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:57:44.924063] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234A-0-0] [lt=12] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1002, status:0}], ls_array_=[{tenant_id:1004, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729059563578310, last_statistic_bg_stat_time=1725269669481368, last_statistic_schema_version=1725266205820024, last_statistic_max_transfer_task_id={id:5}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:57:44.930681] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22455][T1003_TenantWea][T1003][YB4A0A000407-00061E9B3ABCFCE5-0-0] [lt=17] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.103558874130249023e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.008835578279190031e+03) [2024-10-16 14:57:44.930827] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061864930554000, start_id=1729061864930554000, end_id=1729061864930554001) [2024-10-16 14:57:44.931194] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE0-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061864898855, keepalive_ts:-1}) [2024-10-16 14:57:44.931212] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE0-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061864898855, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:44.932988] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.938466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061864937941000, start_id=1729061864937941000, end_id=1729061864937941001) [2024-10-16 14:57:44.939042] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061860936121, last_first_register_time_us=1722494150998907, register_time_us=1729061860936121, curr_time_us=1729061864939041) [2024-10-16 14:57:44.939070] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=14] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061864939057, last_submit_register_req_time_us=1729061864939057, register_parent_reason=3) [2024-10-16 14:57:44.939087] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=6] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:44.941794] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:44.941813] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=14] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:57:44.941822] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:44.942507] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2984, mem_used=6239232) [2024-10-16 14:57:44.951548] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=9] ====== tenant freeze timer task ====== [2024-10-16 14:57:44.966201] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:44.969371] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:44.978500] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2015-0-0] [lt=15] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:44.978522] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2015-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:44.985640] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=229, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:44.985658] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=37, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:44.985979] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FED1-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=285, avg_log_submit_to_slide_time=583) [2024-10-16 14:57:44.990553] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FE3-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1333, avg_group_log_size=121) [2024-10-16 14:57:44.990771] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061864989798000, start_id=1729061864989798000, end_id=1729061864989798001) [2024-10-16 14:57:44.998681] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 265.75; ) [2024-10-16 14:57:44.998702] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:44.998738] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 839.25; ) [2024-10-16 14:57:44.998747] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:44.998758] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:45.003216] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061865002664000, start_id=1729061865002664000, end_id=1729061865002664001) [2024-10-16 14:57:45.003679] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061865003678, bytes: 4446815426843, bw: 0.431007 MB/s, add_ts: 1000898, add_bytes: 452349 [2024-10-16 14:57:45.006168] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:45.009072] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Cache wash clean map node details(ret=0, clean_node_count=34, clean_time=0, clean_start_pos=1132452, clean_num=62914) [2024-10-16 14:57:45.012980] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:45.012994] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:45.013166] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=191, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:45.013189] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=67, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:45.013215] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:45.013449] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4347-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=281, avg_log_submit_to_slide_time=560) [2024-10-16 14:57:45.016429] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:57:45.016526] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAC-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061864937941000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061875590307000, latest_log_ts:{val:1729061858932674001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:45.016584] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAC-0-0] [lt=55] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:45.033053] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.042532] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:45.042549] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:45.042563] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=291940713, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:45.050805] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061865049785000, start_id=1729061865049785000, end_id=1729061865049785001) [2024-10-16 14:57:45.050833] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=20] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:45.050864] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=536, get_gts_with_stc_cnt=80, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=77, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:45.050873] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061865050857], gts:1729061865049785000, latest_srr:[mts=1729061865050857]}) [2024-10-16 14:57:45.066055] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77B3-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:45.069433] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.074909] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:45.084684] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=25] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:45.085405] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:45.085429] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:45.088206] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=19] PNIO [ratelimit] time: 1729061865088204, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007608, add_bytes: 0 [2024-10-16 14:57:45.089169] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:57:45.089889] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:45.089914] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:45.089938] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:45.090039] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061877008855000, latest_log_ts:{val:1729061860546474001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:45.090084] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=42] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:45.090097] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:57:45.090114] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=15] get rec log scn(service_type_=0, rec_log_ts={val:1729061860546474001, v:0}) [2024-10-16 14:57:45.090121] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:45.090125] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:45.090132] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:45.090146] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:57:45.090156] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:45.090161] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:45.090496] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FE5-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:45.090699] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=277, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:45.090715] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=381, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:45.090743] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:45.095805] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:45.095827] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151784220, tenant_id=1) [2024-10-16 14:57:45.095843] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:45.116856] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:45.116875] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:45.119623] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:45.120922] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061865119325000, start_id=1729061865119325000, end_id=1729061865119325001) [2024-10-16 14:57:45.121507] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061865121314000, start_id=1729061865121314000, end_id=1729061865121314001) [2024-10-16 14:57:45.130931] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061865130816000, start_id=1729061865130816000, end_id=1729061865130816001) [2024-10-16 14:57:45.132728] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2532, mem_used=6239232) [2024-10-16 14:57:45.133115] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.139396] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061865137724000, start_id=1729061865137724000, end_id=1729061865137724001) [2024-10-16 14:57:45.139456] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:45.139469] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:45.139504] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:45.139812] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=293, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:45.139826] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=183, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:45.139853] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10239) [2024-10-16 14:57:45.139863] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=11, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:45.140204] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5867-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:45.140210] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5867-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=857, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:45.140214] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5867-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=19, avg_log_gen_to_submit_time=47, avg_log_submit_to_flush_time=631, avg_log_submit_to_slide_time=810) [2024-10-16 14:57:45.140350] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get_max_applied_scn(scn={val:1729061865002664000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618552545}, palf_committed_end_scn_:{val:1729061865137724000, v:0}, last_check_scn_:{val:1729061865137724000, v:0}, max_applied_cb_scn_:{val:1729061865002664000, v:0}}) [2024-10-16 14:57:45.140374] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:45.146064] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9ED-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061865145959000, start_id=1729061865145959000, end_id=1729061865145959001) [2024-10-16 14:57:45.146111] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969B9F0-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061865145959000, start_id=1729061865145959000, end_id=1729061865145959001) [2024-10-16 14:57:45.146232] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9F1-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061865145959000, start_id=1729061865145959000, end_id=1729061865145959001) [2024-10-16 14:57:45.146377] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9EF-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061865145959000, start_id=1729061865145959000, end_id=1729061865145959001) [2024-10-16 14:57:45.146380] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B9EE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061865145959000, start_id=1729061865145959000, end_id=1729061865145959001) [2024-10-16 14:57:45.169494] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.178592] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA883-0-0] [lt=48] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:45.178625] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA883-0-0] [lt=22] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061864646059004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061864646059004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:45.182112] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA883-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061865181380000, start_id=1729061865181380000, end_id=1729061865181380001) [2024-10-16 14:57:45.182189] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA883-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10241, avg_group_log_size=568) [2024-10-16 14:57:45.182199] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA883-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10240) [2024-10-16 14:57:45.182463] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10241) [2024-10-16 14:57:45.182679] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:45.185685] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2182) [2024-10-16 14:57:45.189066] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502F-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:45.189082] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD502F-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:45.189325] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31906][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB20CC6DB-0-0] [lt=22] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:45.190512] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FE7-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:45.190870] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061865189770000, start_id=1729061865189770000, end_id=1729061865189770001) [2024-10-16 14:57:45.191108] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=197, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:45.191123] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=367, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:45.191142] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:45.191365] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1045-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=606, avg_log_submit_to_slide_time=976) [2024-10-16 14:57:45.203293] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061865202564000, start_id=1729061865202564000, end_id=1729061865202564001) [2024-10-16 14:57:45.206110] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EAA-0-0] [lt=24] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061865173782, keepalive_ts:-1}) [2024-10-16 14:57:45.206134] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EAA-0-0] [lt=23] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061865173782, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:45.206154] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EAB-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061865173836, keepalive_ts:-1}) [2024-10-16 14:57:45.206160] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EAB-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061865173836, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:45.209358] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:45.210970] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=54] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:57:45.210983] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=13] try gc retain ctx [2024-10-16 14:57:45.210987] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:45.211611] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061875590307000, latest_log_ts:{val:1729061858932674001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:45.212024] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601418, offset=9630106}, end_cursor:ObLogCursor{file_id=192, log_id=47601419, offset=9630390}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cf0050, len:3686, log_data_len:284, seq:47601418, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:45.213861] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A7-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061865212921000, start_id=1729061865212921000, end_id=1729061865212921001) [2024-10-16 14:57:45.214441] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F416C-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:45.214702] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=50] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601419, offset=9630390}, end_cursor:ObLogCursor{file_id=192, log_id=47601420, offset=9630664}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72f2050, len:3402, log_data_len:274, seq:47601419, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:45.216460] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601420, offset=9630664}, end_cursor:ObLogCursor{file_id=192, log_id=47601421, offset=9630938}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d80050, len:3128, log_data_len:274, seq:47601420, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:45.216629] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601421, offset=9630938}, end_cursor:ObLogCursor{file_id=192, log_id=47601422, offset=9631212}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d78050, len:2854, log_data_len:274, seq:47601421, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:45.216673] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561312, miss_cnt:1, cache_read_size:152590682760, hit_ratio:"9.999999928346905431e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:45.216701] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598899, miss_cnt:9, cache_read_size:149777916271, hit_ratio:"9.999999345925041361e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:45.220945] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D8F-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510333089340, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410604, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:45.222226] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C106B-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061865221975000, start_id=1729061865221975000, end_id=1729061865221975001) [2024-10-16 14:57:45.230210] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257766, offset=39034160}, end_cursor:ObLogCursor{file_id=197, log_id=49257767, offset=39034443}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2ddc050, len:720, log_data_len:283, seq:49257766, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:45.230224] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061864930554000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=299664, in_cluster_service=true, cluster_version={val:1729061864930554000, v:0}, min_cluster_version={val:1729061864930554000, v:0}, max_cluster_version={val:1729061864930554000, v:0}, get_cluster_version_err=0, cluster_version_delta=299664, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244245, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=474300021, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:45.232113] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22628][T1004_TeRec][T1004][YB4A0A000407-00061E9B221FB372-0-0] [lt=23] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=0, MAX_GAP=900000000000, new_sync_scn={val:1729061864049233001, v:0}, new_replayable_scn={val:1729061864049233001, v:0}, new_readable_scn={val:1729061864049233001, v:0}, old_tenant_info={tenant_id:1004, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061863049052001, v:0}, replayable_scn:{val:1729061863049052001, v:0}, standby_scn:{val:1729061863049052001, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"ARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:57:45.232302] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB372-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061865231619000, start_id=1729061865231619000, end_id=1729061865231619001) [2024-10-16 14:57:45.232445] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107492][][T1003][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1003, ret=0, thread_count_=173) [2024-10-16 14:57:45.232463] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107492][][T1003][Y0-0000000000000000-0-0] [lt=16] Init thread local success [2024-10-16 14:57:45.232512] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107492][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C86-0-0] [lt=4] init point thread id with(&point=0x55e3435ca780, idx_=996, point=[thread id=107492, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107492) [2024-10-16 14:57:45.232630] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=18] worker thread created(tenant_->id()=1003, group_id_=1, token=1) [2024-10-16 14:57:45.233164] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.233303] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=24] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257767, offset=39034443}, end_cursor:ObLogCursor{file_id=197, log_id=49257768, offset=39034725}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2de4050, len:437, log_data_len:282, seq:49257767, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:45.235343] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257768, offset=39034725}, end_cursor:ObLogCursor{file_id=197, log_id=49257769, offset=39035007}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d44050, len:4251, log_data_len:282, seq:49257768, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:45.235514] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257769, offset=39035007}, end_cursor:ObLogCursor{file_id=197, log_id=49257770, offset=39035289}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7396050, len:3969, log_data_len:282, seq:49257769, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:45.235559] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656987, miss_cnt:17, cache_read_size:8050169079, hit_ratio:"9.999997286815692776e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:45.235582] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650250, miss_cnt:32, cache_read_size:8050421698, hit_ratio:"9.999994892281570680e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:45.250735] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=202, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:45.250761] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:45.250792] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:45.251023] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD27B-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:45.251039] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD27B-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=548, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:45.266217] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77B7-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:45.266698] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F416D-0-0] [lt=20] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:45.266708] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F416D-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20437, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:45.269562] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.276079] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C97-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:45.276301] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=218, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:45.276323] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=29, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:45.276358] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:45.276534] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FED9-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:45.276546] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FED9-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9530, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:45.276550] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FED9-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=273, avg_log_submit_to_slide_time=9523) [2024-10-16 14:57:45.276578] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:45.285507] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2182, avg_group_log_size=198) [2024-10-16 14:57:45.285527] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2182) [2024-10-16 14:57:45.285553] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1210, avg_group_log_size=121) [2024-10-16 14:57:45.285558] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1210) [2024-10-16 14:57:45.285778] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2182) [2024-10-16 14:57:45.285801] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1210) [2024-10-16 14:57:45.292958] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1038-0-0] [lt=12] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:45.293873] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:45.317655] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7302, mem_used=14558208) [2024-10-16 14:57:45.318368] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=15][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:45.319699] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:45.321201] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061865320169000, start_id=1729061865320169000, end_id=1729061865320169001) [2024-10-16 14:57:45.329697] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=10] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:45.331063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061865330813000, start_id=1729061865330813000, end_id=1729061865330813001) [2024-10-16 14:57:45.333223] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.335960] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=6144, avg_group_log_size=512) [2024-10-16 14:57:45.338750] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=40] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061865202564000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=136180, in_cluster_service=true, cluster_version={val:1729061864802925000, v:0}, min_cluster_version={val:1729061864802925000, v:0}, max_cluster_version={val:1729061864802925000, v:0}, get_cluster_version_err=0, cluster_version_delta=535819, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244493, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=239734915, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:45.339555] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061865338273000, start_id=1729061865338273000, end_id=1729061865338273001) [2024-10-16 14:57:45.340741] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1001, new_min_version={val:1729061865137724000, v:0}, new_max_version={val:1729061865137724000, v:0}, last_min_version={val:1729061864802925000, v:0}, last_max_version={val:1729061864802925000, v:0}, total_time_us=1958, delta_us=203015) [2024-10-16 14:57:45.340759] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1001, current_version={val:1729061865137724000, v:0}, min_version={val:1729061865137724000, v:0}, max_version={val:1729061865137724000, v:0}, in_service=true, leader_epoch=33705) [2024-10-16 14:57:45.352595] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B916D-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=249, avg_log_submit_to_slide_time=699) [2024-10-16 14:57:45.352619] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:45.366724] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:45.366778] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=48, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:45.366792] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:45.368737] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAF-0-0] [lt=3] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:45.368751] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AAF-0-0] [lt=9] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=15) [2024-10-16 14:57:45.369622] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.376438] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2C99-0-0] [lt=33] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:45.380915] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:45.380937] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=215) [2024-10-16 14:57:45.385780] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:45.391290] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061865390345000, start_id=1729061865390345000, end_id=1729061865390345001) [2024-10-16 14:57:45.391359] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:45.391376] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:45.394674] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:45.396459] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=9] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:45.403492] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061865403039000, start_id=1729061865403039000, end_id=1729061865403039001) [2024-10-16 14:57:45.409646] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:45.413202] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:45.413221] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:45.413544] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:45.433291] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.455972] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:7668488755}, base_lsn={lsn:7582838784}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:45.456031] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=59] get_replay_process success(id={id:1}, replayed_log_size=85649971, unreplayed_log_size=0) [2024-10-16 14:57:45.456042] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=7] replay status is not follower(min_unreplayed_lsn={lsn:6198988121}, base_lsn={lsn:6106533888}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:45.456072] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=30] get_replay_process success(id={id:1001}, replayed_log_size=92454233, unreplayed_log_size=0) [2024-10-16 14:57:45.456077] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=8163539897, unreplayed_log_size=0) [2024-10-16 14:57:45.456083] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1003}, replayed_log_size=8111635800, unreplayed_log_size=0) [2024-10-16 14:57:45.456101] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1004, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=15691, last_replayed_log_size(MB)=15691, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:45.466491] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=217, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:45.466516] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=144, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:45.469671] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.473350] INFO [SHARE] get_server_tenant_configs (ob_unit_getter.cpp:231) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=7] get_server_tenant_configs(unit_infos=[{unit:unit_id:1, resource_pool_id:1, unit_group_id:1, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, pool:{resource_pool_id:1, name:"sys_pool", unit_count:1, unit_config_id:1, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1, replica_type:0}}, {unit:unit_id:1001, resource_pool_id:1001, unit_group_id:1001, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1001, name:"ocp_pool", unit_count:1, unit_config_id:1001, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1002, replica_type:0}}, {unit:unit_id:1005, resource_pool_id:1003, unit_group_id:1002, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1003, name:"pool_bddb_myoceanbase1_oav", unit_count:1, unit_config_id:1003, zone_list:["myoceanbase1"], tenant_id:1004, replica_type:0}}]) [2024-10-16 14:57:45.473394] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=40] jx_debug: get tenant compatibility mode(tenant_id=1002, compat_mode=0) [2024-10-16 14:57:45.473400] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=5] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:45.473414] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=13] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:45.473430] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=15] jx_debug: get tenant compatibility mode(tenant_id=1004, compat_mode=0) [2024-10-16 14:57:45.473433] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=2] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:45.473442] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:45.474647] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A83-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:45.474663] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A83-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:45.475543] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:191) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=10] refresh resource plan success(tenant_id=1, plan_name=, directives=[]) [2024-10-16 14:57:45.476213] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] refresh resource user mapping rule(tenant_id=1, plan=, user_rules=[]) [2024-10-16 14:57:45.476880] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] refresh_resource_function_mapping_rule(tenant_id=1, plan=, rules=[]) [2024-10-16 14:57:45.476889] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] refresh resource mapping rule success(tenant_id=1, plan=) [2024-10-16 14:57:45.477517] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=30, max_blocked_buffer_size_=9830400, max_buffer_size_=19660800) [2024-10-16 14:57:45.477575] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:45.477582] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=6] tenant dfc status(ret=0, get_tenant_id()=1, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=30, get_max_blocked_buffer_size()=9830400, get_max_buffer_size()=19660800, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:45.479510] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=7] refresh resource user mapping rule(tenant_id=1001, plan=, user_rules=[]) [2024-10-16 14:57:45.480172] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] refresh_resource_function_mapping_rule(tenant_id=1001, plan=, rules=[]) [2024-10-16 14:57:45.480180] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] refresh resource mapping rule success(tenant_id=1001, plan=) [2024-10-16 14:57:45.480859] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:45.480915] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:45.480923] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=5] tenant dfc status(ret=0, get_tenant_id()=1001, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:45.482869] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=13] refresh resource user mapping rule(tenant_id=1002, plan=, user_rules=[]) [2024-10-16 14:57:45.483537] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] refresh_resource_function_mapping_rule(tenant_id=1002, plan=, rules=[]) [2024-10-16 14:57:45.483545] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] refresh resource mapping rule success(tenant_id=1002, plan=) [2024-10-16 14:57:45.484198] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:45.484256] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:45.484262] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=5] tenant dfc status(ret=0, get_tenant_id()=1002, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=4, get_max_size_per_channel()=655360) [2024-10-16 14:57:45.487430] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=6] refresh resource user mapping rule(tenant_id=1003, plan=, user_rules=[]) [2024-10-16 14:57:45.488103] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] refresh_resource_function_mapping_rule(tenant_id=1003, plan=, rules=[]) [2024-10-16 14:57:45.488111] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] refresh resource mapping rule success(tenant_id=1003, plan=) [2024-10-16 14:57:45.488326] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=63] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:45.488738] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:45.488793] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:45.488799] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=5] tenant dfc status(ret=0, get_tenant_id()=1003, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:45.488897] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:45.488941] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=8] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061858725452000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:45.489163] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:45.490104] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:45.490113] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:45.490122] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:45.490132] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:45.490135] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:45.490138] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:45.490141] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:45.490143] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:45.490145] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:45.490148] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:45.490151] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:45.490168] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:45.490175] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=4] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:45.490186] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=70718110, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:45.490324] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] get_max_applied_scn(scn={val:1729061865390345000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390138489}, palf_committed_end_scn_:{val:1729061865390345000, v:0}, last_check_scn_:{val:1729061865390345000, v:0}, max_applied_cb_scn_:{val:1729061865390345000, v:0}}) [2024-10-16 14:57:45.490341] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:45.490735] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=6] refresh resource user mapping rule(tenant_id=1004, plan=, user_rules=[]) [2024-10-16 14:57:45.490751] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A24-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:45.490759] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A24-0-0] [lt=7] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:45.491377] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] refresh_resource_function_mapping_rule(tenant_id=1004, plan=, rules=[]) [2024-10-16 14:57:45.491387] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] refresh resource mapping rule success(tenant_id=1004, plan=) [2024-10-16 14:57:45.491413] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FED-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:45.491421] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FED-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=34401, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:45.491425] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FED-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497453, log_task={header:{begin_lsn:{lsn:207589551952}, end_lsn:{lsn:207589552074}, log_id:218497453, min_scn:{val:1729061865145959002, v:0}, max_scn:{val:1729061865145959002, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589551830}, prev_proposal_id:81, committed_end_lsn:{lsn:207589551952}, data_checksum:940009244, accum_checksum:1793798551, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061865390636, freeze_ts:1729061865390636, submit_ts:1729061865390640, flushed_ts:1729061865391662, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1022}, fs_cb_begin_ts=1729061865491406, log_life_time=100770) [2024-10-16 14:57:45.492049] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:45.492116] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=18] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:45.492125] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] tenant dfc status(ret=0, get_tenant_id()=1004, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:45.492142] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:108) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] refresh tenant units(sys_unit_cnt=3, units=[{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, {tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, {tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}], ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:45.492172] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=28] finish set nway(tenant_id_=1, nway=3) [2024-10-16 14:57:45.492181] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1, nway=3, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}) [2024-10-16 14:57:45.492194] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=11] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=1.638399963378906250e+02, this=0x2b0ddc2dc170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=819, Throttle Trigger(MB)=491, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.948457122979588263e+01) [2024-10-16 14:57:45.492208] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=13] succ to set_memstore_threshold of memstore allocator(tenant_id=1, ret=0) [2024-10-16 14:57:45.492213] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=3] finish set nway(tenant_id_=1002, nway=1) [2024-10-16 14:57:45.492219] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=5] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1002, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:45.492232] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e0c298170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:45.492238] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=6] succ to set_memstore_threshold of memstore allocator(tenant_id=1002, ret=0) [2024-10-16 14:57:45.492243] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=3] finish set nway(tenant_id_=1001, nway=1) [2024-10-16 14:57:45.492248] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=4] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1001, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:45.492257] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0df7098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:45.492266] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] succ to set_memstore_threshold of memstore allocator(tenant_id=1001, ret=0) [2024-10-16 14:57:45.492271] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=3] finish set nway(tenant_id_=1004, nway=1) [2024-10-16 14:57:45.492276] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=6] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1004, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:45.492285] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e40098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:45.492294] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] succ to set_memstore_threshold of memstore allocator(tenant_id=1004, ret=0) [2024-10-16 14:57:45.492299] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=2] finish set nway(tenant_id_=1003, nway=1) [2024-10-16 14:57:45.492304] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=5] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1003, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:45.492314] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e2c698170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:45.492323] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=8] succ to set_memstore_threshold of memstore allocator(tenant_id=1003, ret=0) [2024-10-16 14:57:45.492907] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:45.492921] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=13] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1001, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:45.492925] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=4] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:45.492929] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=4] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1003, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:45.492936] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=6] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:45.492954] INFO [SERVER] cal_all_part_disk_default_percentage (ob_server_utils.cpp:301) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=3] cal_all_part_disk_default_percentage succ(data_dir="/mnt/myoceanbase/oceanbase/data//sstable", clog_dir="/mnt/myoceanbase/oceanbase/data//clog", shared_mode=true, data_disk_total_size=214429704192, data_disk_default_percentage=60, clog_disk_total_size=214429704192, clog_disk_default_percentage=30) [2024-10-16 14:57:45.492963] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:337) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=9] decide disk size finished(suggested_disk_size=32212254720, suggested_disk_percentage=0, default_disk_percentage=30, total_space=214429704192, disk_size=32212254720) [2024-10-16 14:57:45.492968] INFO [SERVER] get_log_disk_info_in_config (ob_server_utils.cpp:88) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=5] get_log_disk_info_in_config(suggested_data_disk_size=32212254720, suggested_clog_disk_size=32212254720, suggested_data_disk_percentage=0, suggested_clog_disk_percentage=0, log_disk_size=32212254720, log_disk_percentage=0, total_log_disk_size=214429704192) [2024-10-16 14:57:45.492977] INFO [CLOG] try_resize (ob_server_log_block_mgr.cpp:800) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=7] try_resize success(ret=0, log_disk_size=32212254720, total_log_disk_size=214429704192, this={dir::"/mnt/myoceanbase/oceanbase/data//clog/log_pool", dir_fd:15, meta_fd:16, log_pool_meta:{curr_total_size:32212254720, next_total_size:32212254720, status:0}, min_block_id:15037, max_block_id:15335, min_log_disk_size_for_all_tenants_:15032385536, is_inited:true}) [2024-10-16 14:57:45.492988] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:132) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0C-0-0] [lt=10] refresh tenant config(tenants=[1, 1001, 1002, 1003, 1004], ret=0) [2024-10-16 14:57:45.493125] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=46, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:57:45.493135] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=1689, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:57:45.493139] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:57:45.493147] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=8] [APPLY STAT CB EXECUTE TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=-785, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:57:45.493150] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=950, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:57:45.493227] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FEE1-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:45.493232] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FEE1-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1254, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:45.509010] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=14] replay status is not follower(min_unreplayed_lsn={lsn:45686897415}, base_lsn={lsn:45497032704}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:45.509067] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=56] get_replay_process success(id={id:1}, replayed_log_size=189864711, unreplayed_log_size=0) [2024-10-16 14:57:45.509090] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=11] dump tenant replay process(tenant_id=1001, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=181, last_replayed_log_size(MB)=181, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:45.513183] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:45.513203] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:45.513534] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1646) [2024-10-16 14:57:45.513744] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4175-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:45.513754] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4175-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=555, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:45.513791] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4174-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=381, avg_log_submit_to_slide_time=582) [2024-10-16 14:57:45.519753] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:45.520417] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061865049785002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=470625, in_cluster_service=true, cluster_version={val:1729061864646059003, v:0}, min_cluster_version={val:1729061864646059003, v:0}, max_cluster_version={val:1729061864646059003, v:0}, get_cluster_version_err=0, cluster_version_delta=874351, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228272, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=410911931, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:45.521348] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061865520753000, start_id=1729061865520753000, end_id=1729061865520753001) [2024-10-16 14:57:45.531185] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061865530463000, start_id=1729061865530463000, end_id=1729061865530463001) [2024-10-16 14:57:45.531487] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=22] [PALF STAT WRITE LOG COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=701, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:45.531503] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=187, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:45.531629] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:45.532191] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C8D-0-0] [lt=23] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=12737) [2024-10-16 14:57:45.532230] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C8D-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:45.532235] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C8D-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=1210, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:45.532238] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C8D-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=22, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=918, avg_log_submit_to_slide_time=1181) [2024-10-16 14:57:45.533349] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.539696] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061865539038000, start_id=1729061865539038000, end_id=1729061865539038001) [2024-10-16 14:57:45.542293] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=909004800, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=164737024, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:45.542349] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=24] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:45.542357] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:45.542389] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=967602176, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=106139648, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:45.542427] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:45.542435] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:45.542465] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=940462080, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=133279744, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:45.542502] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:45.542509] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011865, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:45.542540] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=956841984, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=116899840, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:45.542575] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:45.542582] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:45.544137] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [22809][T1004_SSTableGC][T1004][YB4A0A000407-00061E997A9CB61C-0-0] [lt=9] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60379}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015311100270042}, old_min_reserved_snapshot=1729060061837115000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060061837115000) [2024-10-16 14:57:45.551204] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061865550656000, start_id=1729061865550656000, end_id=1729061865550656001) [2024-10-16 14:57:45.551229] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061865551225], gts:1729061865550656000, latest_srr:[mts=1729061865551225]}) [2024-10-16 14:57:45.554744] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:45.568779] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.568802] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.569619] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.569653] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.569702] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.572356] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC46-0-0] [lt=10] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:45.572377] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC46-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:45.572394] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC46-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:45.578865] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [22809][T1004_SSTableGC][T1004][YB4A0A000407-00061E997A9CB61C-0-0] [lt=24] SSTableGCTask(cost_ts=34969) [2024-10-16 14:57:45.585403] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1001}, {id:1003}]) [2024-10-16 14:57:45.586440] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FEE3-0-0] [lt=2] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=333, avg_log_submit_to_slide_time=1214) [2024-10-16 14:57:45.587875] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:45.587892] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:45.591179] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=40] get number(ret=0, service_type_=0, range=1, base_id=1729061865590789000, start_id=1729061865590789000, end_id=1729061865590789001) [2024-10-16 14:57:45.591196] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=11] new mysql sessid created(conn.sessid_=3221740079, support_ssl=false) [2024-10-16 14:57:45.591220] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=9] sm conn init succ(conn.sessid_=3221740079, sess.client_addr_="10.0.4.7:53418") [2024-10-16 14:57:45.591226] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0f27406030, session_id:3221740079, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:45.591242] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4392) [2024-10-16 14:57:45.591491] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=16] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:45.591500] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:295) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=8] mysql login queue(mysql_queue->queue_.size()=1) [2024-10-16 14:57:45.591922] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1049-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4392) [2024-10-16 14:57:45.592078] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B0-0-0] [lt=11] MySQL LOGIN(direct_client_ip="10.0.4.7", client_ip=10.0.4.7, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=%, sessid=3221740079, proxy_sessid=13882479996272574487, client_sessid=2107375639, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=4062023, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:45.592114] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4392) [2024-10-16 14:57:45.592234] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:45.592247] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=914, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:45.592328] INFO [SERVER] process (obmp_init_db.cpp:53) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55CA-0-0] [lt=2] init db(db_name=oceanbase) [2024-10-16 14:57:45.602689] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061865145959003, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=456724, in_cluster_service=true, cluster_version={val:1729061864646059004, v:0}, min_cluster_version={val:1729061864646059004, v:0}, max_cluster_version={val:1729061864646059004, v:0}, get_cluster_version_err=0, cluster_version_delta=956624, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244194, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=536932522, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:45.603537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061865603504000, start_id=1729061865603504000, end_id=1729061865603504001) [2024-10-16 14:57:45.604614] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D7-0-0] [lt=20] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061865572379, keepalive_ts:-1}) [2024-10-16 14:57:45.604630] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D7-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061865572379, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:45.608111] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C7-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:45.608135] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C7-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061865145959003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061865145959003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:45.609163] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9C7-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061865608724000, start_id=1729061865608724000, end_id=1729061865608724001) [2024-10-16 14:57:45.609563] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5101, mem_used=10398720) [2024-10-16 14:57:45.609938] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:45.611555] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C7-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061865611034000, start_id=1729061865611034000, end_id=1729061865611034001) [2024-10-16 14:57:45.611635] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C7-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=23, avg_log_batch_cnt=1, total_group_log_size=12899, avg_group_log_size=560) [2024-10-16 14:57:45.611646] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C7-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=12737) [2024-10-16 14:57:45.611848] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=12899) [2024-10-16 14:57:45.612249] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:45.633364] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.636031] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:57:45.636047] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=15] try gc retain ctx [2024-10-16 14:57:45.636052] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:45.636723] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061877008855000, latest_log_ts:{val:1729061860546474001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:45.644940] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=40] success to update min reserved snapshot(reserved_snapshot=1729060060198021000, duration=1800, snapshot_gc_ts=1729061860198021000) [2024-10-16 14:57:45.646167] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=16] update reserved snapshot finished(cost_ts=1230, reserved_snapshot=1729060060198021000) [2024-10-16 14:57:45.646418] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9F2-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061865645903000, start_id=1729061865645903000, end_id=1729061865645903001) [2024-10-16 14:57:45.646493] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B9F3-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061865645903000, start_id=1729061865645903000, end_id=1729061865645903001) [2024-10-16 14:57:45.646638] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9F6-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061865645903000, start_id=1729061865645903000, end_id=1729061865645903001) [2024-10-16 14:57:45.647413] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9F4-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061865646963000, start_id=1729061865646963000, end_id=1729061865646963001) [2024-10-16 14:57:45.659760] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E20-0-0] [lt=13][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173189, log_task={header:{begin_lsn:{lsn:14352379224}, end_lsn:{lsn:14352379346}, log_id:115173189, min_scn:{val:1729061865145959004, v:0}, max_scn:{val:1729061865145959004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352379102}, prev_proposal_id:17, committed_end_lsn:{lsn:14352379224}, data_checksum:940009244, accum_checksum:448286928, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061865550892, freeze_ts:1729061865550892, submit_ts:1729061865550897, flushed_ts:1729061865554769, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:3872}, fs_cb_begin_ts=1729061865659752, log_life_time=108860) [2024-10-16 14:57:45.661209] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C54-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:45.661236] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C54-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061865145959004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061865145959004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:45.664856] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C54-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061865664614000, start_id=1729061865664614000, end_id=1729061865664614001) [2024-10-16 14:57:45.668404] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:45.669762] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:45.669779] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:45.669785] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.669951] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77BF-0-0] [lt=8][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938308, log_task={header:{begin_lsn:{lsn:213058243144}, end_lsn:{lsn:213058243266}, log_id:220938308, min_scn:{val:1729061865145959005, v:0}, max_scn:{val:1729061865145959005, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058243022}, prev_proposal_id:88, committed_end_lsn:{lsn:213058243144}, data_checksum:1715074492, accum_checksum:1884541834, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061865566371, freeze_ts:1729061865566371, submit_ts:1729061865566377, flushed_ts:1729061865567829, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:1452}, fs_cb_begin_ts=1729061865669945, log_life_time=103574) [2024-10-16 14:57:45.670189] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:45.680649] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=7] query dest_info_ succ(dest_info=[[0, "file:///mnt/oceanbasedata/bddb/archive"]], archive_dest_changed=false) [2024-10-16 14:57:45.680665] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=13] finish to resize log external storage handler(current_ts=1729061865680663, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:45.689674] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D6-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061865680901, keepalive_ts:-1}) [2024-10-16 14:57:45.689691] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D6-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061865680901, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:45.698922] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:45.714253] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=229, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:45.714275] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=194, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:45.714310] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:45.719830] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:45.721459] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061865720947000, start_id=1729061865720947000, end_id=1729061865720947001) [2024-10-16 14:57:45.722348] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=22] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:45.724892] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:45.731406] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061865730456000, start_id=1729061865730456000, end_id=1729061865730456001) [2024-10-16 14:57:45.732179] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get_max_applied_scn(scn={val:1729061865611034000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264776248}, palf_committed_end_scn_:{val:1729061865730456000, v:0}, last_check_scn_:{val:1729061865730456000, v:0}, max_applied_cb_scn_:{val:1729061865611034000, v:0}}) [2024-10-16 14:57:45.732207] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:45.733425] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.736055] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:45.736070] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=9] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=215) [2024-10-16 14:57:45.739763] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061865738914000, start_id=1729061865738914000, end_id=1729061865738914001) [2024-10-16 14:57:45.750732] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E22-0-0] [lt=29] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:45.750753] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E22-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:45.750762] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E22-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:45.751024] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:111799417, max_lsn:17493475361, max_scn:{val:1729061865550656001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7667642653}, reuse_lsn:{lsn:17493475361}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17451532321}}, last_submit_lsn:{lsn:17493475239}, last_submit_end_lsn:{lsn:17493475361}, last_submit_log_id:111799417, last_submit_log_pid:17, max_flushed_lsn:{lsn:17493475239}, max_flushed_end_lsn:{lsn:17493475361}, max_flushed_log_pid:17, committed_end_lsn:{lsn:17493475361}, last_slide_log_id:111799417, last_slide_scn:{val:1729061865550656001, v:0}, last_slide_lsn:{lsn:17493475239}, last_slide_end_lsn:{lsn:17493475361}, last_slide_log_pid:17, last_slide_log_accum_checksum:1083783857, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431d4530}, [StateMgr]={this:0x2b0e431d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:111799418, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261853177, palf_role_change_cb:0x2b0e431df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:18}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:18}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494261753074, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494261753074, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262053331, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728249494772393002, v:0}, min_block_id:241, min_block_min_scn:{val:1728207118648006001, v:0}, base_lsn_for_block_gc:{lsn:17380134912}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:17380134912}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:757760}, readable_log_tail:{lsn:757760}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17514344448}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/meta", dir_fd:193, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66347008, block_header_serialize_buf_:0x2b0e431da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17493475361}, readable_log_tail:{lsn:17493475361}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17447239680}, min_scn:{val:1729032503563174001, v:0}, curr_block_id:260, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/log", dir_fd:195, min_block_id:241, max_block_id:261, curr_writable_block_id:260}, logical_block_size_:67104768, curr_block_writable_size_:20869087, block_header_serialize_buf_:0x2b0e431dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261852794, this:0x2b0e431d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:7668488755}, saved_end_lsn:{lsn:7668488755}, last_submit_prepare_req_time_us:1722494261853360, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431d89b0}) [2024-10-16 14:57:45.751222] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=194] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061865686270, last_flushed_end_lsn:{lsn:17493475361}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061865686671, last_flushed_end_lsn:{lsn:17493475361}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061865686646, last_flushed_end_lsn:{lsn:17493475361}}]) [2024-10-16 14:57:45.751233] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=111799418, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061865686270, last_flushed_end_lsn:{lsn:17493475361}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061865686671, last_flushed_end_lsn:{lsn:17493475361}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061865686646, last_flushed_end_lsn:{lsn:17493475361}}]) [2024-10-16 14:57:45.756693] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:45.756720] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:45.756731] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=10] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:45.766437] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77C1-0-0] [lt=19] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:45.769795] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.771450] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:45.782431] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=22] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:45.786368] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:45.788615] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788831] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788848] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788864] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788876] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788888] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788902] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788916] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788927] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788942] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.788957] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:45.791381] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061865790680000, start_id=1729061865790680000, end_id=1729061865790680001) [2024-10-16 14:57:45.804594] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061865804061000, start_id=1729061865804061000, end_id=1729061865804061001) [2024-10-16 14:57:45.810230] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:45.825723] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1001}, {id:1002}]) [2024-10-16 14:57:45.828609] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:57:45.828662] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=49] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:45.828674] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52461112, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:45.833486] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.840022] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=21] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:57:45.850920] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D71-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:45.850952] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D71-0-0] [lt=19] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061865646963000, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061865646963000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:45.851088] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061861843185, last_first_register_time_us=1722494157071428, register_time_us=1729061861843185, curr_time_us=1729061865851087) [2024-10-16 14:57:45.851123] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061865851103, last_submit_register_req_time_us=1729061865851103, register_parent_reason=3) [2024-10-16 14:57:45.851131] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:45.851433] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=53] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=32, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:45.851444] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:45.852020] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D71-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061865851479000, start_id=1729061865851479000, end_id=1729061865851479001) [2024-10-16 14:57:45.853432] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4770, mem_used=10398720) [2024-10-16 14:57:45.853917] INFO [LIB] runTimerTask (ob_work_queue.cpp:24) [21225][RSInspectTask][T0][Y0-0000000000000000-0-0] [lt=19] add async task(this=tasktype:N9oceanbase10rootserver11ObInspectorE) [2024-10-16 14:57:45.862290] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:45.865209] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D71-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061865864288000, start_id=1729061865864288000, end_id=1729061865864288001) [2024-10-16 14:57:45.865306] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D71-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:45.865321] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D71-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:45.869854] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.870449] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27406030, session_id:3221740079, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061865591851, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:794, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:45.870472] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=17] kill and revert session(conn.sessid_=3221740079, proxy_sessid=13882479996272574487, server_id=1, ret=0) [2024-10-16 14:57:45.870478] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] sock ref clean, do destroy(*s={this:0x2b0f27406030, session_id:3221740079, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061865591851, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:794, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:45.870490] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221740079, proxy_sessid=13882479996272574487, conn.tenant_id_=1) [2024-10-16 14:57:45.870495] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221740079, proxy_sessid=13882479996272574487, tenant_id=1, server_id=1, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B3A0A55CC-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:8, cur_pkt_pos_:10, pkt_rec[0]:{send:obp_mysql_header_{len_:12134, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{send:obp20_header_:{payload_len:14, pkt_seq:0, req_id:4287278, flag_.flags:42, ext_flags:0}, obp_mysql_header_{len_:554, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[2]:{send:obp20_header_:{payload_len:43, pkt_seq:0, req_id:4287279, flag_.flags:42, ext_flags:0}, obp_mysql_header_{len_:447, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{receive:obp20_header_:{payload_len:261, pkt_seq:0, req_id:4287280, flag_.flags:58, ext_flags:0}, obp_mysql_header_{len_:257, rec_:261, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[4]:{send:obp_compess_header_{com_len:547, com_seq:1}, obp20_header_:{payload_len:519, pkt_seq:1, req_id:4287280, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_compess_header_{com_len:547, com_seq:1}, obp20_header_:{payload_len:519, pkt_seq:1, req_id:4287280, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_compess_header_{com_len:547, com_seq:1}, obp20_header_:{payload_len:519, pkt_seq:1, req_id:4287280, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{send:obp_compess_header_{com_len:547, com_seq:1}, obp20_header_:{payload_len:519, pkt_seq:1, req_id:4287280, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[8]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:4287280, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:35, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:4287280, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:12, seq_:14}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:45.870540] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=26][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27404030, session_id:3221734324, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:1729061865587348, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:1398, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:45.870552] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9] kill and revert session(conn.sessid_=3221734324, proxy_sessid=13882479996272574487, server_id=1, ret=0) [2024-10-16 14:57:45.870555] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] sock ref clean, do destroy(*s={this:0x2b0f27404030, session_id:3221734324, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:1729061865587348, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:1398, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:45.870564] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221734324, proxy_sessid=13882479996272574487, conn.tenant_id_=1) [2024-10-16 14:57:45.870568] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221734324, proxy_sessid=13882479996272574487, tenant_id=1, server_id=1, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B304F3D91-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:25, cur_pkt_pos_:31, pkt_rec[0]:{send:obp_mysql_header_{len_:12134, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{send:obp20_header_:{payload_len:14, pkt_seq:0, req_id:11004258, flag_.flags:42, ext_flags:0}, obp_mysql_header_{len_:554, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[2]:{send:obp20_header_:{payload_len:43, pkt_seq:0, req_id:11004259, flag_.flags:42, ext_flags:0}, obp_mysql_header_{len_:447, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{receive:obp20_header_:{payload_len:162, pkt_seq:0, req_id:11004260, flag_.flags:58, ext_flags:0}, obp_mysql_header_{len_:158, rec_:162, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[4]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004260, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004260, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004260, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004260, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[8]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:11004260, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:35, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:11004260, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:12, seq_:14}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[10]:{receive:obp20_header_:{payload_len:261, pkt_seq:0, req_id:11004261, flag_.flags:62, ext_flags:0}, obp_mysql_header_{len_:257, rec_:261, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[11]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004261, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[12]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004261, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[13]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004261, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[14]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004261, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[15]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004261, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[16]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:11004261, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:12, seq_:14}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[17]:{receive:obp20_header_:{payload_len:175, pkt_seq:0, req_id:11004262, flag_.flags:62, ext_flags:0}, obp_mysql_header_{len_:171, rec_:175, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[18]:{send:obp_compess_header_{com_len:748, com_seq:1}, obp20_header_:{payload_len:720, pkt_seq:1, req_id:11004262, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[19]:{send:obp_compess_header_{com_len:748, com_seq:1}, obp20_header_:{payload_len:720, pkt_seq:1, req_id:11004262, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[20]:{send:obp_compess_header_{com_len:748, com_seq:1}, obp20_header_:{payload_len:720, pkt_seq:1, req_id:11004262, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[21]:{send:obp_compess_header_{com_len:748, com_seq:1}, obp20_header_:{payload_len:720, pkt_seq:1, req_id:11004262, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:4, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[22]:{send:obp_compess_header_{com_len:748, com_seq:1}, obp20_header_:{payload_len:720, pkt_seq:1, req_id:11004262, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[23]:{send:obp_compess_header_{com_len:748, com_seq:1}, obp20_header_:{payload_len:720, pkt_seq:1, req_id:11004262, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:12, seq_:13}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[24]:{receive:obp20_header_:{payload_len:162, pkt_seq:0, req_id:11004263, flag_.flags:62, ext_flags:0}, obp_mysql_header_{len_:158, rec_:162, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[25]:{send:obp_compess_header_{com_len:589, com_seq:1}, obp20_header_:{payload_len:561, pkt_seq:1, req_id:11004263, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[26]:{send:obp_compess_header_{com_len:589, com_seq:1}, obp20_header_:{payload_len:561, pkt_seq:1, req_id:11004263, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[27]:{send:obp_compess_header_{com_len:589, com_seq:1}, obp20_header_:{payload_len:561, pkt_seq:1, req_id:11004263, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[28]:{send:obp_compess_header_{com_len:589, com_seq:1}, obp20_header_:{payload_len:561, pkt_seq:1, req_id:11004263, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[29]:{send:obp_compess_header_{com_len:589, com_seq:1}, obp20_header_:{payload_len:561, pkt_seq:1, req_id:11004263, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[30]:{send:obp_compess_header_{com_len:589, com_seq:1}, obp20_header_:{payload_len:561, pkt_seq:1, req_id:11004263, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:12, seq_:14}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:45.870742] INFO [SERVER] run (obmp_disconnect.cpp:82) [21661][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=10] free session successfully(sessid=3221740079) [2024-10-16 14:57:45.870749] INFO [SERVER] run (obmp_disconnect.cpp:89) [21661][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221740079) [2024-10-16 14:57:45.870775] INFO [SERVER] run (obmp_disconnect.cpp:82) [21661][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221734324) [2024-10-16 14:57:45.870778] INFO [SERVER] run (obmp_disconnect.cpp:89) [21661][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221734324) [2024-10-16 14:57:45.876731] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CA3-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:45.876753] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CA3-0-0] [lt=13][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178498, log_task={header:{begin_lsn:{lsn:14538493223}, end_lsn:{lsn:14538493345}, log_id:115178498, min_scn:{val:1729061865646963001, v:0}, max_scn:{val:1729061865646963001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538493101}, prev_proposal_id:23, committed_end_lsn:{lsn:14538493223}, data_checksum:1715074492, accum_checksum:2445663305, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061865776654, freeze_ts:1729061865776654, submit_ts:1729061865776659, flushed_ts:1729061865776900, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:241}, fs_cb_begin_ts=1729061865876747, log_life_time=100093) [2024-10-16 14:57:45.877784] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:45.886135] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:45.886150] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:45.888086] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=268, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:45.888100] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=547, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:45.888133] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:45.888138] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1466, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:45.888142] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:45.889211] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:45.889396] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D48-0-0] [lt=7] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061865550656002, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061863049052000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880098662000, latest_log_ts:{val:1729061863549302001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:45.889529] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D48-0-0] [lt=90] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061865550656002, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061863049052000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:45.889594] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D48-0-0] [lt=30] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061865646963002, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061863049052000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:45.889664] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D48-0-0] [lt=37] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061865646963000, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061863049052000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:45.889694] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D48-0-0] [lt=29] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:45.893843] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:45.901627] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:45.903886] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:45.913545] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:45.914024] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4183-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:45.914035] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4183-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=677, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:45.923452] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:45.923473] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=9] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:45.925508] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061865924337000, start_id=1729061865924337000, end_id=1729061865924337001) [2024-10-16 14:57:45.929736] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] [APPLY STAT CB APPEND COST TIME](cur_stat_count=50, stat_interval=5000000, avg (count/cost)=43, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:57:45.929747] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=50, stat_interval=5000000, avg (count/cost)=1200, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:57:45.929751] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=50, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:57:45.929754] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=50, stat_interval=5000000, avg (count/cost)=-547, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:57:45.929757] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=2] [APPLY STAT CB TOTAL TIME](cur_stat_count=50, stat_interval=5000000, avg (count/cost)=696, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:57:45.933384] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=46] get number(ret=0, service_type_=0, range=1, base_id=1729061865931617000, start_id=1729061865931617000, end_id=1729061865931617001) [2024-10-16 14:57:45.933585] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.936248] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10002, add_timestamp:1729061865936238}) [2024-10-16 14:57:45.936286] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E43-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10002, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061865936262, sql_renew_time:1729061865936262, is_mark_fail:false}]) [2024-10-16 14:57:45.936312] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E43-0-0] [lt=25] success to process renew task(task={tenant_id:1, table_id:10002, add_timestamp:1729061865936238}, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061865936262, sql_renew_time:1729061865936262, is_mark_fail:false}]) [2024-10-16 14:57:45.936691] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=18][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936709] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=15][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936750] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=16] LogLoopThread round_cost_time(us)(round_cost_time=11) [2024-10-16 14:57:45.936814] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936822] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936828] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936832] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936836] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936841] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936845] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936849] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936853] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936857] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936861] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936865] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936869] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936873] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936877] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936881] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936885] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936891] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936895] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936899] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936903] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936908] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.936913] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.936920] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997708-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_STAT' [2024-10-16 14:57:45.938343] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938361] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=16][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938370] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938378] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938383] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938391] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938395] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938403] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938407] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938414] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938418] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938425] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938429] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938437] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938444] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=5][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938450] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938454] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938461] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938465] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938473] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938477] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938484] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938488] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938496] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938500] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938507] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938511] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938518] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938522] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938529] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938533] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938541] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938548] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938555] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938559] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938566] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938570] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938577] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938581] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938588] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938592] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938596] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938608] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=9][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938615] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938619] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938626] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938630] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938637] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938641] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938648] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938652] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938659] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938663] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938671] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938675] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938682] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938686] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938693] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938697] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938705] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938708] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938716] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938719] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938727] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938730] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938737] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938741] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938748] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938752] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938760] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938763] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938770] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938774] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938782] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938785] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938793] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938796] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938804] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938808] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938815] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938819] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938826] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938830] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938837] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938841] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938848] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938852] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938860] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938863] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938871] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938875] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938880] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938885] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938892] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938896] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938903] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938907] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938911] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938918] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=5][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938925] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938929] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938936] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938940] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938947] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938951] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938959] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938963] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938970] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938974] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938981] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938985] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938991] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.938995] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PLAN_CACHE_PLAN_STAT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.938999] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997709-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PLAN_CACHE_PLAN_STAT' [2024-10-16 14:57:45.940019] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"SCHEMATA", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940034] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=13][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'SCHEMATA' [2024-10-16 14:57:45.940042] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"SCHEMATA", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940048] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'SCHEMATA' [2024-10-16 14:57:45.940060] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"SCHEMATA", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940065] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'SCHEMATA' [2024-10-16 14:57:45.940069] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"SCHEMATA", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940073] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'SCHEMATA' [2024-10-16 14:57:45.940080] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"SCHEMATA", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940084] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'SCHEMATA' [2024-10-16 14:57:45.940088] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"SCHEMATA", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940092] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'SCHEMATA' [2024-10-16 14:57:45.940274] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF1AD-0-0] [lt=16] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.131891441345214844e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=306, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.972258380987642795e+03) [2024-10-16 14:57:45.940426] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061865938879000, start_id=1729061865938879000, end_id=1729061865938879001) [2024-10-16 14:57:45.940861] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"CHARACTER_SETS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940884] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770B-0-0] [lt=21][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'CHARACTER_SETS' [2024-10-16 14:57:45.940892] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770B-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"CHARACTER_SETS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940897] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770B-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'CHARACTER_SETS' [2024-10-16 14:57:45.940901] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"CHARACTER_SETS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940907] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770B-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'CHARACTER_SETS' [2024-10-16 14:57:45.940912] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770B-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"CHARACTER_SETS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.940916] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770B-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'CHARACTER_SETS' [2024-10-16 14:57:45.941626] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770C-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"GLOBAL_VARIABLES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.941644] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770C-0-0] [lt=16][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GLOBAL_VARIABLES' [2024-10-16 14:57:45.941652] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770C-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"GLOBAL_VARIABLES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.941660] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770C-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GLOBAL_VARIABLES' [2024-10-16 14:57:45.942618] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3040, mem_used=6239232) [2024-10-16 14:57:45.946216] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:45.946234] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=13][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:45.946247] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:45.946255] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:45.946264] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:45.946289] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:45.946831] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946850] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=13][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946859] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946869] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=9][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946874] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946881] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946890] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946898] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946902] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946909] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946914] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946920] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946927] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946935] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946939] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946947] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946953] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946961] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946968] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946975] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946981] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.946986] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.946999] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=9][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.947006] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.947013] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.947021] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.947027] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.947034] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.947040] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.947044] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.947054] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=5][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.947061] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.947067] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.947075] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.947079] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"STATISTICS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:45.947086] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770D-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'STATISTICS' [2024-10-16 14:57:45.948538] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99770E-0-0] [lt=2][errcode=-5171] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:45.949608] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:45.949623] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:45.949631] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=150) [2024-10-16 14:57:45.950608] WDIAG [OCCAM] runTimerTask (ob_occam_timer.h:163) [21917][T1001_MultiVers][T1001][Y0-0000000000000000-0-0] [lt=69][errcode=0] task scheduled too delay(scheduled_delay=1115485, expected_run_ts_us_=1729061864835121, ObClockGenerator::getRealClock()=1729061865950606, ret=0, ret="OB_SUCCESS", *this={this:0x2b0df39fd8d0, caller:ob_multi_version_garbage_collector.cpp:start:115, function_type:ZN9oceanbase19concurrency_control30ObMultiVersionGarbageCollector5startEvE5$_938, timer_running_flag:1, total_running_count:1, func_shared_ptr_.ptr:0x2b0df39fdbb0, time_wheel:0x2b0ddbd7b990, thread_pool:0x2b0ddbd7b170, is_running:1, time_interval:60.00s, expected_run_time:14:57:44.835, task_priority:1, with_handle_protected:1}) [2024-10-16 14:57:45.951621] INFO [MVCC] repeat_study (ob_multi_version_garbage_collector.cpp:196) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=9] skip repeat study(ret=0, this={this:0x2b0df71fcf30, last_study_timestamp:1729061625888050, last_refresh_timestamp:1729061805931070, last_reclaim_timestamp:1729061565862103, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061389718673000, v:0}, is_inited:true}, current_timestamp=1729061865951620, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:45.952305] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=15] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=1729061625882867000, snapshot_type=1, create_time=1729061625883884, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:45.952321] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=12] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=1729061389927859000, snapshot_type=1, create_time=1729061389930233, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:45.952328] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=1729061492820106000, snapshot_type=1, create_time=1729061492799805, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:45.952334] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=1729061625603829000, snapshot_type=2, create_time=1729061625883884, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:45.952340] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=1729061389718673000, snapshot_type=2, create_time=1729061389930233, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:45.952345] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=1729061492231899000, snapshot_type=2, create_time=1729061492799805, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:45.952350] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=1729061625869346000, snapshot_type=3, create_time=1729061625883884, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:45.952356] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=1729061389918361000, snapshot_type=3, create_time=1729061389930233, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:45.952361] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=1729061492751527000, snapshot_type=3, create_time=1729061492799805, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:45.952367] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061625883884, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:45.952372] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061389930233, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:45.952377] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1001 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061492799805, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:45.952487] INFO [MVCC] decide_reserved_snapshot_version_ (ob_multi_version_garbage_collector.cpp:618) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=3] succeed to update global reserved snapshot(ret=0, this={this:0x2b0df71fcf30, last_study_timestamp:1729061625888050, last_refresh_timestamp:1729061805931070, last_reclaim_timestamp:1729061565862103, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061389718673000, v:0}, is_inited:true}, global_reserved_snapshot_={val:1729061389718673000, v:0}, reserved_snapshot={val:1729061389718673000, v:0}) [2024-10-16 14:57:45.952499] INFO [MVCC] refresh_ (ob_multi_version_garbage_collector.cpp:552) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=11] multi-version garbage collector refresh successfully(this={this:0x2b0df71fcf30, last_study_timestamp:1729061625888050, last_refresh_timestamp:1729061805931070, last_reclaim_timestamp:1729061565862103, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061389718673000, v:0}, is_inited:true}, collector={reserved_snapshot_version:{val:1729061389718673000, v:0}, reserved_snapshot_type:2, reserved_status:-1, reserved_create_time:1729061389930233, reserved_addr:"10.0.4.8:2890", is_this_server_disabled:false, status:0}) [2024-10-16 14:57:45.952508] INFO [MVCC] repeat_refresh (ob_multi_version_garbage_collector.cpp:231) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=8] repeat refresh successfully(ret=0, this={this:0x2b0df71fcf30, last_study_timestamp:1729061625888050, last_refresh_timestamp:1729061865952508, last_reclaim_timestamp:1729061565862103, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061389718673000, v:0}, is_inited:true}, current_timestamp=1729061865951636, GARBAGE_COLLECT_RETRY_INTERVAL=60000000) [2024-10-16 14:57:45.952514] INFO [MVCC] repeat_reclaim (ob_multi_version_garbage_collector.cpp:266) [21916][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=6] skip repeat reclaim(ret=0, this={this:0x2b0df71fcf30, last_study_timestamp:1729061625888050, last_refresh_timestamp:1729061865952508, last_reclaim_timestamp:1729061565862103, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061389718673000, v:0}, is_inited:true}, current_timestamp=1729061865952514, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:45.966692] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77C5-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938311, log_task={header:{begin_lsn:{lsn:213058243510}, end_lsn:{lsn:213058243632}, log_id:220938311, min_scn:{val:1729061865646963003, v:0}, max_scn:{val:1729061865646963003, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058243388}, prev_proposal_id:88, committed_end_lsn:{lsn:213058243510}, data_checksum:1715074492, accum_checksum:642070132, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061865866685, freeze_ts:1729061865866685, submit_ts:1729061865866689, flushed_ts:1729061865869618, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:2929}, fs_cb_begin_ts=1729061865966686, log_life_time=100001) [2024-10-16 14:57:45.966944] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=609328, this={dio_aligned_buf:{buf_write_offset:3242, buf_padding_size:854, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111160263, truncate_used_ts:41410951}, log_block_size:67108864, dir_fd:177, io_fd:142}, aligned_buf_len=4096, aligned_block_offset=606208, buf_len=122, total_write_size_=13486, total_write_size_after_dio_=425984, ob_pwrite_used_ts=35786, count=101, each_pwrite_cost=354) [2024-10-16 14:57:45.966974] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=28] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:45.969914] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:45.986524] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=273, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:45.986539] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=194, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:45.986745] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD291-0-0] [lt=21] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=848, avg_log_submit_to_slide_time=1426) [2024-10-16 14:57:45.991485] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061865989664000, start_id=1729061865989664000, end_id=1729061865989664001) [2024-10-16 14:57:45.991877] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FF7-0-0] [lt=24] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:45.991896] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FF7-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497458, log_task={header:{begin_lsn:{lsn:207589552562}, end_lsn:{lsn:207589552684}, log_id:218497458, min_scn:{val:1729061865646963002, v:0}, max_scn:{val:1729061865646963002, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589552440}, prev_proposal_id:81, committed_end_lsn:{lsn:207589552562}, data_checksum:940009244, accum_checksum:3109385912, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061865891648, freeze_ts:1729061865891648, submit_ts:1729061865891652, flushed_ts:1729061865893880, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:2228}, fs_cb_begin_ts=1729061865991891, log_life_time=100243) [2024-10-16 14:57:45.994856] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15DC-0-0] [lt=12] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:46.003903] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=23] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1002, min_version={val:1729061865550656001, v:0}, base_version={val:1729061865145959003, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061865550656003, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061866003889, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061865550656001, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061865873429, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061865550656002, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061865900712, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:46.004781] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061866003398000, start_id=1729061866003398000, end_id=1729061866003398001) [2024-10-16 14:57:46.005744] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.97, rt: 142.75; ) [2024-10-16 14:57:46.005762] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:46.005807] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.97, rt: 178.00; ) [2024-10-16 14:57:46.005813] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:46.005822] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:46.008781] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061866008780, bytes: 4446815806221, bw: 0.359967 MB/s, add_ts: 1005102, add_bytes: 379378 [2024-10-16 14:57:46.010592] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:46.013445] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=1454, avg_group_log_size=121) [2024-10-16 14:57:46.013477] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1454) [2024-10-16 14:57:46.013500] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:46.013509] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:46.013934] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=230, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:46.013949] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=196, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:46.013965] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:46.014083] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4367-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=447, avg_log_submit_to_slide_time=633) [2024-10-16 14:57:46.026889] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866025868000, start_id=1729061866025868000, end_id=1729061866025868001) [2024-10-16 14:57:46.028349] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028368] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=14][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028377] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028385] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028393] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028400] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028408] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028415] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028423] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028431] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028438] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028483] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028490] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028498] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028561] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.028628] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997754-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.033704] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.037636] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775B-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.037655] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775B-0-0] [lt=13][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.037664] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775B-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.037672] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775B-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.037689] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775B-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.037709] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775B-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.039557] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775C-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.039583] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775C-0-0] [lt=14][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.039641] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775C-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.039709] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99775C-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051581] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=1][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051613] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=16][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051634] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051646] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051663] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051675] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051711] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051725] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051740] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051754] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051833] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051842] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051850] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.051857] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997765-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.052266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061866050909000, start_id=1729061866050909000, end_id=1729061866050909001) [2024-10-16 14:57:46.052285] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:46.052299] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061866052296], gts:1729061866050909000, latest_srr:[mts=1729061866052296]}) [2024-10-16 14:57:46.052663] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E28-0-0] [lt=22][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173193, log_task={header:{begin_lsn:{lsn:14352379712}, end_lsn:{lsn:14352379834}, log_id:115173193, min_scn:{val:1729061865645903003, v:0}, max_scn:{val:1729061865645903003, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352379590}, prev_proposal_id:17, committed_end_lsn:{lsn:14352379712}, data_checksum:940009244, accum_checksum:185731509, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061865950856, freeze_ts:1729061865950857, submit_ts:1729061865950861, flushed_ts:1729061865952742, gen_to_freeze cost time:1, gen_to_submit cost time:5, submit_to_flush cost time:1881}, fs_cb_begin_ts=1729061866052656, log_life_time=101800) [2024-10-16 14:57:46.059216] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059241] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=19][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059258] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059270] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059282] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059292] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059305] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059365] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059376] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059387] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.059402] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99776B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.064484] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997770-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.064508] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997770-0-0] [lt=17][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.064517] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997770-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.064524] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997770-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.064532] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997770-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.064543] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997770-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.064554] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997770-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.069979] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.086260] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:46.086277] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:46.089005] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997789-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.089073] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997789-0-0] [lt=38][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"PACKAGE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.089332] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997789-0-0] [lt=19][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.089378] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997789-0-0] [lt=22][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"ROUTINE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.089838] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997789-0-0] [lt=17][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.089889] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997789-0-0] [lt=24][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TRIGGER_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.090876] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:46.091070] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D46-0-0] [lt=10] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061865550656004, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061863049052001, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880721265000, latest_log_ts:{val:1729061864144741001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:46.091280] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D46-0-0] [lt=99] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061865550656004, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061863049052001, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:46.091434] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D46-0-0] [lt=78] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061865646963003, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061863049052001, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:46.091612] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D46-0-0] [lt=90] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061865645903003, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061863049052001, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:46.091707] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D46-0-0] [lt=75] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:46.092783] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FF9-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:46.092990] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=186, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:46.093005] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=429, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:46.093055] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:46.095824] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061866095823, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007619, add_bytes: 0 [2024-10-16 14:57:46.105750] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99778D-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"A", column:"DATA_TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.105822] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99778D-0-0] [lt=35][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"A", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.105854] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99778D-0-0] [lt=15][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"A", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.105893] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99778D-0-0] [lt=18][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"A", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.106148] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99778D-0-0] [lt=19][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"A", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.116975] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:46.116992] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:46.117890] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.117923] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=13][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.117960] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.118025] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.118042] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.118069] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.118081] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.118131] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.118148] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.118250] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997791-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.123502] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:46.125575] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866124552000, start_id=1729061866124552000, end_id=1729061866124552001) [2024-10-16 14:57:46.125751] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997795-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"DATA_TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.125818] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997795-0-0] [lt=40][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.125993] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997795-0-0] [lt=15][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.126034] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997795-0-0] [lt=18][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.126988] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=13] current monitor number(seq_=-1) [2024-10-16 14:57:46.130781] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [21543][T1_CdcSrv][T1][Y0-0000000000000000-0-0] [lt=12] finish to resize log external storage handler(current_ts=1729061866130778, tenant_max_cpu=3, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:46.131549] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997797-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.131576] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997797-0-0] [lt=11][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.131673] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997797-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.131691] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997797-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.131716] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997797-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.131728] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997797-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.131770] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997797-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.131787] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997797-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.131875] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997797-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.134142] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866131876000, start_id=1729061866131876000, end_id=1729061866131876001) [2024-10-16 14:57:46.134234] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=2754, mem_used=6239232) [2024-10-16 14:57:46.134262] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.139684] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997799-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.139744] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997799-0-0] [lt=34][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"PACKAGE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.140018] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997799-0-0] [lt=15][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.140058] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997799-0-0] [lt=18][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"ROUTINE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.140538] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997799-0-0] [lt=14][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.140584] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997799-0-0] [lt=22][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TRIGGER_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.144407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061866139348000, start_id=1729061866139348000, end_id=1729061866139348001) [2024-10-16 14:57:46.144474] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:46.144489] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:46.145026] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=248, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:46.145039] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=95, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:46.145054] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9681) [2024-10-16 14:57:46.145062] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:46.145654] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:46.146042] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F951C-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:46.146053] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F951C-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=832, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:46.146057] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F951C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=380, avg_log_submit_to_slide_time=800) [2024-10-16 14:57:46.148220] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B9F8-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061866147856000, start_id=1729061866147856000, end_id=1729061866147856001) [2024-10-16 14:57:46.148247] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0C85-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866147856000, start_id=1729061866147856000, end_id=1729061866147856001) [2024-10-16 14:57:46.148418] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9F7-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061866147856000, start_id=1729061866147856000, end_id=1729061866147856001) [2024-10-16 14:57:46.148437] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0C84-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061866147856000, start_id=1729061866147856000, end_id=1729061866147856001) [2024-10-16 14:57:46.148455] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969B9FB-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061866147856000, start_id=1729061866147856000, end_id=1729061866147856001) [2024-10-16 14:57:46.152868] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.152897] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=11][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.152931] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.152988] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.153005] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.153030] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.153042] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.153086] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.153104] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.153194] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99779E-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.156820] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A0-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"DATA_TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.157030] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A0-0-0] [lt=39][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.161989] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A2-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.162023] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A2-0-0] [lt=15][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.162145] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A2-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.162171] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A2-0-0] [lt=11][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.162199] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A2-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.162214] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A2-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.162260] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A2-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.162280] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A2-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.162369] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977A2-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.166771] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77C9-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1333, avg_group_log_size=121) [2024-10-16 14:57:46.170038] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.170112] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:46.170127] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=8] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:46.170134] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:46.170137] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:46.170140] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:46.170143] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16726) [2024-10-16 14:57:46.173974] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC84-0-0] [lt=10] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:46.173991] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC84-0-0] [lt=12] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:46.174007] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC84-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:46.185047] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA884-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:46.185070] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA884-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061865646963003, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061865646963003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:46.186588] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:46.187495] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=12] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:46.190218] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5030-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:46.190232] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5030-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:46.191370] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061866189607000, start_id=1729061866189607000, end_id=1729061866189607001) [2024-10-16 14:57:46.191679] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=278, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:46.191695] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=31, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:46.191712] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:46.191824] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE84FFB-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:46.192126] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C104F-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=339, avg_log_submit_to_slide_time=618) [2024-10-16 14:57:46.192255] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA884-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061866191503000, start_id=1729061866191503000, end_id=1729061866191503001) [2024-10-16 14:57:46.192312] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA884-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9681, avg_group_log_size=569) [2024-10-16 14:57:46.192321] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA884-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9681) [2024-10-16 14:57:46.192555] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9681) [2024-10-16 14:57:46.192903] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:46.204756] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061866203273000, start_id=1729061866203273000, end_id=1729061866203273001) [2024-10-16 14:57:46.211361] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:46.211547] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] [CACHE] tenant_id= 1 cache memory info: [CACHE] tenant_id= 1 | cache_name= schema_cache | cache_size= 91552768 | cache_store_size= 89473024 | cache_map_size= 2079744 | kv_cnt= 5669 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= tablet_table_cache | cache_size= 12481536 | cache_store_size= 6242304 | cache_map_size= 6239232 | kv_cnt= 43492 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= vtable_cache | cache_size= 147733504 | cache_store_size= 145653760 | cache_map_size= 2079744 | kv_cnt= 76 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= index_block_cache | cache_size= 68662272 | cache_store_size= 62423040 | cache_map_size= 6239232 | kv_cnt= 31222 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= user_block_cache | cache_size= 222639104 | cache_store_size= 216399872 | cache_map_size= 6239232 | kv_cnt= 32779 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= user_row_cache | cache_size= 12483584 | cache_store_size= 10403840 | cache_map_size= 2079744 | kv_cnt= 2891 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= bf_cache | cache_size= 10402816 | cache_store_size= 8323072 | cache_map_size= 2079744 | kv_cnt= 193 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= fuse_row_cache | cache_size= 14564352 | cache_store_size= 12484608 | cache_map_size= 2079744 | kv_cnt= 4381 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= storage_meta_cache | cache_size= 285058048 | cache_store_size= 270499840 | cache_map_size= 14558208 | kv_cnt= 80863 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 17 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 25 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 234 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_column_stat_cache | cache_size= 16645120 | cache_store_size= 14565376 | cache_map_size= 2079744 | kv_cnt= 1027 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_ds_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 9 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= BACKUP_INDEX_CACHE | cache_size= 0 | cache_store_size= 0 | cache_map_size= 0 | kv_cnt= 0 | hold_size= 0 [2024-10-16 14:57:46.211587] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] [CACHE] tenant_id= 1001 cache memory info: [CACHE] tenant_id= 1001 | cache_name= index_block_cache | cache_size= 22887424 | cache_store_size= 20807680 | cache_map_size= 2079744 | kv_cnt= 5229 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= user_block_cache | cache_size= 187268096 | cache_store_size= 185188352 | cache_map_size= 2079744 | kv_cnt= 10159 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= user_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1229 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= bf_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 28 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= fuse_row_cache | cache_size= 10402816 | cache_store_size= 8323072 | cache_map_size= 2079744 | kv_cnt= 2520 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= storage_meta_cache | cache_size= 41614336 | cache_store_size= 39534592 | cache_map_size= 2079744 | kv_cnt= 10145 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 39 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 6 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 133 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= opt_column_stat_cache | cache_size= 8322048 | cache_store_size= 6242304 | cache_map_size= 2079744 | kv_cnt= 362 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [2024-10-16 14:57:46.211626] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=5] [CACHE] tenant_id= 1002 cache memory info: [CACHE] tenant_id= 1002 | cache_name= index_block_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 441 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= user_block_cache | cache_size= 37452800 | cache_store_size= 35373056 | cache_map_size= 2079744 | kv_cnt= 882 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= user_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1406 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= bf_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 19 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= fuse_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 2759 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= storage_meta_cache | cache_size= 16645120 | cache_store_size= 14565376 | cache_map_size= 2079744 | kv_cnt= 2728 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= tmp_block_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 3691 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 24 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1078 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= opt_column_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 624 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [2024-10-16 14:57:46.211661] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=5] [CACHE] tenant_id= 1003 cache memory info: [CACHE] tenant_id= 1003 | cache_name= index_block_cache | cache_size= 24968192 | cache_store_size= 22888448 | cache_map_size= 2079744 | kv_cnt= 5100 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= user_block_cache | cache_size= 164379648 | cache_store_size= 162299904 | cache_map_size= 2079744 | kv_cnt= 11493 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= user_row_cache | cache_size= 8322048 | cache_store_size= 6242304 | cache_map_size= 2079744 | kv_cnt= 5065 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= bf_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 49 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= fuse_row_cache | cache_size= 8322048 | cache_store_size= 6242304 | cache_map_size= 2079744 | kv_cnt= 2220 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= storage_meta_cache | cache_size= 49937408 | cache_store_size= 47857664 | cache_map_size= 2079744 | kv_cnt= 10774 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= tmp_block_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 31 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 24 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 134 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= opt_column_stat_cache | cache_size= 12483584 | cache_store_size= 10403840 | cache_map_size= 2079744 | kv_cnt= 764 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [2024-10-16 14:57:46.211695] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=5] [CACHE] tenant_id= 1004 cache memory info: [CACHE] tenant_id= 1004 | cache_name= index_block_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 747 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= user_block_cache | cache_size= 14564352 | cache_store_size= 12484608 | cache_map_size= 2079744 | kv_cnt= 388 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= user_row_cache | cache_size= 10402816 | cache_store_size= 8323072 | cache_map_size= 2079744 | kv_cnt= 573 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= fuse_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1098 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= storage_meta_cache | cache_size= 24968192 | cache_store_size= 22888448 | cache_map_size= 2079744 | kv_cnt= 5924 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= tmp_block_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 6 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 27 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 161 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_column_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 461 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_ds_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 9 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= BACKUP_INDEX_CACHE | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 3 | hold_size= 0 [2024-10-16 14:57:46.211708] INFO print_current_status (ob_kvcache_hazard_version.cpp:328) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=6] [KVCACHE-HAZARD] hazard version status info: current_version: 207913368 | min_version=207913368 | total_nodes_count: 41 | [KVCACHE-HAZARD] i= 25 | acquire_version=18446744073709551615 | waiting_nodes_count= 34 | last_retire_version=207913327 | [KVCACHE-HAZARD] i= 79 | acquire_version=18446744073709551615 | waiting_nodes_count= 2 | last_retire_version=207864171 | [KVCACHE-HAZARD] i= 108 | acquire_version=18446744073709551615 | waiting_nodes_count= 1 | last_retire_version=207913327 | [KVCACHE-HAZARD] i= 132 | acquire_version=18446744073709551615 | waiting_nodes_count= 4 | last_retire_version=207913159 | [2024-10-16 14:57:46.212860] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587B-0-0] [lt=11] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:46.212948] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587B-0-0] [lt=11][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:46.215756] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1210) [2024-10-16 14:57:46.218180] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601422, offset=9631212}, end_cursor:ObLogCursor{file_id=192, log_id=47601423, offset=9631496}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7272050, len:2580, log_data_len:284, seq:47601422, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:46.219913] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A8-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061866218366000, start_id=1729061866218366000, end_id=1729061866218366001) [2024-10-16 14:57:46.223856] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D92-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510334092250, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410605, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.224221] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601423, offset=9631496}, end_cursor:ObLogCursor{file_id=192, log_id=47601424, offset=9631770}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cac050, len:2296, log_data_len:274, seq:47601423, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:46.231519] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C107C-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866230686000, start_id=1729061866230686000, end_id=1729061866230686001) [2024-10-16 14:57:46.232577] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601424, offset=9631770}, end_cursor:ObLogCursor{file_id=192, log_id=47601425, offset=9632044}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c78050, len:2022, log_data_len:274, seq:47601424, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:46.232878] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601425, offset=9632044}, end_cursor:ObLogCursor{file_id=192, log_id=47601426, offset=9632318}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72aa050, len:1748, log_data_len:274, seq:47601425, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:46.232970] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561322, miss_cnt:1, cache_read_size:152590683980, hit_ratio:"9.999999928346909872e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:46.232993] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598909, miss_cnt:9, cache_read_size:149777917491, hit_ratio:"9.999999345925089100e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:46.233371] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:57:46.233382] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=9] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=85, cache_node_num=85) [2024-10-16 14:57:46.233396] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=12] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=85, cache_node_num=85) [2024-10-16 14:57:46.233401] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:46.235502] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=85, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}, {obj_id:78893917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:78893918, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893919, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}]) [2024-10-16 14:57:46.235616] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=112] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:46.235657] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] no need to update min/max version(new_version={val:1729061865931617000, v:0}, min_version={val:1729061865931617000, v:0}, max_version={val:1729061865931617000, v:0}) [2024-10-16 14:57:46.235776] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.241723] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] worker thread began to shrink(tenant_->id()=1003, group_id_=1, token=0) [2024-10-16 14:57:46.245216] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:57:46.245574] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245592] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=11][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245610] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245618] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245626] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245634] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245641] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245649] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245656] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245664] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245715] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245723] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245731] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245792] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245850] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977E6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.245974] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061866244914000, start_id=1729061866244914000, end_id=1729061866244914001) [2024-10-16 14:57:46.247008] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257770, offset=39035289}, end_cursor:ObLogCursor{file_id=197, log_id=49257771, offset=39035572}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cf8050, len:3687, log_data_len:283, seq:49257770, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:46.247414] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107492][T1003_L0_G1][T1003][Y0-0000000000000000-0-0] [lt=10] worker thread exit(tenant_->id()=1003, workers_.get_size()=2) [2024-10-16 14:57:46.247428] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107492][T1003_L0_G1][T0][Y0-0000000000000000-0-0] [lt=9] tenant thread end_run(id_=1003, ret=0, thread_count_=172) [2024-10-16 14:57:46.250587] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB373-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061866248770000, start_id=1729061866248770000, end_id=1729061866248770001) [2024-10-16 14:57:46.251616] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21682][T1_L0_G0][T1][YB4A0A000409-00061E9A24F6AD0C-0-0] [lt=2][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:46.254076] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=383, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:46.254095] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=548, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:46.254129] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:46.256209] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256228] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=13][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256237] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256245] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256253] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256261] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256274] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256285] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256293] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256300] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256308] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256316] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256323] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256331] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256388] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256452] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256507] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256562] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256574] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EA-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.256627] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257771, offset=39035572}, end_cursor:ObLogCursor{file_id=197, log_id=49257772, offset=39035854}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c64050, len:3404, log_data_len:282, seq:49257771, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:46.261634] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257772, offset=39035854}, end_cursor:ObLogCursor{file_id=197, log_id=49257773, offset=39036136}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c94050, len:3122, log_data_len:282, seq:49257772, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:46.265827] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257773, offset=39036136}, end_cursor:ObLogCursor{file_id=197, log_id=49257774, offset=39036418}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7316050, len:2840, log_data_len:282, seq:49257773, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:46.265905] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650258, miss_cnt:32, cache_read_size:8050422796, hit_ratio:"9.999994892282222381e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:46.265929] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62656997, miss_cnt:17, cache_read_size:8050170299, hit_ratio:"9.999997286816125763e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:46.266735] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77CB-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:46.267166] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5404-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:46.267185] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5404-0-0] [lt=19] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30380, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:46.268224] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268249] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=18][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268264] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268280] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268291] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268305] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268320] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268342] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268357] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268367] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268378] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268388] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268399] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268409] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268420] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268485] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268555] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268628] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268690] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.268701] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B9977EF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.270263] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.276986] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CAB-0-0] [lt=22] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:46.277006] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CAB-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:46.277012] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CAB-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=54180, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:46.277016] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CAB-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=462, avg_log_submit_to_slide_time=54173) [2024-10-16 14:57:46.277084] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1098, hit_cnt=8, read_cnt=8, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:46.277301] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=235, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:46.277315] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=193, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:46.277345] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:46.286345] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B3769AB9A-0-0] [lt=11][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:46.286932] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:46.286950] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:46.288734] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=8] ls blacklist refresh finish(cost_time=10646) [2024-10-16 14:57:46.289319] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997803-0-0] [lt=1][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.289343] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997803-0-0] [lt=18][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294576] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294597] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=15][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294616] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=13][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294648] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294656] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294679] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294687] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294695] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294703] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294710] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294718] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294726] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294738] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294746] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294754] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.294762] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997806-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.296780] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21656][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3DF6-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510334165175, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410606, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298022] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298038] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=10][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298047] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298055] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298081] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298089] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298113] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298127] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=9][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298135] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298143] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298151] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298159] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298166] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298178] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298186] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298194] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.298202] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997807-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.307867] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:46.308625] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:46.308638] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:57:46.308652] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:46.312063] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:46.312076] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:46.312090] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=373635715, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:46.312159] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:46.312230] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAB-0-0] [lt=5] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061879029444000, latest_log_ts:{val:1729061862417342001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:46.312281] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAB-0-0] [lt=47] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:46.313164] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997814-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.313208] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997814-0-0] [lt=18][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.313241] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997814-0-0] [lt=9][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.313257] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997814-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.313270] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997814-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.313287] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997814-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.313307] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997814-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.313326] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997814-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.313347] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997814-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.315616] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:46.315642] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=22] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:46.315652] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:46.319657] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7404, mem_used=14558208) [2024-10-16 14:57:46.324975] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2201990, replace_num=31457) [2024-10-16 14:57:46.324989] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:46.326488] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866325230000, start_id=1729061866325230000, end_id=1729061866325230001) [2024-10-16 14:57:46.328522] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99781C-0-0] [lt=0][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.328560] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99781C-0-0] [lt=15][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.328569] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99781C-0-0] [lt=4][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.328575] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99781C-0-0] [lt=4][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.328592] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99781C-0-0] [lt=3][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.328629] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99781C-0-0] [lt=3][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.328637] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99781C-0-0] [lt=3][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.328643] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99781C-0-0] [lt=3][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.329792] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:46.335715] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1003, min_version={val:1729061865931617000, v:0}, base_version={val:1729061865931617000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061866131876000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061866335704, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061865931617000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061866165124, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061866131876000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061866316529, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:46.335739] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] no need to update min/max version(new_version={val:1729061865931617000, v:0}, min_version={val:1729061865931617000, v:0}, max_version={val:1729061865931617000, v:0}) [2024-10-16 14:57:46.335765] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=5] [WRS] update tenant weak read server version(tenant_id=1003, server_version={version:{val:1729061866131876000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061866335744}, version_delta=203889) [2024-10-16 14:57:46.335844] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.341353] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=57] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3536, avg_group_log_size=353) [2024-10-16 14:57:46.343365] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=17] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:46.344572] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061866343189000, start_id=1729061866343189000, end_id=1729061866343189001) [2024-10-16 14:57:46.345319] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997823-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.345436] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1001, new_min_version={val:1729061866139348000, v:0}, new_max_version={val:1729061866139348000, v:0}, last_min_version={val:1729061865938879000, v:0}, last_max_version={val:1729061865938879000, v:0}, total_time_us=1591, delta_us=206087) [2024-10-16 14:57:46.345450] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1001, current_version={val:1729061866139348000, v:0}, min_version={val:1729061866139348000, v:0}, max_version={val:1729061866139348000, v:0}, in_service=true, leader_epoch=33705) [2024-10-16 14:57:46.345454] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997823-0-0] [lt=31][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.345552] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997823-0-0] [lt=16][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.351131] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E2E-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:46.351143] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E2E-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=41305, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:46.357369] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E06-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061866356207000, start_id=1729061866356207000, end_id=1729061866356207001) [2024-10-16 14:57:46.367294] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:46.367331] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=5] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:46.367338] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:46.370290] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.377049] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CAD-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:46.386564] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:46.386580] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1342) [2024-10-16 14:57:46.386619] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:46.386624] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1342) [2024-10-16 14:57:46.386903] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD00E-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061866377880, keepalive_ts:-1}) [2024-10-16 14:57:46.386915] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD00E-0-0] [lt=12] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061866377880, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:46.386955] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:46.391578] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061866389885000, start_id=1729061866389885000, end_id=1729061866389885001) [2024-10-16 14:57:46.391642] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:46.391654] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:46.392312] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B99784C-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"a", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.393012] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:46.393061] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=17] LogLoopThread round_cost_time(us)(round_cost_time=8) [2024-10-16 14:57:46.394734] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:46.407295] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866405976000, start_id=1729061866405976000, end_id=1729061866405976001) [2024-10-16 14:57:46.408419] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get_max_applied_scn(scn={val:1729061866050909001, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959704304}, palf_committed_end_scn_:{val:1729061866230686000, v:0}, last_check_scn_:{val:1729061866230686000, v:0}, max_applied_cb_scn_:{val:1729061866050909001, v:0}}) [2024-10-16 14:57:46.408440] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:46.408488] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=40] get_max_applied_scn(scn={val:1729061866050909001, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023106300}, palf_committed_end_scn_:{val:1729061866230686000, v:0}, last_check_scn_:{val:1729061866230686000, v:0}, max_applied_cb_scn_:{val:1729061866050909001, v:0}}) [2024-10-16 14:57:46.408505] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:46.408536] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=23] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:57:46.408543] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] get_min_unreplayed_log_info(lsn={lsn:213058244242}, scn={val:1729061866147856003, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061866367267, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058244242}, next_to_submit_scn_:{val:1729061866147856003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058244120}, end_lsn:{lsn:213058244242}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061866147856002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061866147856002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:55818712, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061866147856002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561324, miss_cnt:1, cache_read_size:152590684224, hit_ratio:"9.999999928346910982e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:46.411980] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=4] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:46.415518] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:46.415531] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:57:46.415852] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:46.432148] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=6] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:57:46.432161] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:57:46.432165] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:46.432878] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061879029444000, latest_log_ts:{val:1729061862417342001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:46.436551] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061866435635000, start_id=1729061866435635000, end_id=1729061866435635001) [2024-10-16 14:57:46.436674] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.445194] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=9] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:133533322, max_lsn:66390142025, max_scn:{val:1729061866389885000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:33216076378}, reuse_lsn:{lsn:66390142025}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:66348198985}}, last_submit_lsn:{lsn:66390141438}, last_submit_end_lsn:{lsn:66390142025}, last_submit_log_id:133533322, last_submit_log_pid:48, max_flushed_lsn:{lsn:66390141438}, max_flushed_end_lsn:{lsn:66390142025}, max_flushed_log_pid:48, committed_end_lsn:{lsn:66390142025}, last_slide_log_id:133533322, last_slide_scn:{val:1729061866389885000, v:0}, last_slide_lsn:{lsn:66390141438}, last_slide_end_lsn:{lsn:66390142025}, last_slide_log_pid:48, last_slide_log_accum_checksum:1995012718, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0de0acc530}, [StateMgr]={this:0x2b0de0ad0830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:133533323, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261740374, palf_role_change_cb:0x2b0de0ad7230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, reconfig_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, persistent_config_version:{proposal_id:48, config_seq:11}, ms_ack_list:0, resend_config_version:{proposal_id:48, config_seq:11}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494260838312, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494260838312, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0de0acdc70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261941355, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728741813401825001, v:0}, min_block_id:965, min_block_min_scn:{val:1728727703672931001, v:0}, base_lsn_for_block_gc:{lsn:66098196480}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, log_config_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, log_snapshot_meta:{version:1, base_lsn:{lsn:66098196480}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1634304}, readable_log_tail:{lsn:1634304}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66433720320}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/meta", dir_fd:128, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65470464, block_header_serialize_buf_:0x2b0de0ad29f8, flashback_version:0}, log_storage:{log_tail:{lsn:66390142025}, readable_log_tail:{lsn:66390142025}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66366615552}, min_scn:{val:1729057321384354001, v:0}, curr_block_id:989, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/log", dir_fd:134, min_block_id:965, max_block_id:990, curr_writable_block_id:989}, logical_block_size_:67104768, curr_block_writable_size_:43578295, block_header_serialize_buf_:0x2b0de0ad46b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261739251, this:0x2b0de0ad0f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:48, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:47, majority_max_lsn:{lsn:33216819320}, saved_end_lsn:{lsn:33216819320}, last_submit_prepare_req_time_us:1722494261740552, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0de0ad09b0}) [2024-10-16 14:57:46.445333] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=136] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061866391902, last_flushed_end_lsn:{lsn:66390142025}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061866392702, last_flushed_end_lsn:{lsn:66390142025}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061866392920, last_flushed_end_lsn:{lsn:66390142025}}]) [2024-10-16 14:57:46.445344] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=133533323, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061866391902, last_flushed_end_lsn:{lsn:66390142025}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061866392702, last_flushed_end_lsn:{lsn:66390142025}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061866392920, last_flushed_end_lsn:{lsn:66390142025}}]) [2024-10-16 14:57:46.451504] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9183-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=954, avg_log_submit_to_slide_time=37436) [2024-10-16 14:57:46.451540] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:46.466455] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997890-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.466523] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997890-0-0] [lt=41][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.466577] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997890-0-0] [lt=18][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.466618] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997890-0-0] [lt=16][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.467068] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=194, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:46.467091] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=252, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:46.468170] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997891-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.468229] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997891-0-0] [lt=34][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.468284] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997891-0-0] [lt=18][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.468317] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997891-0-0] [lt=16][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.470320] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.475385] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A84-0-0] [lt=11] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:46.475400] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A84-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:46.476593] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:57:46.476634] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=39] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:46.476640] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:46.476646] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:46.478260] WDIAG [SQL.RESV] check_table_column_namespace (ob_column_namespace_checker.cpp:124) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997898-0-0] [lt=0][errcode=-5207] column in all tables is ambiguous(q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:46.478293] WDIAG [SQL.RESV] resolve_table_column_expr (ob_dml_resolver.cpp:4587) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997898-0-0] [lt=33][errcode=-5207] column not found in table(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:46.478320] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997898-0-0] [lt=17][errcode=-5207] resolve table column expr failed(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6c2c860 0x688c2c2 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.478783] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:57:46.478915] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=131] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:46.479826] WDIAG [SQL.RESV] check_table_column_namespace (ob_column_namespace_checker.cpp:124) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997899-0-0] [lt=0][errcode=-5207] column in all tables is ambiguous(q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:46.479861] WDIAG [SQL.RESV] resolve_table_column_expr (ob_dml_resolver.cpp:4587) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997899-0-0] [lt=35][errcode=-5207] column not found in table(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:46.479890] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1][YB4A0A000407-00061E9B3B997899-0-0] [lt=20][errcode=-5207] resolve table column expr failed(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6c2c860 0x688c2c2 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.486994] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD00F-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061866477923, keepalive_ts:-1}) [2024-10-16 14:57:46.487026] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD00F-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061866477923, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:46.492466] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5409-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:46.492478] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5409-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27684, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:46.515687] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:46.515706] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:46.517679] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4194-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1454) [2024-10-16 14:57:46.517738] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4194-0-0] [lt=27] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=521, avg_log_submit_to_slide_time=1092) [2024-10-16 14:57:46.517955] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4377-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:46.517967] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4377-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1133, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:46.519498] INFO [COMMON] generate_mod_stat_task (memory_dump.cpp:218) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] task info(*task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:46.519517] INFO [COMMON] handle (memory_dump.cpp:550) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=18] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:46.520552] INFO [COMMON] update_check_range (ob_sql_mem_leak_checker.cpp:62) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] update_check_range(min_check_version=652899, max_check_version=652899, global_version=652901) [2024-10-16 14:57:46.532080] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061866530811000, start_id=1729061866530811000, end_id=1729061866530811001) [2024-10-16 14:57:46.532449] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:46.532630] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=600, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:46.532646] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=366, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:46.532662] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=8381) [2024-10-16 14:57:46.532671] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:46.533610] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:46.533626] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:46.533640] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444550632, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:46.541892] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9978BE-0-0] [lt=0] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1001, table_id:10002, add_timestamp:1729061866541884}) [2024-10-16 14:57:46.543130] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107496][][T1003][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1003, ret=0, thread_count_=173) [2024-10-16 14:57:46.543149] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107496][][T1003][Y0-0000000000000000-0-0] [lt=17] Init thread local success [2024-10-16 14:57:46.543174] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107496][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C9C-0-0] [lt=4] init point thread id with(&point=0x55e3435ca880, idx_=1000, point=[thread id=107496, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107496) [2024-10-16 14:57:46.543226] INFO [LIB] stat (utility.h:1165) [107496][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C9C-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:46.543232] INFO [LIB] stat (utility.h:1165) [107496][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C9C-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=2391, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:46.543236] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107496][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797C9C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=5, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=988, avg_log_submit_to_slide_time=2356) [2024-10-16 14:57:46.543353] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] worker thread created(tenant_->id()=1003, group_id_=1, token=1) [2024-10-16 14:57:46.543599] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061866542658000, start_id=1729061866542658000, end_id=1729061866542658001) [2024-10-16 14:57:46.543759] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.545832] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866542658000, start_id=1729061866542658000, end_id=1729061866542658001) [2024-10-16 14:57:46.552625] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061866552331000, start_id=1729061866552331000, end_id=1729061866552331001) [2024-10-16 14:57:46.552802] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061866552796], gts:1729061866552331000, latest_srr:[mts=1729061866552796]}) [2024-10-16 14:57:46.555661] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E44-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1001, table_id=10002, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061866555645, sql_renew_time:1729061866555645, is_mark_fail:false}]) [2024-10-16 14:57:46.555689] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E44-0-0] [lt=26] success to process renew task(task={tenant_id:1001, table_id:10002, add_timestamp:1729061866541884}, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061866555645, sql_renew_time:1729061866555645, is_mark_fail:false}]) [2024-10-16 14:57:46.568620] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.569620] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.569637] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.570151] INFO handle (memory_dump.cpp:677) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=11] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 204 258 0 1 2 117 1 0 1 4 1 1 0 1 5 38 4 0 1 7 1 2 0 1 8 65 1 0 1 12 18 3 0 1 13 6 1 0 1 14 1 2 0 1 15 1 1 0 1 18 26 3 0 1 20 2 2 0 1 22 1 1 0 500 0 139 249 0 500 7 7 4 0 500 8 50 1 0 500 9 2 1 0 500 10 10 2 0 500 16 3 1 0 500 17 10 7 0 500 22 111 63 0 500 23 61 11 0 508 0 4 13 0 508 8 5 1 0 509 0 3 8 0 509 8 5 1 0 1001 0 130 232 0 1001 2 22 1 0 1001 4 1 1 0 1001 5 23 4 0 1001 8 44 1 0 1001 12 10 3 0 1001 13 3 1 0 1001 14 1 2 0 1001 18 14 3 0 1001 20 1 2 0 1002 0 173 258 0 1002 2 21 1 0 1002 4 2 1 0 1002 5 24 4 0 1002 8 50 1 0 1002 12 19 3 0 1002 13 7 1 0 1002 14 1 2 0 1002 16 1 1 0 1002 18 14 3 0 1002 20 7 2 0 1003 0 142 235 0 1003 2 29 1 0 1003 4 1 1 0 1003 5 23 4 0 1003 8 44 1 0 1003 12 10 3 0 1003 13 3 1 0 1003 14 1 2 0 1003 16 1 1 0 1003 18 13 3 0 1003 20 1 2 0 1004 0 290 256 0 1004 2 19 1 0 1004 4 2 1 0 1004 5 34 4 0 1004 8 52 1 0 1004 12 19 3 0 1004 13 10 1 0 1004 14 2 2 0 1004 15 2 1 0 1004 18 25 3 0 1004 20 11 2 0 cost_time: 49642 [2024-10-16 14:57:46.570178] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=11] [MEMORY] tenant: 1, limit: 2,147,483,648 hold: 1,982,304,256 rpc_hold: 0 cache_hold: 861,929,472 cache_used: 861,929,472 cache_item_count: 411 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 539,443,200 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MEMSTORE_CTX_ID hold_bytes= 245,366,784 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TRANS_CTX_MGR_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PLAN_CACHE_CTX_ID hold_bytes= 79,691,776 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 136,314,880 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= META_OBJ_CTX_ID hold_bytes= 37,748,736 limit= 429,496,720 [MEMORY] ctx_id= TX_CALLBACK_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOB_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PS_CACHE_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TX_DATA_TABLE hold_bytes= 54,546,432 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MDS_DATA_ID hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:46.570368] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22697][T1004_TntTransf][T1004][YB4A0A000407-00061E9ABD5C8937-0-0] [lt=12] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:57:46.570397] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= DEFAULT_CTX_ID hold= 539,443,200 used= 426,895,920 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 32 [MEMORY] hold= 56,180,736 used= 56,153,088 count= 27 avg_used= 2,079,744 block_cnt= 27 chunk_cnt= 27 mod=CACHE_MAP_NODE [MEMORY] hold= 41,963,520 used= 41,943,040 count= 1 avg_used= 41,943,040 block_cnt= 1 chunk_cnt= 1 mod=LogGroupBuffer [MEMORY] hold= 30,965,760 used= 27,525,120 count= 420 avg_used= 65,536 block_cnt= 420 chunk_cnt= 50 mod=MinorMergeMgr [MEMORY] hold= 23,511,040 used= 23,433,792 count= 8 avg_used= 2,929,224 block_cnt= 8 chunk_cnt= 7 mod=IoControl [MEMORY] hold= 22,880,256 used= 21,568,040 count= 161 avg_used= 133,962 block_cnt= 161 chunk_cnt= 38 mod=SqlPlanMon [MEMORY] hold= 18,874,368 used= 16,967,424 count= 256 avg_used= 66,279 block_cnt= 256 chunk_cnt= 63 mod=SqlDtlBuf [MEMORY] hold= 17,694,720 used= 16,942,080 count= 240 avg_used= 70,592 block_cnt= 240 chunk_cnt= 30 mod=[T]ObSessionDIB [MEMORY] hold= 16,797,696 used= 16,777,336 count= 1 avg_used= 16,777,336 block_cnt= 1 chunk_cnt= 1 mod=MrgMemPoolChk [MEMORY] hold= 15,034,848 used= 14,886,881 count= 98 avg_used= 151,906 block_cnt= 31 chunk_cnt= 7 mod=OmtTenant [MEMORY] hold= 11,771,904 used= 9,838,592 count= 236 avg_used= 41,688 block_cnt= 236 chunk_cnt= 61 mod=[T]char [MEMORY] hold= 10,911,744 used= 9,699,328 count= 148 avg_used= 65,536 block_cnt= 148 chunk_cnt= 38 mod=MajorMergeMgr [MEMORY] hold= 8,777,728 used= 8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv [MEMORY] hold= 8,409,088 used= 8,388,640 count= 1 avg_used= 8,388,640 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 7,061,504 used= 7,050,176 count= 5 avg_used= 1,410,035 block_cnt= 5 chunk_cnt= 5 mod=MysqlRequesReco [MEMORY] hold= 6,094,848 used= 6,033,408 count= 8 avg_used= 754,176 block_cnt= 8 chunk_cnt= 7 mod=ResourceGroup [MEMORY] hold= 4,964,352 used= 4,959,488 count= 3 avg_used= 1,653,162 block_cnt= 3 chunk_cnt= 3 mod=SqlFltSpanRec [MEMORY] hold= 4,943,872 used= 4,915,456 count= 2 avg_used= 2,457,728 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe [MEMORY] hold= 4,235,264 used= 4,194,304 count= 2 avg_used= 2,097,152 block_cnt= 2 chunk_cnt= 2 mod=SHARE_BLOCK [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 12 mod=CkptDgnMemCU [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMem [MEMORY] hold= 3,772,416 used= 3,751,936 count= 1 avg_used= 3,751,936 block_cnt= 1 chunk_cnt= 1 mod=DiskIteratorSto [MEMORY] hold= 3,723,264 used= 3,698,688 count= 2 avg_used= 1,849,344 block_cnt= 2 chunk_cnt= 2 mod=LogDIOAligned [MEMORY] hold= 3,162,112 used= 3,145,808 count= 2 avg_used= 1,572,904 block_cnt= 2 chunk_cnt= 2 mod=TenantDLMgr [MEMORY] hold= 3,145,728 used= 2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 5 mod=SqlDtlQueue [MEMORY] hold= 2,828,928 used= 2,708,352 count= 604 avg_used= 4,484 block_cnt= 576 chunk_cnt= 72 mod=WorkerMap [MEMORY] hold= 2,777,088 used= 2,760,192 count= 1 avg_used= 2,760,192 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo [MEMORY] hold= 2,415,648 used= 2,387,720 count= 104 avg_used= 22,958 block_cnt= 104 chunk_cnt= 4 mod=TabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=FastFrezCkr [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 2,375,680 used= 2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta [MEMORY] hold= 2,248,704 used= 2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb [MEMORY] hold= 2,198,864 used= 416,700 count= 25,277 avg_used= 16 block_cnt= 270 chunk_cnt= 2 mod=Number [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=LogReplayTask [MEMORY] hold= 1,843,200 used= 1,770,696 count= 9 avg_used= 196,744 block_cnt= 9 chunk_cnt= 2 mod=DagMap [MEMORY] hold= 1,777,664 used= 1,770,696 count= 1 avg_used= 1,770,696 block_cnt= 1 chunk_cnt= 1 mod=TableProc [MEMORY] hold= 1,548,288 used= 1,507,118 count= 5 avg_used= 301,423 block_cnt= 5 chunk_cnt= 1 mod=LDIOSetup [MEMORY] hold= 1,548,288 used= 1,488,096 count= 9 avg_used= 165,344 block_cnt= 9 chunk_cnt= 8 mod=SQLSessionInfo [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=TransTimeWheel [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=XATimeWheel [MEMORY] hold= 1,261,568 used= 681,912 count= 77 avg_used= 8,856 block_cnt= 77 chunk_cnt= 31 mod=[T]ObDSActionAr [MEMORY] hold= 1,196,032 used= 1,182,720 count= 2 avg_used= 591,360 block_cnt= 2 chunk_cnt= 2 mod=ServerObjecPool [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsCache [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsInfo [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=RewriteRuleMap [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPlanCac [MEMORY] hold= 1,024,000 used= 1,015,808 count= 1 avg_used= 1,015,808 block_cnt= 1 chunk_cnt= 1 mod=CommitVersions [MEMORY] hold= 983,488 used= 960,256 count= 121 avg_used= 7,936 block_cnt= 121 chunk_cnt= 9 mod=TltMemtablMgr [MEMORY] hold= 958,464 used= 950,272 count= 1 avg_used= 950,272 block_cnt= 1 chunk_cnt= 1 mod=IOWorkerLQ [MEMORY] hold= 958,464 used= 858,728 count= 13 avg_used= 66,056 block_cnt= 13 chunk_cnt= 13 mod=CommSysVarFac [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=TransService [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=ArcSenderQueue [MEMORY] hold= 802,816 used= 394,352 count= 49 avg_used= 8,048 block_cnt= 49 chunk_cnt= 22 mod=[T]ObPerfEventR [MEMORY] hold= 798,400 used= 779,200 count= 100 avg_used= 7,792 block_cnt= 100 chunk_cnt= 20 mod=CkptDgnMemNode [MEMORY] hold= 798,112 used= 789,624 count= 3 avg_used= 263,208 block_cnt= 2 chunk_cnt= 2 mod=PartitTableTask [MEMORY] hold= 786,688 used= 524,352 count= 33 avg_used= 15,889 block_cnt= 33 chunk_cnt= 4 mod=LogAlloc [MEMORY] hold= 765,440 used= 730,112 count= 184 avg_used= 3,968 block_cnt= 184 chunk_cnt= 44 mod=CkpDgn [MEMORY] hold= 663,552 used= 655,360 count= 1 avg_used= 655,360 block_cnt= 1 chunk_cnt= 1 mod=FetchLog [MEMORY] hold= 663,552 used= 659,200 count= 1 avg_used= 659,200 block_cnt= 1 chunk_cnt= 1 mod=MulLevelQueue [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DupTbLease [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTF [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=ElectTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MultiVersionGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTR [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=FrzTrigger [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MdsT [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DetectorTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=OBJLockGC [MEMORY] hold= 598,016 used= 590,232 count= 1 avg_used= 590,232 block_cnt= 1 chunk_cnt= 1 mod=DagNetIdMap [MEMORY] hold= 458,752 used= 440,928 count= 3 avg_used= 146,976 block_cnt= 3 chunk_cnt= 3 mod=bf_queue [MEMORY] hold= 421,504 used= 404,416 count= 89 avg_used= 4,544 block_cnt= 89 chunk_cnt= 31 mod=[T]ObTraceEvent [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ReplaySrv [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ApplySrv [MEMORY] hold= 406,400 used= 396,800 count= 50 avg_used= 7,936 block_cnt= 50 chunk_cnt= 29 mod=LinearHashMapNo [MEMORY] hold= 399,200 used= 389,600 count= 50 avg_used= 7,792 block_cnt= 50 chunk_cnt= 32 mod=CkptDgnCUNode [MEMORY] hold= 385,024 used= 377,120 count= 1 avg_used= 377,120 block_cnt= 1 chunk_cnt= 1 mod=ClogGe [MEMORY] hold= 338,112 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 44 chunk_cnt= 2 mod=LSMap [MEMORY] hold= 311,296 used= 262,144 count= 256 avg_used= 1,024 block_cnt= 44 chunk_cnt= 1 mod=TabletToLS [MEMORY] hold= 311,104 used= 303,904 count= 6 avg_used= 50,650 block_cnt= 6 chunk_cnt= 2 mod=PoolFreeList [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=DmlStatHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompactMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompStMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 294,912 used= 261,632 count= 4 avg_used= 65,408 block_cnt= 4 chunk_cnt= 3 mod=MemTblObj [MEMORY] hold= 268,224 used= 261,888 count= 33 avg_used= 7,936 block_cnt= 33 chunk_cnt= 18 mod=CtxBlockPl [MEMORY] hold= 260,224 used= 253,952 count= 31 avg_used= 8,192 block_cnt= 31 chunk_cnt= 22 mod=ParamCharsType [MEMORY] hold= 212,992 used= 212,160 count= 1 avg_used= 212,160 block_cnt= 1 chunk_cnt= 1 mod=MetaMemMgr [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 19 mod=QueryAllocator [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 20 mod=SqlSession [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=T3MBucket [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=TenDLBucket [MEMORY] hold= 204,800 used= 196,744 count= 1 avg_used= 196,744 block_cnt= 1 chunk_cnt= 1 mod=DagNetMap [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=ResourMapLock [MEMORY] hold= 163,776 used= 156,741 count= 2 avg_used= 78,370 block_cnt= 2 chunk_cnt= 2 mod=CsSstableReader [MEMORY] hold= 147,456 used= 139,264 count= 1 avg_used= 139,264 block_cnt= 1 chunk_cnt= 1 mod=RFLTaskQueue [MEMORY] hold= 147,456 used= 138,240 count= 2 avg_used= 69,120 block_cnt= 2 chunk_cnt= 1 mod=LDBlockBitMap [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 11 mod=RawPl [MEMORY] hold= 128,576 used= 33,168 count= 471 avg_used= 70 block_cnt= 175 chunk_cnt= 33 mod=Coro [MEMORY] hold= 128,000 used= 124,928 count= 16 avg_used= 7,808 block_cnt= 16 chunk_cnt= 2 mod=TabletIDSetNode [MEMORY] hold= 122,880 used= 110,608 count= 2 avg_used= 55,304 block_cnt= 2 chunk_cnt= 1 mod=TabletIDSetBkt [MEMORY] hold= 114,688 used= 106,496 count= 1 avg_used= 106,496 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlMgr [MEMORY] hold= 114,688 used= 111,096 count= 1 avg_used= 111,096 block_cnt= 1 chunk_cnt= 1 mod=AsyncFreezeTabl [MEMORY] hold= 106,496 used= 98,432 count= 1 avg_used= 98,432 block_cnt= 1 chunk_cnt= 1 mod=tenant_mf_mgr [MEMORY] hold= 106,496 used= 98,528 count= 1 avg_used= 98,528 block_cnt= 1 chunk_cnt= 1 mod=LSSvr [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=PlanVaIdx [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 10 mod=SqlSessiVarMap [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=DictPl [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=RlePl [MEMORY] hold= 81,920 used= 73,896 count= 1 avg_used= 73,896 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairCache [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 10 mod=ConstPl [MEMORY] hold= 77,824 used= 65,536 count= 64 avg_used= 1,024 block_cnt= 12 chunk_cnt= 3 mod=TxCtxMgr [MEMORY] hold= 73,728 used= 72,736 count= 1 avg_used= 72,736 block_cnt= 1 chunk_cnt= 1 mod=LogSharedQueueT [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushMeta [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=StorMetaCacheIO [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=DEVICE_MANAGER [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxDataMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxCtxMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LockMemObj [MEMORY] hold= 73,728 used= 66,176 count= 1 avg_used= 66,176 block_cnt= 1 chunk_cnt= 1 mod=Rpc [MEMORY] hold= 73,728 used= 65,536 count= 1 avg_used= 65,536 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushLog [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 9 mod=IntDiffPl [MEMORY] hold= 72,240 used= 1,024 count= 896 avg_used= 1 block_cnt= 10 chunk_cnt= 1 mod=CharsetUtil [MEMORY] hold= 65,216 used= 60,712 count= 4 avg_used= 15,178 block_cnt= 4 chunk_cnt= 4 mod=HB_SERVICE [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 8 mod=MdsTableMgr [MEMORY] hold= 64,000 used= 62,464 count= 8 avg_used= 7,808 block_cnt= 8 chunk_cnt= 8 mod=HashNodeLCSta [MEMORY] hold= 58,400 used= 55,296 count= 16 avg_used= 3,456 block_cnt= 11 chunk_cnt= 8 mod=CACHE_INST [MEMORY] hold= 57,344 used= 50,496 count= 1 avg_used= 50,496 block_cnt= 1 chunk_cnt= 1 mod=palf_env [MEMORY] hold= 54,000 used= 5,760 count= 240 avg_used= 24 block_cnt= 146 chunk_cnt= 27 mod=[T]MemoryContex [MEMORY] hold= 48,960 used= 44,840 count= 2 avg_used= 22,420 block_cnt= 2 chunk_cnt= 2 mod=DynamicFactor [MEMORY] hold= 48,880 used= 44,760 count= 2 avg_used= 22,380 block_cnt= 2 chunk_cnt= 2 mod=Autoincrement [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 5 mod=ColEqualPl [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 4 mod=HexStrPl [MEMORY] hold= 45,792 used= 35,616 count= 53 avg_used= 672 block_cnt= 51 chunk_cnt= 23 mod=[T]ObWarningBuf [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DagWarnHisBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoBkt [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 1 mod=ObDMMDL [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=StrDiffPl [MEMORY] hold= 32,912 used= 22,464 count= 54 avg_used= 416 block_cnt= 23 chunk_cnt= 7 mod=OMT_Worker [MEMORY] hold= 32,768 used= 17,408 count= 2 avg_used= 8,704 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTTP [MEMORY] hold= 32,768 used= 25,664 count= 1 avg_used= 25,664 block_cnt= 1 chunk_cnt= 1 mod=TSQLSessionMgr [MEMORY] hold= 32,768 used= 24,688 count= 2 avg_used= 12,344 block_cnt= 2 chunk_cnt= 2 mod=TLD_AssignedMgr [MEMORY] hold= 32,704 used= 26,079 count= 2 avg_used= 13,039 block_cnt= 2 chunk_cnt= 2 mod=CGReadInfoMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 3 mod=StrPrefixPl [MEMORY] hold= 32,000 used= 31,232 count= 4 avg_used= 7,808 block_cnt= 4 chunk_cnt= 4 mod=HashNodePlanCac [MEMORY] hold= 24,576 used= 22,592 count= 1 avg_used= 22,592 block_cnt= 1 chunk_cnt= 1 mod=IO_MGR [MEMORY] hold= 24,576 used= 16,384 count= 1 avg_used= 16,384 block_cnt= 1 chunk_cnt= 1 mod=SlogWriteBuffer [MEMORY] hold= 24,368 used= 20,136 count= 2 avg_used= 10,068 block_cnt= 2 chunk_cnt= 2 mod=TabletStats [MEMORY] hold= 24,336 used= 20,104 count= 2 avg_used= 10,052 block_cnt= 2 chunk_cnt= 2 mod=TLD_ResourceMgr [MEMORY] hold= 21,152 used= 16,896 count= 22 avg_used= 768 block_cnt= 15 chunk_cnt= 12 mod=TGTimer [MEMORY] hold= 17,056 used= 8,448 count= 3 avg_used= 2,816 block_cnt= 2 chunk_cnt= 2 mod=BaseLogWriter [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=DiaTabletMap [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TabletSvrBucket [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=SlogNopLog [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_ClientTask [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MedCheckSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_TableCtxMgr [MEMORY] hold= 16,384 used= 11,584 count= 1 avg_used= 11,584 block_cnt= 1 chunk_cnt= 1 mod=LogApplyStatus [MEMORY] hold= 16,384 used= 13,824 count= 1 avg_used= 13,824 block_cnt= 1 chunk_cnt= 1 mod=LogReplayStatus [MEMORY] hold= 16,384 used= 8,448 count= 1 avg_used= 8,448 block_cnt= 1 chunk_cnt= 1 mod=PalfEnv [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=ColSubStrPl [MEMORY] hold= 15,648 used= 15,264 count= 2 avg_used= 7,632 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoNode [MEMORY] hold= 11,984 used= 11,592 count= 2 avg_used= 5,796 block_cnt= 2 chunk_cnt= 1 mod=StopWordSet [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=Scheduler [MEMORY] hold= 9,376 used= 8,984 count= 2 avg_used= 4,492 block_cnt= 2 chunk_cnt= 1 mod=FTPluginMap [MEMORY] hold= 9,248 used= 6,732 count= 13 avg_used= 517 block_cnt= 13 chunk_cnt= 12 mod=SqlSessiQuerSql [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapCn [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapDi [MEMORY] hold= 8,192 used= 7,760 count= 2 avg_used= 3,880 block_cnt= 1 chunk_cnt= 1 mod=HashBucPsSessIn [MEMORY] hold= 8,160 used= 5,512 count= 13 avg_used= 424 block_cnt= 12 chunk_cnt= 7 mod=PackStateMap [MEMORY] hold= 8,144 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 9 mod=SequenceIdMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairs [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MatchLsnMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=APPLY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LockWaitMgr [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=PurgeThrottle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LsElectRefInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FetchLogTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=REPLAY_STATUS [MEMORY] hold= 8,128 used= 5,512 count= 13 avg_used= 424 block_cnt= 12 chunk_cnt= 8 mod=SequenceMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=IORunners [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSIter [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ls_id [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=HandleSubmit [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=DASIDCache [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ShareBlksMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MdsDebugMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FTPluginHandle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LCLSender [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompCkmItems [MEMORY] hold= 8,112 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 8 mod=ContextsMap [MEMORY] hold= 8,096 used= 7,904 count= 1 avg_used= 7,904 block_cnt= 1 chunk_cnt= 1 mod=PsSessionInfo [MEMORY] hold= 8,000 used= 7,808 count= 1 avg_used= 7,808 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsInfo [MEMORY] hold= 7,984 used= 7,792 count= 1 avg_used= 7,792 block_cnt= 1 chunk_cnt= 1 mod=ColUsagNode [MEMORY] hold= 7,920 used= 7,728 count= 1 avg_used= 7,728 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsCache [MEMORY] hold= 7,904 used= 7,712 count= 1 avg_used= 7,712 block_cnt= 1 chunk_cnt= 1 mod=DmlStatNode [MEMORY] hold= 7,824 used= 7,632 count= 1 avg_used= 7,632 block_cnt= 1 chunk_cnt= 1 mod=CheckMapNode [MEMORY] hold= 7,136 used= 66 count= 33 avg_used= 2 block_cnt= 31 chunk_cnt= 11 mod=[T]TSIUseWeak [MEMORY] hold= 6,800 used= 6,216 count= 3 avg_used= 2,072 block_cnt= 2 chunk_cnt= 2 mod=DetectManager [MEMORY] hold= 5,056 used= 4,656 count= 2 avg_used= 2,328 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResGrpIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=PxPoolBkt [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResRuleIdMap [MEMORY] hold= 4,288 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 4,160 used= 3,968 count= 1 avg_used= 3,968 block_cnt= 1 chunk_cnt= 1 mod=TinyAllocator [MEMORY] hold= 3,328 used= 256 count= 16 avg_used= 16 block_cnt= 16 chunk_cnt= 14 mod=[T]TSILastOper [MEMORY] hold= 3,328 used= 1,792 count= 8 avg_used= 224 block_cnt= 5 chunk_cnt= 2 mod=LogIOTask [MEMORY] hold= 3,200 used= 3,008 count= 1 avg_used= 3,008 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlDfc [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=IRMMemHashBuck [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=MedCheckMap [MEMORY] hold= 2,368 used= 1,536 count= 4 avg_used= 384 block_cnt= 4 chunk_cnt= 3 mod=TableLockNode [MEMORY] hold= 2,048 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=LogIOWS [MEMORY] hold= 2,000 used= 1,800 count= 1 avg_used= 1,800 block_cnt= 1 chunk_cnt= 1 mod=PxResMgr [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=MSTXCTX [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TabStatMgrLock [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TXBFRetryId [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GCMemtableMap [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GROUP_INDEX_MAP [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=DUP_LS_SET [MEMORY] hold= 1,344 used= 960 count= 2 avg_used= 480 block_cnt= 1 chunk_cnt= 1 mod=PoolArenaArray [MEMORY] hold= 1,280 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=ModuleInitCtx [MEMORY] hold= 1,248 used= 1,056 count= 1 avg_used= 1,056 block_cnt= 1 chunk_cnt= 1 mod=LOG_HASH_MAP [MEMORY] hold= 1,216 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=LockMemtableMgr [MEMORY] hold= 1,152 used= 960 count= 1 avg_used= 960 block_cnt= 1 chunk_cnt= 1 mod=FTParserMgr [MEMORY] hold= 1,120 used= 120 count= 5 avg_used= 24 block_cnt= 2 chunk_cnt= 1 mod=FreezeTask [MEMORY] hold= 976 used= 776 count= 1 avg_used= 776 block_cnt= 1 chunk_cnt= 1 mod=DepTabletSet [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=tenant_rm_mgr [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=CheckPointTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=FlushTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=DiskUsageTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TabShellTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TLD_TIMER [MEMORY] hold= 848 used= 648 count= 1 avg_used= 648 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfo [MEMORY] hold= 752 used= 552 count= 1 avg_used= 552 block_cnt= 1 chunk_cnt= 1 mod=LSFreeze [MEMORY] hold= 672 used= 288 count= 2 avg_used= 144 block_cnt= 2 chunk_cnt= 2 mod=ObTxLSLogCb [MEMORY] hold= 672 used= 472 count= 1 avg_used= 472 block_cnt= 1 chunk_cnt= 1 mod=Election [MEMORY] hold= 576 used= 384 count= 1 avg_used= 384 block_cnt= 1 chunk_cnt= 1 mod=HAScheduler [MEMORY] hold= 544 used= 144 count= 2 avg_used= 72 block_cnt= 1 chunk_cnt= 1 mod=TntSrvObjPool [MEMORY] hold= 512 used= 120 count= 2 avg_used= 60 block_cnt= 2 chunk_cnt= 2 mod=UserResourceMgr [MEMORY] hold= 448 used= 64 count= 2 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfFSCbNode [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTHandler [MEMORY] hold= 400 used= 208 count= 1 avg_used= 208 block_cnt= 1 chunk_cnt= 1 mod=[T]ObMacroBlock [MEMORY] hold= 320 used= 102 count= 1 avg_used= 102 block_cnt= 1 chunk_cnt= 1 mod=TxLSLogBuf [MEMORY] hold= 304 used= 104 count= 1 avg_used= 104 block_cnt= 1 chunk_cnt= 1 mod=RedoLogBuckLock [MEMORY] hold= 304 used= 112 count= 1 avg_used= 112 block_cnt= 1 chunk_cnt= 1 mod=Coordinator [MEMORY] hold= 256 used= 48 count= 1 avg_used= 48 block_cnt= 1 chunk_cnt= 1 mod=[T]TSITabletTab [MEMORY] hold= 256 used= 64 count= 1 avg_used= 64 block_cnt= 1 chunk_cnt= 1 mod=NegParamIdex [MEMORY] hold= 256 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfoMap [MEMORY] hold= 240 used= 41 count= 1 avg_used= 41 block_cnt= 1 chunk_cnt= 1 mod=KeepAliveBuf [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfRCCbNode [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=RebuildCbNode [MEMORY] hold= 208 used= 16 count= 1 avg_used= 16 block_cnt= 1 chunk_cnt= 1 mod=logservice [MEMORY] hold= 426,895,920 used= 409,703,684 count= 33,078 avg_used= 12,385 mod=SUMMARY [2024-10-16 14:57:46.570467] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=63] [MEMORY] tenant_id= 1 ctx_id= MEMSTORE_CTX_ID hold= 245,366,784 used= 243,449,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 block_cnt= 117 chunk_cnt= 117 mod=Memstore [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 mod=SUMMARY [2024-10-16 14:57:46.570478] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TRANS_CTX_MGR_ID hold= 2,097,152 used= 729,088 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 block_cnt= 1 chunk_cnt= 1 mod=PartTranCtxMgr [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 mod=SUMMARY [2024-10-16 14:57:46.570488] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= PLAN_CACHE_CTX_ID hold= 79,691,776 used= 73,331,136 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 68,663,616 used= 62,143,518 count= 2,832 avg_used= 21,943 block_cnt= 2,832 chunk_cnt= 37 mod=SqlPhyPlan [MEMORY] hold= 3,414,336 used= 3,146,720 count= 326 avg_used= 9,652 block_cnt= 326 chunk_cnt= 37 mod=SqlPlanCache [MEMORY] hold= 1,251,200 used= 1,140,800 count= 575 avg_used= 1,984 block_cnt= 192 chunk_cnt= 37 mod=CreateContext [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 73,331,136 used= 66,432,638 count= 3,735 avg_used= 17,786 mod=SUMMARY [2024-10-16 14:57:46.570502] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= GLIBC hold= 2,097,152 used= 81,040 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 80,944 used= 59,018 count= 263 avg_used= 224 block_cnt= 20 chunk_cnt= 1 mod=PlJit [MEMORY] hold= 96 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PlCodeGen [MEMORY] hold= 81,040 used= 59,050 count= 264 avg_used= 223 mod=SUMMARY [2024-10-16 14:57:46.570513] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= CO_STACK hold= 136,314,880 used= 133,668,864 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 block_cnt= 259 chunk_cnt= 65 mod=CoStack [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 mod=SUMMARY [2024-10-16 14:57:46.570529] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= KVSTORE_CACHE_ID hold= 861,929,472 used= 861,929,472 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 mod=SUMMARY [2024-10-16 14:57:46.570541] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= META_OBJ_CTX_ID hold= 37,748,736 used= 36,290,496 limit= 429,496,720 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 29,491,200 used= 26,236,400 count= 400 avg_used= 65,591 block_cnt= 400 chunk_cnt= 18 mod=L_TabletPool [MEMORY] hold= 6,709,248 used= 5,952,128 count= 91 avg_used= 65,408 block_cnt= 91 chunk_cnt= 15 mod=N_TabletPool [MEMORY] hold= 90,048 used= 84,768 count= 2 avg_used= 42,384 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 36,290,496 used= 32,273,296 count= 493 avg_used= 65,463 mod=SUMMARY [2024-10-16 14:57:46.570552] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= TX_CALLBACK_CTX_ID hold= 12,582,912 used= 6,234,176 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 block_cnt= 767 chunk_cnt= 6 mod=MemtableCallbac [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:46.570563] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= LOB_CTX_ID hold= 2,097,152 used= 81,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LobAllocator [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 81,856 used= 73,344 count= 2 avg_used= 36,672 mod=SUMMARY [2024-10-16 14:57:46.570573] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= PS_CACHE_CTX_ID hold= 2,097,152 used= 16,288 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 block_cnt= 4 chunk_cnt= 1 mod=SqlPsCache [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 mod=SUMMARY [2024-10-16 14:57:46.570587] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= TX_DATA_TABLE hold= 54,546,432 used= 52,771,456 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 49,580,800 used= 48,409,600 count= 6,100 avg_used= 7,936 block_cnt= 6,100 chunk_cnt= 25 mod=TX_DATA_SLICE [MEMORY] hold= 3,182,528 used= 3,153,728 count= 3 avg_used= 1,051,242 block_cnt= 3 chunk_cnt= 3 mod=MEMTABLE_ARENA [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 52,771,456 used= 51,571,264 count= 6,104 avg_used= 8,448 mod=SUMMARY [2024-10-16 14:57:46.570608] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= MDS_DATA_ID hold= 4,194,304 used= 398,272 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 260,096 used= 253,952 count= 32 avg_used= 7,936 block_cnt= 32 chunk_cnt= 2 mod=MdsTable [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 2 mod=TX_OP [MEMORY] hold= 398,272 used= 388,864 count= 49 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:46.570621] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= SCHEMA_SERVICE hold= 2,097,152 used= 8,128 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=NonPartTenCac [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:46.570641] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,082,318,848 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 371,380,224 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 14,680,064 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 104,857,600 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 20,971,520 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= RPC_CTX_ID hold_bytes= 6,291,456 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 26,357,760 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 237,785,088 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= UNEXPECTED_IN_500 hold_bytes= 295,800,832 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:46.570841] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 371,380,224 used= 311,868,240 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 66,584,784 used= 66,551,824 count= 33 avg_used= 2,016,721 block_cnt= 33 chunk_cnt= 33 mod=Compressor [MEMORY] hold= 31,477,760 used= 31,458,304 count= 1 avg_used= 31,458,304 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 23,584,768 used= 23,499,200 count= 9 avg_used= 2,611,022 block_cnt= 9 chunk_cnt= 8 mod=IoControl [MEMORY] hold= 15,925,248 used= 15,247,872 count= 216 avg_used= 70,592 block_cnt= 216 chunk_cnt= 42 mod=[T]ObSessionDIB [MEMORY] hold= 12,873,728 used= 12,856,352 count= 1 avg_used= 12,856,352 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 9,457,664 used= 9,437,784 count= 1 avg_used= 9,437,784 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 5,505,024 used= 5,238,784 count= 32 avg_used= 163,712 block_cnt= 32 chunk_cnt= 22 mod=Iteratorget_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5118, mem_used=10398720) [2024-10-16 14:57:46.612261] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:46.614169] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C8-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:46.614187] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C8-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061866147856003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061866147856003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:46.615239] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9C8-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061866614843000, start_id=1729061866614843000, end_id=1729061866614843001) [2024-10-16 14:57:46.618278] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C8-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061866617943000, start_id=1729061866617943000, end_id=1729061866617943001) [2024-10-16 14:57:46.618343] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C8-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=9566, avg_group_log_size=597) [2024-10-16 14:57:46.618352] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C8-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9567) [2024-10-16 14:57:46.618560] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9566) [2024-10-16 14:57:46.619410] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=2] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:46.646646] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.647327] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969B9FD-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061866646597000, start_id=1729061866646597000, end_id=1729061866646597001) [2024-10-16 14:57:46.648648] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0C87-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866647932000, start_id=1729061866647932000, end_id=1729061866647932001) [2024-10-16 14:57:46.648824] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969B9FE-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061866647932000, start_id=1729061866647932000, end_id=1729061866647932001) [2024-10-16 14:57:46.648875] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0C8B-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061866647932000, start_id=1729061866647932000, end_id=1729061866647932001) [2024-10-16 14:57:46.649261] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969B9FC-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061866648993000, start_id=1729061866648993000, end_id=1729061866648993001) [2024-10-16 14:57:46.651316] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E34-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173199, log_task={header:{begin_lsn:{lsn:14352380444}, end_lsn:{lsn:14352380566}, log_id:115173199, min_scn:{val:1729061866147856004, v:0}, max_scn:{val:1729061866147856004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352380322}, prev_proposal_id:17, committed_end_lsn:{lsn:14352380444}, data_checksum:940009244, accum_checksum:2261889595, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061866551214, freeze_ts:1729061866551214, submit_ts:1729061866551218, flushed_ts:1729061866553161, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1943}, fs_cb_begin_ts=1729061866651309, log_life_time=100095) [2024-10-16 14:57:46.651594] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:46.669054] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C55-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:46.669082] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C55-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061866147856004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061866147856004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:46.669781] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:46.672686] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.673725] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C55-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061866673474000, start_id=1729061866673474000, end_id=1729061866673474001) [2024-10-16 14:57:46.690300] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D94-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510334558694, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410607, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.691242] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997924-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.691301] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997924-0-0] [lt=35][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"PACKAGE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.691562] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D98-0-0] [lt=13][errcode=-4201] hashtable not init [2024-10-16 14:57:46.691570] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D98-0-0] [lt=8][errcode=-4016] get cursor info failed(cursor_id=1115, get_sessid()=3221565766) [2024-10-16 14:57:46.691581] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997924-0-0] [lt=15][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.691636] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061866690655000, start_id=1729061866690655000, end_id=1729061866690655001) [2024-10-16 14:57:46.691632] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997924-0-0] [lt=28][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"ROUTINE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.691698] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4123) [2024-10-16 14:57:46.692181] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85005-0-0] [lt=5][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497465, log_task={header:{begin_lsn:{lsn:207589553416}, end_lsn:{lsn:207589553538}, log_id:218497465, min_scn:{val:1729061866147856005, v:0}, max_scn:{val:1729061866147856005, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589553294}, prev_proposal_id:81, committed_end_lsn:{lsn:207589553416}, data_checksum:940009244, accum_checksum:3169336668, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061866592085, freeze_ts:1729061866592085, submit_ts:1729061866592089, flushed_ts:1729061866596717, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:4628}, fs_cb_begin_ts=1729061866692175, log_life_time=100090) [2024-10-16 14:57:46.692286] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997924-0-0] [lt=14][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.692341] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997924-0-0] [lt=24][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TRIGGER_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.694520] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D44-0-0] [lt=23] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061866685737, keepalive_ts:-1}) [2024-10-16 14:57:46.694536] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D44-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061866685737, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:46.694566] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D45-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061866685781, keepalive_ts:-1}) [2024-10-16 14:57:46.694571] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D45-0-0] [lt=4] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061866685781, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:46.697042] INFO [STORAGE] runTimerTask (ob_locality_manager.cpp:708) [21260][LocalityReload][T0][Y0-0000000000000000-0-0] [lt=11] runTimer to refresh locality_info(ret=0) [2024-10-16 14:57:46.697826] INFO [SHARE] set_server_locality_array (ob_server_locality_cache.cpp:346) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=12] set_server_locality_array success(server_locality_array=[{inited:true, addr:"10.0.4.7:2890", zone:"myoceanbase1", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494256558000, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.8:2890", zone:"myoceanbase2", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494430723273, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.9:2890", zone:"myoceanbase3", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494593087951, server_stop_time:0, server_status:1}], has_readonly_zone=false) [2024-10-16 14:57:46.697855] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=25] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1, primary_zone=RANDOM) [2024-10-16 14:57:46.697861] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=6] add_locality_zone success(locality_zone={tenant_id:1, region_priority:18446744073709551615}) [2024-10-16 14:57:46.697869] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=5] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1001, primary_zone=RANDOM) [2024-10-16 14:57:46.697871] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1001, region_priority:18446744073709551615}) [2024-10-16 14:57:46.697877] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1002, primary_zone=RANDOM) [2024-10-16 14:57:46.697880] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] add_locality_zone success(locality_zone={tenant_id:1002, region_priority:18446744073709551615}) [2024-10-16 14:57:46.697889] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=8] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1003, primary_zone=RANDOM) [2024-10-16 14:57:46.697891] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1003, region_priority:18446744073709551615}) [2024-10-16 14:57:46.697895] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1004, primary_zone=RANDOM) [2024-10-16 14:57:46.697897] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1004, region_priority:18446744073709551615}) [2024-10-16 14:57:46.697901] INFO [SHARE] load_region (ob_locality_table_operator.cpp:373) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] load region(ret=0, locality_info={version:0, local_region:"sys_region", local_zone:"myoceanbase1", local_idc:"", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["myoceanbase1", "myoceanbase2", "myoceanbase3"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:18446744073709551615}, {tenant_id:1002, region_priority:18446744073709551615}, {tenant_id:1003, region_priority:18446744073709551615}, {tenant_id:1004, region_priority:18446744073709551615}]}) [2024-10-16 14:57:46.698984] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:46.706399] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15E2-0-0] [lt=5] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:46.709524] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B99792B-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"DATA_TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.709775] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B99792B-0-0] [lt=40][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.713592] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:46.723877] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:57:46.723888] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:57:46.723892] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:46.724496] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061864049233000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880098662000, latest_log_ts:{val:1729061863549302001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:46.725178] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=43] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061864049233000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:46.725811] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=31] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061864049233000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:46.726418] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=25] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061864049233000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:46.726559] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:46.726756] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=371, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:46.726767] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=331, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:46.726776] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:46.732114] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061866730977000, start_id=1729061866730977000, end_id=1729061866730977001) [2024-10-16 14:57:46.733338] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:46.734621] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=28] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:46.743784] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1729061866742099000, start_id=1729061866742099000, end_id=1729061866742099001) [2024-10-16 14:57:46.745944] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061866744373000, start_id=1729061866744373000, end_id=1729061866744373001) [2024-10-16 14:57:46.751293] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E36-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:46.751736] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9189-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:46.754254] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.756774] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=2] gc stale ls task succ [2024-10-16 14:57:46.756794] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=11] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:46.756801] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:46.760268] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488E-0-0] [lt=5] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061866728020, keepalive_ts:-1}) [2024-10-16 14:57:46.760284] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488E-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061866728020, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:46.767882] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:46.768670] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F419C-0-0] [lt=21] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:46.771991] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:46.772747] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.781849] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=22] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:46.788625] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788833] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788843] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788859] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788868] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788876] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788887] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788897] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788909] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788921] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.788964] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:46.792368] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866791604000, start_id=1729061866791604000, end_id=1729061866791604001) [2024-10-16 14:57:46.792712] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85007-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497466, log_task={header:{begin_lsn:{lsn:207589553538}, end_lsn:{lsn:207589553660}, log_id:218497466, min_scn:{val:1729061866647932000, v:0}, max_scn:{val:1729061866647932000, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589553416}, prev_proposal_id:81, committed_end_lsn:{lsn:207589553538}, data_checksum:940009244, accum_checksum:2643978179, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061866692167, freeze_ts:1729061866692167, submit_ts:1729061866692170, flushed_ts:1729061866693860, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:1690}, fs_cb_begin_ts=1729061866792705, log_life_time=100538) [2024-10-16 14:57:46.802990] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:46.809531] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061866808004000, start_id=1729061866808004000, end_id=1729061866808004001) [2024-10-16 14:57:46.812934] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:46.817574] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC586-0-0] [lt=25] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:57:46.818210] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC586-0-0] [lt=12] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1002) [2024-10-16 14:57:46.818219] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC586-0-0] [lt=8] no transfer partition job [2024-10-16 14:57:46.818225] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC586-0-0] [lt=3] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:57:46.820452] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC586-0-0] [lt=6] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1002, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:57:46.820472] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC586-0-0] [lt=14] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:57:46.821578] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:500) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC586-0-0] [lt=12][errcode=-4152] get obj error(ret=-4152) [2024-10-16 14:57:46.821619] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC586-0-0] [lt=9] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1001, status:0}], ls_array_=[{tenant_id:1002, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729057174083520, last_statistic_bg_stat_time=1729008957751717, last_statistic_schema_version=1729008953019688, last_statistic_max_transfer_task_id={id:-1}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:57:46.830624] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1003}, {id:1001}]) [2024-10-16 14:57:46.848154] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997987-0-0] [lt=0][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.848197] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997987-0-0] [lt=22][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.848209] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997987-0-0] [lt=7][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.848218] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997987-0-0] [lt=7][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.848242] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997987-0-0] [lt=4][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.848263] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997987-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.848282] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997987-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.848291] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B997987-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:46.851378] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E38-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:46.852045] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:46.852057] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:46.853488] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5044, mem_used=10398720) [2024-10-16 14:57:46.854319] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.862094] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22703][T1004_TntShared][T1004][Y0-0000000000000000-0-0] [lt=15] clear dtl interm result cost(us)(clear_cost=936, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:46.865243] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:46.865421] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B99798E-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.865565] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B99798E-0-0] [lt=37][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.865683] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B99798E-0-0] [lt=22][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.870851] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D72-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:46.870874] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D72-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061866647932001, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061866647932001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:46.871806] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D72-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061866871120000, start_id=1729061866871120000, end_id=1729061866871120001) [2024-10-16 14:57:46.872872] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.877682] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CB7-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:46.878898] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D72-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061866878639000, start_id=1729061866878639000, end_id=1729061866878639001) [2024-10-16 14:57:46.878956] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D72-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:46.878969] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D72-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:46.895343] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=30] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:46.895960] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=22] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:46.896013] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061857241966000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:46.901705] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:46.902691] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9979AC-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"a", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.902729] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:46.902744] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:46.902972] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=291, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:46.902982] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=82, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:46.903000] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:46.903272] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF0B-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:46.903278] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF0B-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1061, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:46.920451] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:46.920469] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=214) [2024-10-16 14:57:46.921269] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DDA-0-0] [lt=7] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:57:46.923619] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=6] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:46.928522] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:46.932447] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061866930823000, start_id=1729061866930823000, end_id=1729061866930823001) [2024-10-16 14:57:46.934700] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:46.935802] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE2-0-0] [lt=23] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061866899996, keepalive_ts:-1}) [2024-10-16 14:57:46.935815] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE2-0-0] [lt=11] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061866899996, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:46.935858] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4383-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:46.935864] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4383-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3331, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:46.942725] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3077, mem_used=6239232) [2024-10-16 14:57:46.943837] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061866942552000, start_id=1729061866942552000, end_id=1729061866942552001) [2024-10-16 14:57:46.945957] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF1BC-0-0] [lt=12] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.132042837142944336e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=306, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.972065176117725287e+03) [2024-10-16 14:57:46.946083] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061866944999000, start_id=1729061866944999000, end_id=1729061866944999001) [2024-10-16 14:57:46.949090] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9979D4-0-0] [lt=0][errcode=-5171] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:46.952881] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9979D7-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.952946] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9979D7-0-0] [lt=38][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.953000] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9979D7-0-0] [lt=18][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.953034] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9979D7-0-0] [lt=17][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.954405] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.955418] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061856954480, end_time=1729061866955418, dag_status={}) [2024-10-16 14:57:46.955430] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:46.955436] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:46.955440] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:46.955444] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:46.955447] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:46.955450] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:46.955454] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:46.955458] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:46.955461] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:46.955465] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955471] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955475] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955479] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955483] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955487] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955494] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955499] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955503] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955507] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955510] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955514] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955519] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955522] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955526] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955530] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955534] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955538] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955542] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955545] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955549] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955553] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955557] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955561] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955565] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955569] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955573] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955576] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955580] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955584] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955588] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955592] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955595] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955604] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955608] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955612] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955616] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955620] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955624] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955628] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955632] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955635] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955639] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955643] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955647] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955651] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955654] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955658] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955662] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955665] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955669] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955673] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955677] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955681] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955684] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955688] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955692] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955696] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:46.955700] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:46.955703] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:46.955707] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:46.955710] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:46.955712] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:46.955715] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:46.955717] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:46.955719] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:46.955722] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:46.955724] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:46.955727] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:46.958672] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBC-0-0] [lt=17] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:46.958689] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBC-0-0] [lt=12] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:46.958803] WDIAG [SQL.RESV] check_table_column_namespace (ob_column_namespace_checker.cpp:124) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9979DB-0-0] [lt=0][errcode=-5207] column in all tables is ambiguous(q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:46.958844] WDIAG [SQL.RESV] resolve_table_column_expr (ob_dml_resolver.cpp:4587) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9979DB-0-0] [lt=40][errcode=-5207] column not found in table(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:46.958871] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1001][YB4A0A000407-00061E9B3B9979DB-0-0] [lt=17][errcode=-5207] resolve table column expr failed(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6c2c860 0x688c2c2 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:46.959766] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:46.959778] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27628320, tenant_id=1004) [2024-10-16 14:57:46.959786] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:46.960967] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=17] ====== tenant freeze timer task ====== [2024-10-16 14:57:46.967258] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:46.972933] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:46.977532] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:46.991762] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] no need to update min/max version(new_version={val:1729061866589913000, v:0}, min_version={val:1729061866589913000, v:0}, max_version={val:1729061866589913000, v:0}) [2024-10-16 14:57:46.992466] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:47.003015] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=262, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:47.003029] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=53, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:47.003716] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF0E-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=378, avg_log_submit_to_slide_time=1066) [2024-10-16 14:57:47.009758] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867007062000, start_id=1729061867007062000, end_id=1729061867007062001) [2024-10-16 14:57:47.009888] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 791.75; ) [2024-10-16 14:57:47.009902] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:47.009939] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 1579.50; ) [2024-10-16 14:57:47.009944] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:47.009952] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:47.011494] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061867011494, bytes: 4446816251569, bw: 0.423567 MB/s, add_ts: 1002714, add_bytes: 445348 [2024-10-16 14:57:47.012619] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] Garbage Collector is running(seq_=656699, gc_interval=10000000) [2024-10-16 14:57:47.012648] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=10] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:47.012657] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=29) [2024-10-16 14:57:47.012663] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:47.013335] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:47.013348] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=10] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:47.013354] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] execute_gc cost time(ret=0, time_us=7) [2024-10-16 14:57:47.013358] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:47.013567] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:47.016289] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Cache wash clean map node details(ret=0, clean_node_count=15, clean_time=0, clean_start_pos=1195366, clean_num=62914) [2024-10-16 14:57:47.017960] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B9979FC-0-0] [lt=0] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1002, table_id:10002, add_timestamp:1729061867017946}) [2024-10-16 14:57:47.017986] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E45-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1002, table_id=10002, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061867017973, sql_renew_time:1729061867017973, is_mark_fail:false}]) [2024-10-16 14:57:47.018002] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E45-0-0] [lt=17] success to process renew task(task={tenant_id:1002, table_id:10002, add_timestamp:1729061867017946}, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061867017973, sql_renew_time:1729061867017973, is_mark_fail:false}]) [2024-10-16 14:57:47.027557] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=25] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:47.027572] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:47.027593] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:47.027608] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:47.028039] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=447, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:47.028057] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=252, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:47.028080] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:47.028165] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4386-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=13, avg_log_submit_to_flush_time=1816, avg_log_submit_to_slide_time=2519) [2024-10-16 14:57:47.032809] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4002, dropped:340, tid:21226}, {errcode:-5171, dropped:24360, tid:21226}]) [2024-10-16 14:57:47.033046] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=0][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLLATIONS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.033068] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=21][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLLATIONS' [2024-10-16 14:57:47.033079] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=7][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLLATIONS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.033087] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLLATIONS' [2024-10-16 14:57:47.033093] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLLATIONS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.033100] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLLATIONS' [2024-10-16 14:57:47.033104] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLLATIONS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.033112] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLLATIONS' [2024-10-16 14:57:47.033115] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLLATIONS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.033123] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLLATIONS' [2024-10-16 14:57:47.033127] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLLATIONS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.033134] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A04-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLLATIONS' [2024-10-16 14:57:47.033851] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A05-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLLATION_CHARACTER_SET_APPLICABILITY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.033871] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A05-0-0] [lt=18][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLLATION_CHARACTER_SET_APPLICABILITY' [2024-10-16 14:57:47.033879] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A05-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLLATION_CHARACTER_SET_APPLICABILITY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.033885] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A05-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLLATION_CHARACTER_SET_APPLICABILITY' [2024-10-16 14:57:47.034744] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROCESSLIST", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.034763] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=17][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROCESSLIST' [2024-10-16 14:57:47.034771] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROCESSLIST", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.034780] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROCESSLIST' [2024-10-16 14:57:47.034792] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROCESSLIST", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.034799] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROCESSLIST' [2024-10-16 14:57:47.034804] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROCESSLIST", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.034811] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROCESSLIST' [2024-10-16 14:57:47.034815] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROCESSLIST", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.034823] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROCESSLIST' [2024-10-16 14:57:47.034829] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROCESSLIST", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.034836] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROCESSLIST' [2024-10-16 14:57:47.034840] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROCESSLIST", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.034848] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROCESSLIST' [2024-10-16 14:57:47.034851] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROCESSLIST", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.034859] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A06-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROCESSLIST' [2024-10-16 14:57:47.037729] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037745] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=15][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037754] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037760] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037768] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=6][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037776] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037780] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037787] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037791] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037796] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037800] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037808] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037812] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037819] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037824] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037831] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037838] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037846] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037852] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037859] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037865] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037872] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.037878] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"KEY_COLUMN_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.037885] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A07-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'KEY_COLUMN_USAGE' [2024-10-16 14:57:47.038785] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ENGINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.038800] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=13][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ENGINES' [2024-10-16 14:57:47.038810] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ENGINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.038815] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ENGINES' [2024-10-16 14:57:47.038821] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ENGINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.038826] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ENGINES' [2024-10-16 14:57:47.038832] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ENGINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.038836] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ENGINES' [2024-10-16 14:57:47.038842] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ENGINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.038846] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ENGINES' [2024-10-16 14:57:47.038852] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ENGINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.038856] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A08-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ENGINES' [2024-10-16 14:57:47.041304] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041319] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=14][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041330] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041335] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041342] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041347] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041353] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041357] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041363] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041367] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041374] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041378] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041403] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041408] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041443] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041450] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041470] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041474] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041501] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041513] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=11][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041528] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041536] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041553] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041559] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041571] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041575] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041653] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041659] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041666] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041670] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041676] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041681] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041686] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041691] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041697] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041701] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041707] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041711] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041717] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041721] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041727] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041732] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041737] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041741] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=3][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041747] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041753] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041759] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041767] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041773] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041780] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041787] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041794] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041800] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041806] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041812] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041816] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041821] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041826] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041831] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041836] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.041841] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"ROUTINES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.041846] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A09-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'ROUTINES' [2024-10-16 14:57:47.042620] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:47.042633] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:47.042646] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291947519, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:47.042814] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042831] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=16][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042845] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=7][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042853] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042860] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042867] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042873] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042880] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042887] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042894] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042900] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042907] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042913] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042920] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042925] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042933] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042938] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042944] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042949] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042953] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042959] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042963] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042968] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042973] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042978] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042982] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.042991] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=6][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.042998] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.043005] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.043012] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.043020] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.043024] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.043030] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.043034] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.043039] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"PROFILING", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.043044] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0A-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'PROFILING' [2024-10-16 14:57:47.043975] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.043994] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=17][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044002] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044011] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044019] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044028] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044032] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044039] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044044] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044051] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044054] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044062] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044065] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044072] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044077] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044084] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044088] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044093] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044096] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044101] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044104] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044109] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044125] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044131] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044137] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044146] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=9][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044150] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044158] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044161] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044169] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.044173] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_EVENT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.044179] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0B-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_EVENT' [2024-10-16 14:57:47.045031] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0C-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$SESSION_WAIT", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:47.045045] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A0C-0-0] [lt=13][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$SESSION_WAIT' [2024-10-16 14:57:47.046454] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=15] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:47.052957] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867051892000, start_id=1729061867051892000, end_id=1729061867051892001) [2024-10-16 14:57:47.052976] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061867052952], gts:1729061867051892000, latest_srr:[mts=1729061867052952]}) [2024-10-16 14:57:47.052986] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:47.054438] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:47.054453] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:47.054459] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.072998] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.083575] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=10] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:47.084165] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:47.084211] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061860198021000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:47.084249] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=38] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061860198021000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1002, freeze_info:{latest_snapshot_gc_scn:{val:1729061860198021000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}}) [2024-10-16 14:57:47.092562] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061867091473000, start_id=1729061867091473000, end_id=1729061867091473001) [2024-10-16 14:57:47.092749] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8500D-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:47.099618] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:47.099633] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151784220, tenant_id=1) [2024-10-16 14:57:47.099645] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:47.099663] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=6] TxData Memory Statistic(logstream info): (ls_id={id:1}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=38) [2024-10-16 14:57:47.099674] INFO [STORAGE] check_and_freeze_tx_data_ (ob_tenant_freezer.cpp:566) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] TxData Memory Statistic : (Tenant Total Memory(MB)=2048, Tenant Frozen TxData Memory(MB)=0, Tenant Active TxData Memory(MB)=38, Freeze TxData Trigger Memory(MB)=40, Total TxDataTable Hold Memory(MB)=52, Total TxDataTable Memory Limit(MB)=409) [2024-10-16 14:57:47.102822] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:47.102839] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:47.103440] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061867103438, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007615, add_bytes: 0 [2024-10-16 14:57:47.118148] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4D-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.118171] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4D-0-0] [lt=15][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.118180] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4D-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.118188] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4D-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.118204] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4D-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.118224] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4D-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.119636] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:47.120060] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4E-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.120082] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4E-0-0] [lt=10][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.120136] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4E-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.120198] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A4E-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.122436] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15DE-0-0] [lt=13] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:47.124207] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15E0-0-0] [lt=11] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:47.126638] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061857113468, end_time=1729061867126637, dag_status={}) [2024-10-16 14:57:47.126651] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:47.126658] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:47.126666] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:47.126669] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:47.126673] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:47.126676] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:47.126679] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:47.126682] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:47.126686] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:47.126689] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126695] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126699] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126703] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126707] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126711] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126715] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126719] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126723] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126726] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126730] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126734] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126738] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126742] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126746] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126750] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126754] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126757] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126761] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126765] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126769] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126773] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126777] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126781] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126785] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126789] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126792] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126796] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126800] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126804] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126807] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126811] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126815] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126819] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126823] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126827] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126831] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126835] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126839] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126842] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126846] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126850] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126854] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126858] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126861] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126865] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126869] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126873] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126877] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126880] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126884] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126888] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126891] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126895] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126899] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126903] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126907] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126910] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:47.126915] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:47.126918] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:47.126922] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:47.126924] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:47.126927] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:47.126929] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:47.126932] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:47.126934] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:47.126936] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:47.126939] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:47.126942] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:47.126961] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:47.126971] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:47.129299] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129320] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=13][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129340] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129352] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129369] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129381] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129416] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129431] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129445] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129460] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129538] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129547] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129554] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.129562] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A55-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132325] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15DA-0-0] [lt=14] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:47.132355] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132371] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=10][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132388] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132396] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132405] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132413] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132425] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132478] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132486] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132495] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.132507] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A57-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.134654] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061867133829000, start_id=1729061867133829000, end_id=1729061867133829001) [2024-10-16 14:57:47.134774] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:47.136636] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=8] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3110, mem_used=6239232) [2024-10-16 14:57:47.144455] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867143651000, start_id=1729061867143651000, end_id=1729061867143651001) [2024-10-16 14:57:47.145715] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:47.147352] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061867145793000, start_id=1729061867145793000, end_id=1729061867145793001) [2024-10-16 14:57:47.147404] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:47.147417] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:47.147708] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=417, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:47.147722] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=280, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:47.147736] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10666) [2024-10-16 14:57:47.147745] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=6, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:47.147870] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA04-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061867147581000, start_id=1729061867147581000, end_id=1729061867147581001) [2024-10-16 14:57:47.148058] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F952E-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:47.148063] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F952E-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=1571, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:47.148067] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F952E-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=808, avg_log_submit_to_slide_time=1540) [2024-10-16 14:57:47.148184] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA03-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061867147581000, start_id=1729061867147581000, end_id=1729061867147581001) [2024-10-16 14:57:47.148223] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA02-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867147581000, start_id=1729061867147581000, end_id=1729061867147581001) [2024-10-16 14:57:47.148302] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA01-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061867147581000, start_id=1729061867147581000, end_id=1729061867147581001) [2024-10-16 14:57:47.148320] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA05-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061867147581000, start_id=1729061867147581000, end_id=1729061867147581001) [2024-10-16 14:57:47.148446] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=9] current monitor number(seq_=-1) [2024-10-16 14:57:47.151427] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A62-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.151532] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A62-0-0] [lt=40][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"PACKAGE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.151946] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A62-0-0] [lt=23][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.152041] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A62-0-0] [lt=32][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"ROUTINE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.152616] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A62-0-0] [lt=17][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.152671] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A62-0-0] [lt=29][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TRIGGER_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.154525] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.166166] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.166203] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=20][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.166244] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.166317] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.166338] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.166368] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.166383] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.166429] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.166449] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.166543] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A67-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.167184] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77DD-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:47.170335] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A69-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"DATA_TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.170540] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A69-0-0] [lt=36][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.173050] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.175933] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A6B-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.175962] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A6B-0-0] [lt=12][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.176120] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A6B-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.176147] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A6B-0-0] [lt=12][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.176175] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A6B-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.176191] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A6B-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.176237] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A6B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.176258] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A6B-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.176352] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997A6B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.190947] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5031-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:47.190969] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5031-0-0] [lt=21] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:47.193104] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8500F-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:47.193123] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8500F-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:47.193502] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=208, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:47.193515] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=543, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:47.193554] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:47.194906] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA885-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:47.194926] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA885-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061867147581000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061867147581000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:47.197694] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:57:47.197813] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D86-0-0] [lt=8] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061877008855000, latest_log_ts:{val:1729061860546474001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:47.197876] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D86-0-0] [lt=59] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:47.198920] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA885-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061867198663000, start_id=1729061867198663000, end_id=1729061867198663001) [2024-10-16 14:57:47.198996] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA885-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10665, avg_group_log_size=592) [2024-10-16 14:57:47.199005] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA885-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10665) [2024-10-16 14:57:47.199018] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:142) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA885-0-0] [lt=6] transaction rpc statistics(total_rt=1871, total_process=20, avg_rt=89) [2024-10-16 14:57:47.199040] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=57] success to update min reserved snapshot(reserved_snapshot=1729060064434291000, duration=1800, snapshot_gc_ts=1729061864434291000) [2024-10-16 14:57:47.199373] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10665) [2024-10-16 14:57:47.199406] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=279, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:47.199421] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=159, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:47.199453] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:47.199763] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1059-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=456, avg_log_submit_to_slide_time=1881) [2024-10-16 14:57:47.199816] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=6] update reserved snapshot finished(cost_ts=778, reserved_snapshot=1729060064434291000) [2024-10-16 14:57:47.199968] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:47.203186] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:47.209816] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061867208779000, start_id=1729061867208779000, end_id=1729061867208779001) [2024-10-16 14:57:47.210483] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=43, this=0x2b0e07718280, extra_info="") [2024-10-16 14:57:47.210493] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=1034, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:57:47.210497] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:57:47.210501] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=-650, this=0x2b0e07718910, extra_info="") [2024-10-16 14:57:47.210504] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=427, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:57:47.216583] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:47.228447] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D9A-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510335096842, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410608, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.233748] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:47.243156] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C108D-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867242867000, start_id=1729061867242867000, end_id=1729061867242867001) [2024-10-16 14:57:47.255025] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.256848] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256868] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=13][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256877] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256885] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256892] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256900] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256908] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256915] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256923] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256930] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256972] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256981] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.256989] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.257048] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.257110] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA2-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.261725] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601426, offset=9632318}, end_cursor:ObLogCursor{file_id=192, log_id=47601427, offset=9632602}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cfc050, len:1474, log_data_len:284, seq:47601426, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:47.263154] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21531][T1_TntSharedTim][T1][Y0-0000000000000000-0-0] [lt=15] clear dtl interm result cost(us)(clear_cost=1817, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:47.265090] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4A9-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061867264301000, start_id=1729061867264301000, end_id=1729061867264301001) [2024-10-16 14:57:47.269619] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269639] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=14][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269647] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269655] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269663] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269670] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269684] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269695] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269703] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269711] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269718] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269726] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269733] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269764] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269769] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77DF-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:47.269831] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269915] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.269997] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.270061] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.270069] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AA6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.273109] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.276647] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601427, offset=9632602}, end_cursor:ObLogCursor{file_id=192, log_id=47601428, offset=9632876}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dec050, len:1190, log_data_len:274, seq:47601427, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:47.276746] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Slow write(duration=10216, write_len=4096) [2024-10-16 14:57:47.276754] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=8] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257774, offset=39036418}, end_cursor:ObLogCursor{file_id=197, log_id=49257775, offset=39036701}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7292050, len:2558, log_data_len:283, seq:49257774, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:47.278385] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CBF-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:47.278648] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=264, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:47.278662] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=277, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:47.278698] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:47.280367] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB374-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061867278126000, start_id=1729061867278126000, end_id=1729061867278126001) [2024-10-16 14:57:47.287650] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601428, offset=9632876}, end_cursor:ObLogCursor{file_id=192, log_id=47601429, offset=9633150}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e58050, len:916, log_data_len:274, seq:47601428, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:47.289738] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601429, offset=9633150}, end_cursor:ObLogCursor{file_id=192, log_id=47601430, offset=9633424}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d14050, len:642, log_data_len:274, seq:47601429, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:47.293155] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293180] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=18][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293189] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293220] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293232] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293256] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293267] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293279] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293289] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293300] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293311] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293322] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293336] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293348] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293358] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.293369] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AB7-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.294390] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867292604000, start_id=1729061867292604000, end_id=1729061867292604001) [2024-10-16 14:57:47.296227] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1039-0-0] [lt=9] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:47.298724] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561332, miss_cnt:1, cache_read_size:152590685200, hit_ratio:"9.999999928346915423e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:47.298755] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598919, miss_cnt:9, cache_read_size:149777918711, hit_ratio:"9.999999345925136840e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:47.299646] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=6] Slow write(duration=18596, write_len=4096) [2024-10-16 14:57:47.299656] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257775, offset=39036701}, end_cursor:ObLogCursor{file_id=197, log_id=49257776, offset=39036983}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72ba050, len:2275, log_data_len:282, seq:49257775, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:47.303481] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2B8-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:47.304844] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:47.305137] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305176] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=17][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305208] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305225] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305241] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305261] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305284] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305305] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305327] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305348] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305371] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305393] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305415] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305438] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305461] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305483] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305510] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305715] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305729] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305740] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305751] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.305761] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ABF-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.311937] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867310885000, start_id=1729061867310885000, end_id=1729061867310885001) [2024-10-16 14:57:47.316819] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.316857] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=19][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.316883] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.316906] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.316942] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.316967] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317000] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317062] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317082] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317180] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317196] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317211] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317225] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317240] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317255] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317272] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317288] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317304] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317320] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317381] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317596] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317620] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=16][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317652] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.317674] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC4-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.318582] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:47.319757] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:171) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] Eliminate task evict sql audit(request_manager_->get_tenant_id()=1003, queue_size=100000, config_mem_limit_=32212254, request_manager_->get_size_used()=7474, evict_high_size_level=90000, evict_low_size_level=80000, allocator->allocated()=14558208, evict_high_mem_level=16106127, evict_low_mem_level=0) [2024-10-16 14:57:47.319771] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7474, mem_used=14558208) [2024-10-16 14:57:47.321226] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=0][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.321347] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=85][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.321358] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=7][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.321367] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.321394] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.321415] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.321425] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.321434] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.321695] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.321714] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC5-0-0] [lt=12][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.324940] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.324973] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=14][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.324995] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325014] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325045] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325066] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325097] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325118] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325323] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325343] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=12][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325359] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325374] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325390] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325405] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325421] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325438] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325454] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325475] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325562] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325878] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325895] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=10][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.325930] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AC6-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.329846] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=10] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:47.333607] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257776, offset=39036983}, end_cursor:ObLogCursor{file_id=197, log_id=49257777, offset=39037265}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db739e050, len:1993, log_data_len:282, seq:49257776, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:47.335227] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061867334336000, start_id=1729061867334336000, end_id=1729061867334336001) [2024-10-16 14:57:47.335704] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=8] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:47.336046] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=26] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257777, offset=39037265}, end_cursor:ObLogCursor{file_id=197, log_id=49257778, offset=39037547}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d4c050, len:1711, log_data_len:282, seq:49257777, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:47.337672] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650269, miss_cnt:32, cache_read_size:8050424138, hit_ratio:"9.999994892283119441e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:47.337699] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657007, miss_cnt:17, cache_read_size:8050171519, hit_ratio:"9.999997286816558750e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:47.338961] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=37] success to update min reserved snapshot(reserved_snapshot=1729060058725452000, duration=1800, snapshot_gc_ts=1729061858725452000) [2024-10-16 14:57:47.339623] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=9] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060058725452000) [2024-10-16 14:57:47.339635] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=10] update reserved snapshot finished(cost_ts=675, reserved_snapshot=1729060058725452000) [2024-10-16 14:57:47.340063] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ACC-0-0] [lt=1][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.340199] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ACC-0-0] [lt=32][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.340309] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ACC-0-0] [lt=24][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.342769] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ACD-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.342788] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ACD-0-0] [lt=14][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.342797] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ACD-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.342806] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997ACD-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.346407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867345327000, start_id=1729061867345327000, end_id=1729061867345327001) [2024-10-16 14:57:47.347565] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061867346607000, start_id=1729061867346607000, end_id=1729061867346607001) [2024-10-16 14:57:47.348516] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1729061867143651000, v:0}, new_max_version={val:1729061867143651000, v:0}, last_min_version={val:1729061866942552000, v:0}, last_max_version={val:1729061866942552000, v:0}, total_time_us=2824, delta_us=204864) [2024-10-16 14:57:47.348545] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1729061867143651000, v:0}, min_version={val:1729061867143651000, v:0}, max_version={val:1729061867143651000, v:0}, in_service=true, leader_epoch=33576) [2024-10-16 14:57:47.351671] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AD5-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.351693] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E42-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:47.351703] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E42-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20355, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:47.351720] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AD5-0-0] [lt=14][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.352735] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=9] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:47.352762] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:47.352769] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:47.352772] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:47.352775] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:47.352778] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17116) [2024-10-16 14:57:47.352932] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=268, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:47.352945] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=290, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:47.352974] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:47.357633] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.367373] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77E1-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:47.367385] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77E1-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10322, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:47.370705] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB0-0-0] [lt=11] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:47.370719] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB0-0-0] [lt=10] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=15) [2024-10-16 14:57:47.373672] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:47.373719] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:47.373729] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:47.373778] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.377581] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CC1-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:47.377611] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CC1-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:47.377616] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CC1-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27777, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:47.377620] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CC1-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=540, avg_log_submit_to_slide_time=27770) [2024-10-16 14:57:47.377646] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:47.377904] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AEA-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"a", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.393073] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AF7-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.394770] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:47.397643] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:47.400250] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AFD-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.400293] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997AFD-0-0] [lt=15][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.403020] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:47.403035] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:47.403061] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:47.403066] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:47.403318] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:47.416876] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:47.425336] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B15-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.425405] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B15-0-0] [lt=41][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.425460] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B15-0-0] [lt=18][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.425493] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B15-0-0] [lt=16][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.429835] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B18-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.429867] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B18-0-0] [lt=14][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.429915] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B18-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.429940] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B18-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.429989] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B18-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.430008] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B18-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.430103] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B18-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.431047] WDIAG [SQL.RESV] check_table_column_namespace (ob_column_namespace_checker.cpp:124) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=0][errcode=-5207] column in all tables is ambiguous(q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:47.431083] WDIAG [SQL.RESV] resolve_table_column_expr (ob_dml_resolver.cpp:4587) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=35][errcode=-5207] column not found in table(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:47.431116] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=24][errcode=-5207] resolve table column expr failed(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6c2c860 0x688c2c2 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.431509] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=20][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.431522] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.431543] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.431552] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.431560] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.431584] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.431597] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.431614] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1002][YB4A0A000407-00061E9B3B997B19-0-0] [lt=10][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.434694] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:47.434706] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:47.437812] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:47.451832] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E44-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=656, avg_log_submit_to_slide_time=33639) [2024-10-16 14:57:47.451845] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E44-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173207, log_task={header:{begin_lsn:{lsn:14352381420}, end_lsn:{lsn:14352381542}, log_id:115173207, min_scn:{val:1729061867147581002, v:0}, max_scn:{val:1729061867147581002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352381298}, prev_proposal_id:17, committed_end_lsn:{lsn:14352381420}, data_checksum:940009244, accum_checksum:4161863344, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061867351676, freeze_ts:1729061867351676, submit_ts:1729061867351680, flushed_ts:1729061867352955, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1275}, fs_cb_begin_ts=1729061867451825, log_life_time=100149) [2024-10-16 14:57:47.451888] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:47.452678] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=20, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:47.452699] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=4213, avg_group_log_size=351) [2024-10-16 14:57:47.452708] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:47.457653] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.467409] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77E3-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938326, log_task={header:{begin_lsn:{lsn:213058245340}, end_lsn:{lsn:213058245462}, log_id:220938326, min_scn:{val:1729061867148646002, v:0}, max_scn:{val:1729061867148646002, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058245218}, prev_proposal_id:88, committed_end_lsn:{lsn:213058245340}, data_checksum:1715074492, accum_checksum:1659036876, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061867367349, freeze_ts:1729061867367353, submit_ts:1729061867367359, flushed_ts:1729061867373943, gen_to_freeze cost time:4, gen_to_submit cost time:10, submit_to_flush cost time:6584}, fs_cb_begin_ts=1729061867467403, log_life_time=100054) [2024-10-16 14:57:47.467703] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=294, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:47.467718] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=650, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:47.476159] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A85-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:47.476176] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A85-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:47.477652] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CC3-0-0] [lt=16][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178514, log_task={header:{begin_lsn:{lsn:14538495175}, end_lsn:{lsn:14538495297}, log_id:115178514, min_scn:{val:1729061867148646002, v:0}, max_scn:{val:1729061867148646002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538495053}, prev_proposal_id:23, committed_end_lsn:{lsn:14538495175}, data_checksum:1715074492, accum_checksum:179878790, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061867377576, freeze_ts:1729061867377576, submit_ts:1729061867377594, flushed_ts:1729061867379818, gen_to_freeze cost time:0, gen_to_submit cost time:18, submit_to_flush cost time:2224}, fs_cb_begin_ts=1729061867477646, log_life_time=100070) [2024-10-16 14:57:47.477909] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=43869505, this={dio_aligned_buf:{buf_write_offset:1467, buf_padding_size:2629, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35369134, truncate_used_ts:14641088}, log_block_size:67108864, dir_fd:201, io_fd:197}, aligned_buf_len=4096, aligned_block_offset=43868160, buf_len=122, total_write_size_=12294, total_write_size_after_dio_=425984, ob_pwrite_used_ts=32801, count=101, each_pwrite_cost=324) [2024-10-16 14:57:47.478620] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.489703] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997B3A-0-0] [lt=0] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1003, table_id:10002, add_timestamp:1729061867489691}) [2024-10-16 14:57:47.489730] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E46-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1003, table_id=10002, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061867489716, sql_renew_time:1729061867489716, is_mark_fail:false}]) [2024-10-16 14:57:47.489749] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E46-0-0] [lt=19] success to process renew task(task={tenant_id:1003, table_id:10002, add_timestamp:1729061867489691}, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061867489716, sql_renew_time:1729061867489716, is_mark_fail:false}]) [2024-10-16 14:57:47.490297] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:47.490307] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=7] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:47.490319] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=70738456, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:47.490578] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:47.490585] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=7] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:47.490606] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:47.490611] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:47.490614] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:47.490617] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:47.490619] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:47.490622] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:47.490624] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:47.490627] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:47.490629] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:47.490652] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A25-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:47.490661] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A25-0-0] [lt=7] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:47.493003] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4393-0-0] [lt=31] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:47.493015] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4393-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=29761, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:47.494513] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061867493194000, start_id=1729061867493194000, end_id=1729061867493194001) [2024-10-16 14:57:47.498615] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480F-0-0] [lt=9] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061867465262, keepalive_ts:-1}) [2024-10-16 14:57:47.498631] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C480F-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061867465262, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:47.498769] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997B3F-0-0] [lt=1][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.498792] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997B3F-0-0] [lt=18][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.512156] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061867510643000, start_id=1729061867510643000, end_id=1729061867510643001) [2024-10-16 14:57:47.526956] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=29] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a837f0, tx_id:{txid:199664291}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061867522663, active_ts:1729061867522663, commit_ts:-1, finish_ts:1729061867522663, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1729061897522657, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061867513481, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:47.527803] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=36] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a85790, tx_id:{txid:199664292}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061867522663, active_ts:1729061867522663, commit_ts:-1, finish_ts:1729061867522663, timeout_us:29999996, lock_timeout_us:-1, expire_ts:1729061897522659, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061867513481, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:47.531133] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=20] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:47.534243] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=552, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:47.534260] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:47.534276] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11431) [2024-10-16 14:57:47.534284] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:47.534714] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:47.534728] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:47.535663] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061867534882000, start_id=1729061867534882000, end_id=1729061867534882001) [2024-10-16 14:57:47.535758] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:57:47.535779] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:47.536092] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4394-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=26, avg_log_submit_to_flush_time=1878, avg_log_submit_to_slide_time=3114) [2024-10-16 14:57:47.536880] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4395-0-0] [lt=29] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:47.536889] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4395-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3770, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:47.548142] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867547014000, start_id=1729061867547014000, end_id=1729061867547014001) [2024-10-16 14:57:47.548982] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061867548282000, start_id=1729061867548282000, end_id=1729061867548282001) [2024-10-16 14:57:47.550034] INFO [LIB] stat (utility.h:1165) [107496][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD19C-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:47.550044] INFO [LIB] stat (utility.h:1165) [107496][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD19C-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=1883, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:47.550049] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107496][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD19C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=1081, avg_log_submit_to_slide_time=1856) [2024-10-16 14:57:47.552476] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] worker thread began to shrink(tenant_->id()=1003, group_id_=1, token=0) [2024-10-16 14:57:47.553636] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061867552634000, start_id=1729061867552634000, end_id=1729061867552634001) [2024-10-16 14:57:47.553650] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] ts response task statistics(alloc_count=261364670, free_count=130682334) [2024-10-16 14:57:47.557714] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.560131] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107496][T1003_L0_G1][T1003][Y0-0000000000000000-0-0] [lt=7] worker thread exit(tenant_->id()=1003, workers_.get_size()=2) [2024-10-16 14:57:47.560145] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107496][T1003_L0_G1][T0][Y0-0000000000000000-0-0] [lt=10] tenant thread end_run(id_=1003, ret=0, thread_count_=172) [2024-10-16 14:57:47.562771] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=18] LogLoopThread round_cost_time(us)(round_cost_time=18) [2024-10-16 14:57:47.568865] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.569643] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.573625] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.574709] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.578092] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCAFE-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061867577266000, start_id=1729061867577266000, end_id=1729061867577266001) [2024-10-16 14:57:47.578867] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.594628] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1003}, {id:1}]) [2024-10-16 14:57:47.596155] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:47.596170] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:47.604204] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF1F-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:47.604237] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF1F-0-0] [lt=33] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=860, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:47.604242] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF1F-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=12, avg_log_submit_to_flush_time=450, avg_log_submit_to_slide_time=848) [2024-10-16 14:57:47.608580] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0F-0-0] [lt=9] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061867599784, keepalive_ts:-1}) [2024-10-16 14:57:47.608594] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A0F-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061867599784, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:47.609767] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5136, mem_used=10398720) [2024-10-16 14:57:47.609783] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997BA0-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.609846] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997BA0-0-0] [lt=37][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"PACKAGE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.610134] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997BA0-0-0] [lt=18][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.610182] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997BA0-0-0] [lt=24][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"ROUTINE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.610694] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997BA0-0-0] [lt=17][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.610743] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997BA0-0-0] [lt=26][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TRIGGER_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.617160] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:47.621361] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C9-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:47.621380] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9C9-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061867147581004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061867147581004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:47.622797] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9C9-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061867622617000, start_id=1729061867622617000, end_id=1729061867622617001) [2024-10-16 14:57:47.626212] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C9-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061867625895000, start_id=1729061867625895000, end_id=1729061867625895001) [2024-10-16 14:57:47.626295] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C9-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=11554, avg_group_log_size=608) [2024-10-16 14:57:47.626305] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9C9-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11553) [2024-10-16 14:57:47.626643] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11554) [2024-10-16 14:57:47.626698] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997BA7-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"DATA_TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.626938] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997BA7-0-0] [lt=49][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.627085] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:47.648982] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA07-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061867648246000, start_id=1729061867648246000, end_id=1729061867648246001) [2024-10-16 14:57:47.649123] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA06-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061867648246000, start_id=1729061867648246000, end_id=1729061867648246001) [2024-10-16 14:57:47.649150] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA0A-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061867648246000, start_id=1729061867648246000, end_id=1729061867648246001) [2024-10-16 14:57:47.651069] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=15] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:57:47.652590] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:47.657731] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.665644] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:57:47.665658] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=114, cache_node_num=114) [2024-10-16 14:57:47.665665] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=114, cache_node_num=114) [2024-10-16 14:57:47.665670] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:47.667797] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=114, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:787547, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787548, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787549, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:173472}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:57:47.667945] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=145] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:47.671623] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:47.675502] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:57:47.676304] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061867675074000, start_id=1729061867675074000, end_id=1729061867675074001) [2024-10-16 14:57:47.676803] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C56-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:47.676824] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C56-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061867577266000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061867577266000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:47.679167] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.680704] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C56-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061867680462000, start_id=1729061867680462000, end_id=1729061867680462001) [2024-10-16 14:57:47.695297] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061867694233000, start_id=1729061867694233000, end_id=1729061867694233001) [2024-10-16 14:57:47.695360] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3504) [2024-10-16 14:57:47.695940] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4091) [2024-10-16 14:57:47.695966] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4091) [2024-10-16 14:57:47.696514] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C27-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:47.696524] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C27-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1351, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:47.699049] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:47.712126] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061867710691000, start_id=1729061867710691000, end_id=1729061867710691001) [2024-10-16 14:57:47.718183] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=12] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:47.726630] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=7] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:47.735185] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=447, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:47.735201] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=59, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:47.735215] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:47.736144] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061867735063000, start_id=1729061867735063000, end_id=1729061867735063001) [2024-10-16 14:57:47.736324] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:47.736694] WDIAG [OCCAM] dump_statistics (ob_vtable_event_recycle_buffer.h:144) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15E3-0-0] [lt=11][errcode=-4006] not init(ret=-4006) [2024-10-16 14:57:47.736707] INFO [MDS] dump_map_holding_item (mds_tenant_service.cpp:473) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15E3-0-0] [lt=6] finish scan map holding items(scan_cnt=0) [2024-10-16 14:57:47.736812] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15E4-0-0] [lt=4] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:47.737897] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=11] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:47.748266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061867746721000, start_id=1729061867746721000, end_id=1729061867746721001) [2024-10-16 14:57:47.749713] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061867748446000, start_id=1729061867748446000, end_id=1729061867748446001) [2024-10-16 14:57:47.751852] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E4A-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:47.752224] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B919D-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:47.757837] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:47.757861] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=12] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:47.757869] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:47.757882] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.760152] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C03-0-0] [lt=0][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.760199] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C03-0-0] [lt=18][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.760211] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C03-0-0] [lt=8][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.760218] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C03-0-0] [lt=3][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.760236] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C03-0-0] [lt=3][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.760256] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C03-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.760266] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C03-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.760275] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C03-0-0] [lt=5][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:47.772054] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:47.776915] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C0A-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.777054] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C0A-0-0] [lt=35][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.777159] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C0A-0-0] [lt=21][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.779233] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.780029] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:47.780043] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:47.780051] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=52) [2024-10-16 14:57:47.781941] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=13] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:47.788627] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=3] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789616] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789638] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789644] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789649] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789661] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789667] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789673] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789679] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=3] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789686] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.789697] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:47.793580] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8501B-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497476, log_task={header:{begin_lsn:{lsn:207589554758}, end_lsn:{lsn:207589554880}, log_id:218497476, min_scn:{val:1729061867648246000, v:0}, max_scn:{val:1729061867648246000, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589554636}, prev_proposal_id:81, committed_end_lsn:{lsn:207589554758}, data_checksum:940009244, accum_checksum:3861062825, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061867692662, freeze_ts:1729061867692663, submit_ts:1729061867692668, flushed_ts:1729061867693285, gen_to_freeze cost time:1, gen_to_submit cost time:6, submit_to_flush cost time:617}, fs_cb_begin_ts=1729061867793574, log_life_time=100912) [2024-10-16 14:57:47.811498] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:207589554880}, scn={val:1729061867648246001, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061867793576, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589554880}, next_to_submit_scn_:{val:1729061867648246001, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589554758}, end_lsn:{lsn:207589554880}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061867648246000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061867648246000, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:3861062825, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061867648246000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598923, miss_cnt:9, cache_read_size:149777919321, hit_ratio:"9.999999345925155714e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:57:47.813591] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C28-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"a", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.817430] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:47.828746] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:47.828764] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:47.828780] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52482782, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:47.830712] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1002}, {id:1001}]) [2024-10-16 14:57:47.839466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061867838445000, start_id=1729061867838445000, end_id=1729061867838445001) [2024-10-16 14:57:47.843292] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=20] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:57:47.846594] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=5] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:47.847186] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=23] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:47.847230] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061864434291000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:47.859597] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.859640] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=5214, mem_used=10398720) [2024-10-16 14:57:47.860639] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22136][T1002_TntTransf][T1002][YB4A0A000407-00061E9AB24CC6BD-0-0] [lt=46] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:57:47.867505] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77E9-0-0] [lt=38] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1817) [2024-10-16 14:57:47.867731] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:47.869101] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C53-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.869168] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C53-0-0] [lt=32][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.869223] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C53-0-0] [lt=17][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.869253] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C53-0-0] [lt=13][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.870737] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:47.875213] WDIAG [SQL.RESV] check_table_column_namespace (ob_column_namespace_checker.cpp:124) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C57-0-0] [lt=0][errcode=-5207] column in all tables is ambiguous(q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:47.875270] WDIAG [SQL.RESV] resolve_table_column_expr (ob_dml_resolver.cpp:4587) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C57-0-0] [lt=56][errcode=-5207] column not found in table(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:47.875301] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1003][YB4A0A000407-00061E9B3B997C57-0-0] [lt=21][errcode=-5207] resolve table column expr failed(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6c2c860 0x688c2c2 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:47.878571] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CCB-0-0] [lt=19] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:47.878683] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107502][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=192) [2024-10-16 14:57:47.878684] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=11] worker thread created(tenant_->id()=1004, group_id_=1, token=1) [2024-10-16 14:57:47.878703] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107502][][T1004][Y0-0000000000000000-0-0] [lt=18] Init thread local success [2024-10-16 14:57:47.879305] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.879358] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:47.879366] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:47.881417] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D73-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:47.881438] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D73-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061867649318002, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061867649318002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:47.882485] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D73-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061867881470000, start_id=1729061867881470000, end_id=1729061867881470001) [2024-10-16 14:57:47.884862] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D73-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061867884622000, start_id=1729061867884622000, end_id=1729061867884622001) [2024-10-16 14:57:47.884920] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D73-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:47.884933] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D73-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:47.893998] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=41] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:47.894582] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=20] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:47.894640] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=17] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061861837115000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:47.895619] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061867890971000, start_id=1729061867890971000, end_id=1729061867890971001) [2024-10-16 14:57:47.896179] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8501D-0-0] [lt=11][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497477, log_task={header:{begin_lsn:{lsn:207589554880}, end_lsn:{lsn:207589555002}, log_id:218497477, min_scn:{val:1729061867648246001, v:0}, max_scn:{val:1729061867648246001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589554758}, prev_proposal_id:81, committed_end_lsn:{lsn:207589554880}, data_checksum:940009244, accum_checksum:2738152067, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061867793563, freeze_ts:1729061867793563, submit_ts:1729061867793568, flushed_ts:1729061867793986, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:418}, fs_cb_begin_ts=1729061867896174, log_life_time=102611) [2024-10-16 14:57:47.901781] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=10] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:47.906074] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=18] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:47.906182] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:47.906193] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:47.906646] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=245, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:47.906661] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=222, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:47.906678] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:47.906698] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:47.909033] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2CB-0-0] [lt=5] init point thread id with(&point=0x55e3435caa00, idx_=1006, point=[thread id=107502, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107502) [2024-10-16 14:57:47.909080] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2CB-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:47.909086] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2CB-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1157, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:47.912268] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=48] get number(ret=0, service_type_=0, range=1, base_id=1729061867911402000, start_id=1729061867911402000, end_id=1729061867911402001) [2024-10-16 14:57:47.912889] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:207589555180}, scn={val:1729061867648246003, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:47.912907] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:31, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061867912845, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589555236}, next_to_submit_scn_:{val:1729061867648246004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589555124}, end_lsn:{lsn:207589555236}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061867648246003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061867648246003, v:0}, data_checksum:4162536794, flag:1}}, init_mode_version:0, accumulate_checksum:887271622, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061867648246003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598926, miss_cnt:9, cache_read_size:149777919677, hit_ratio:"9.999999345925170147e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589555180}, scn_:{val:1729061867648246003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061867878885, first_handle_ts_:1729061867912907, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f38a73f60, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:47.913076] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=65] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060060198021000) [2024-10-16 14:57:47.913083] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589555180}, scn_:{val:1729061867648246003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061867878885, first_handle_ts_:1729061867913057, replay_cost_:26, retry_cost_:-1, read_log_buf_:0x2b0f38a73f70, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061867912845, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589555236}, next_to_submit_scn_:{val:1729061867648246004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589555124}, end_lsn:{lsn:207589555236}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061867648246003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061867648246003, v:0}, data_checksum:4162536794, flag:1}}, init_mode_version:0, accumulate_checksum:887271622, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061867648246003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598926, miss_cnt:9, cache_read_size:149777919677, hit_ratio:"9.999999345925170147e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_queue_idx=5, ret=0) [2024-10-16 14:57:47.923707] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:47.933336] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997C78-0-0] [lt=0] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1004, table_id:10002, add_timestamp:1729061867933324}) [2024-10-16 14:57:47.933366] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E47-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1004, table_id=10002, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061867933352, sql_renew_time:1729061867933352, is_mark_fail:false}]) [2024-10-16 14:57:47.933384] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E47-0-0] [lt=17] success to process renew task(task={tenant_id:1004, table_id:10002, add_timestamp:1729061867933324}, locations=[{table_id:10002, partition_id:10002, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061867933352, sql_renew_time:1729061867933352, is_mark_fail:false}]) [2024-10-16 14:57:47.936399] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=9] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:47.939271] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:47.939622] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41BE-0-0] [lt=22] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:47.939632] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41BE-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1418, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:47.940019] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:47.943451] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=23] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:47.943505] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3431, mem_used=8318976) [2024-10-16 14:57:47.948700] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061867946952000, start_id=1729061867946952000, end_id=1729061867946952001) [2024-10-16 14:57:47.949816] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22455][T1003_TenantWea][T1003][YB4A0A000407-00061E9B3ABCFD12-0-0] [lt=11] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.103558874130249023e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.008835578279190031e+03) [2024-10-16 14:57:47.949971] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061867948875000, start_id=1729061867948875000, end_id=1729061867948875001) [2024-10-16 14:57:47.950906] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997C7F-0-0] [lt=0][errcode=-5171] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:57:47.951716] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:57:47.951729] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=10] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:47.951735] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:47.951740] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:47.953949] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:57:47.954302] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=352] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:47.954529] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E4E-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:47.954627] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=13] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:47.957725] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:47.957740] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:47.957748] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=160) [2024-10-16 14:57:47.962458] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=22] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:47.962787] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.964246] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061863938597, last_first_register_time_us=1722494150998907, register_time_us=1729061863938597, curr_time_us=1729061867964244) [2024-10-16 14:57:47.964278] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=14] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061867964260, last_submit_register_req_time_us=1729061867964260, register_parent_reason=3) [2024-10-16 14:57:47.964285] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=6] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:47.967345] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=16] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:47.971707] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1817) [2024-10-16 14:57:47.979370] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:47.979388] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:47.986151] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:47.986871] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:47.986893] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=22] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:47.986911] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:47.986972] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061879029444000, latest_log_ts:{val:1729061862417342001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:47.987014] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=40] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:47.987030] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=12] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:57:47.987037] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061862417342001, v:0}) [2024-10-16 14:57:47.987041] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:47.987045] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:47.987051] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:47.987060] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:57:47.987067] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:47.987072] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:47.992985] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:48.006804] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=223, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:48.006824] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=231, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:48.006970] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2CD-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=507, avg_log_submit_to_slide_time=1092) [2024-10-16 14:57:48.011120] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=12] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:48.012343] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO [ratelimit] time: 1729061868012342, bytes: 4446816623563, bw: 0.354461 MB/s, add_ts: 1000848, add_bytes: 371994 [2024-10-16 14:57:48.015203] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 6887.75; ) [2024-10-16 14:57:48.015220] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:48.015259] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 7479.00; ) [2024-10-16 14:57:48.015264] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:48.015273] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:48.018232] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:48.039180] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:48.039193] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:48.039221] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:48.039231] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:48.040319] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1371, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:48.040343] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=402, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:48.040371] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=12, avg_log_submit_to_flush_time=2029, avg_log_submit_to_slide_time=2915) [2024-10-16 14:57:48.040384] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:48.040424] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=2] [APPLY STAT CB APPEND COST TIME](cur_stat_count=11, stat_interval=5000000, avg (count/cost)=18, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:57:48.040431] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=8] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=11, stat_interval=5000000, avg (count/cost)=1040, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:57:48.040436] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=11, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:57:48.040441] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB EXECUTE TIME](cur_stat_count=11, stat_interval=5000000, avg (count/cost)=-545, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:57:48.040446] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB TOTAL TIME](cur_stat_count=11, stat_interval=5000000, avg (count/cost)=513, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:57:48.042377] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061868041614000, start_id=1729061868041614000, end_id=1729061868041614001) [2024-10-16 14:57:48.043660] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=14][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5171, dropped:22334, tid:21226}]) [2024-10-16 14:57:48.043954] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=1][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$ENCRYPTED_TABLESPACES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.043977] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=21][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$ENCRYPTED_TABLESPACES' [2024-10-16 14:57:48.043986] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$ENCRYPTED_TABLESPACES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.043994] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$ENCRYPTED_TABLESPACES' [2024-10-16 14:57:48.044014] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$ENCRYPTED_TABLESPACES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.044023] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$ENCRYPTED_TABLESPACES' [2024-10-16 14:57:48.044027] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$ENCRYPTED_TABLESPACES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.044035] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$ENCRYPTED_TABLESPACES' [2024-10-16 14:57:48.044039] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$ENCRYPTED_TABLESPACES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.044047] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$ENCRYPTED_TABLESPACES' [2024-10-16 14:57:48.044052] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$ENCRYPTED_TABLESPACES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.044056] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$ENCRYPTED_TABLESPACES' [2024-10-16 14:57:48.044061] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$ENCRYPTED_TABLESPACES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.044068] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$ENCRYPTED_TABLESPACES' [2024-10-16 14:57:48.044092] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$ENCRYPTED_TABLESPACES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.044100] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$ENCRYPTED_TABLESPACES' [2024-10-16 14:57:48.044104] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$ENCRYPTED_TABLESPACES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.044108] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCA-0-0] [lt=3][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$ENCRYPTED_TABLESPACES' [2024-10-16 14:57:48.044986] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCB-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"CONNECTION_CONTROL_FAILED_LOGIN_ATTEMPTS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.045006] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCB-0-0] [lt=19][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'CONNECTION_CONTROL_FAILED_LOGIN_ATTEMPTS' [2024-10-16 14:57:48.045014] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCB-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"CONNECTION_CONTROL_FAILED_LOGIN_ATTEMPTS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.045020] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCB-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'CONNECTION_CONTROL_FAILED_LOGIN_ATTEMPTS' [2024-10-16 14:57:48.045789] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.045806] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=15][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_TENANT_MEMORY' [2024-10-16 14:57:48.045814] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.045823] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_TENANT_MEMORY' [2024-10-16 14:57:48.045827] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.045835] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_TENANT_MEMORY' [2024-10-16 14:57:48.045839] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.045844] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_TENANT_MEMORY' [2024-10-16 14:57:48.045853] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.045860] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCC-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_TENANT_MEMORY' [2024-10-16 14:57:48.046721] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.046736] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=13][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_TENANT_MEMORY' [2024-10-16 14:57:48.046745] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.046750] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_TENANT_MEMORY' [2024-10-16 14:57:48.046755] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.046759] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_TENANT_MEMORY' [2024-10-16 14:57:48.046763] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.046767] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_TENANT_MEMORY' [2024-10-16 14:57:48.046771] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_TENANT_MEMORY", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.046775] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCD-0-0] [lt=3][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_TENANT_MEMORY' [2024-10-16 14:57:48.047591] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047611] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=19][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047619] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047629] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=9][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047634] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047641] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047650] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047658] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047662] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047681] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=19][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047686] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047694] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047698] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047705] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047719] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=13][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047727] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047731] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047738] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047742] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047749] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.047752] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"GV$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.047760] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCE-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'GV$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048735] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048752] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=16][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048760] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048769] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048774] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048781] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048785] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048793] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048797] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048804] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048808] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048816] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048820] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048827] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048831] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048838] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048842] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048849] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048853] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048860] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.048864] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"V$OB_PX_TARGET_MONITOR", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.048871] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CCF-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'V$OB_PX_TARGET_MONITOR' [2024-10-16 14:57:48.050276] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLUMN_PRIVILEGES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.050291] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=14][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLUMN_PRIVILEGES' [2024-10-16 14:57:48.050301] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLUMN_PRIVILEGES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.050306] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLUMN_PRIVILEGES' [2024-10-16 14:57:48.050313] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLUMN_PRIVILEGES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.050318] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLUMN_PRIVILEGES' [2024-10-16 14:57:48.050323] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLUMN_PRIVILEGES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.050328] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLUMN_PRIVILEGES' [2024-10-16 14:57:48.050333] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLUMN_PRIVILEGES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.050338] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLUMN_PRIVILEGES' [2024-10-16 14:57:48.050371] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLUMN_PRIVILEGES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.050376] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLUMN_PRIVILEGES' [2024-10-16 14:57:48.050386] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"COLUMN_PRIVILEGES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.050391] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD0-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'COLUMN_PRIVILEGES' [2024-10-16 14:57:48.050733] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:48.050783] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1002 and flag like "%DUPLICATE%", tenant_id=1002, exec_tenant_id=1001, need_member_list=false) [2024-10-16 14:57:48.051858] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=0][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"VIEW_TABLE_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.051873] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=13][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'VIEW_TABLE_USAGE' [2024-10-16 14:57:48.051881] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"VIEW_TABLE_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.051886] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=5][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'VIEW_TABLE_USAGE' [2024-10-16 14:57:48.051891] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"VIEW_TABLE_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.051895] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'VIEW_TABLE_USAGE' [2024-10-16 14:57:48.051899] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"VIEW_TABLE_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.051904] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'VIEW_TABLE_USAGE' [2024-10-16 14:57:48.051907] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"VIEW_TABLE_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.051912] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'VIEW_TABLE_USAGE' [2024-10-16 14:57:48.051918] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"VIEW_TABLE_USAGE", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.051922] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD1-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'VIEW_TABLE_USAGE' [2024-10-16 14:57:48.053114] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053135] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=19][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053144] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053152] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053157] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053165] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053169] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053176] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053180] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053188] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053192] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053196] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053200] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053207] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053211] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053218] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053223] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053230] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053234] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053241] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053245] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053253] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053257] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053264] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053268] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053275] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053279] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053286] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053290] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053298] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053302] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053309] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053313] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053320] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053324] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053331] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053335] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053343] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053347] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053354] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053358] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053365] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053369] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053376] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053380] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053388] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053391] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053399] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053402] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053410] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053414] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053421] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053425] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053432] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053436] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053443] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053447] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053454] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053458] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053466] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053469] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053477] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053481] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053488] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053492] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053499] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053503] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053510] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053514] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053521] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053525] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053529] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=4][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053537] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=5][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053544] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053548] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"information_schema", table:"FILES", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.053555] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD2-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'FILES' [2024-10-16 14:57:48.053979] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061868052931000, start_id=1729061868052931000, end_id=1729061868052931001) [2024-10-16 14:57:48.053996] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=561, get_gts_with_stc_cnt=32, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=30, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:48.054004] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:48.054016] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=544, get_gts_with_stc_cnt=8, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:48.054023] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=4] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061868054020], gts:1729061868052931000, latest_srr:[mts=1729061868054020]}) [2024-10-16 14:57:48.055459] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=0][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"DBA_OB_TENANTS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.055479] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=18][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'DBA_OB_TENANTS' [2024-10-16 14:57:48.055488] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=4][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"DBA_OB_TENANTS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.055496] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=8][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'DBA_OB_TENANTS' [2024-10-16 14:57:48.055516] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"DBA_OB_TENANTS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.055524] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'DBA_OB_TENANTS' [2024-10-16 14:57:48.055529] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"DBA_OB_TENANTS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.055536] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'DBA_OB_TENANTS' [2024-10-16 14:57:48.055540] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=3][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"DBA_OB_TENANTS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.055547] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=7][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'DBA_OB_TENANTS' [2024-10-16 14:57:48.055551] WDIAG [SHARE.SCHEMA] check_single_table_priv (ob_schema_getter_guard.cpp:3849) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=2][errcode=-5171] No privilege(tenant_id=1, user_id=200001, table_need_priv={db:"oceanbase", table:"DBA_OB_TENANTS", columns:[], priv_set:4194304, priv_level:3, is_sys_table:false, is_for_update:false, priv_check_type:0}, table_priv_set | user_db_priv_set=4393625911294) [2024-10-16 14:57:48.055559] WDIAG check_single_table_priv (ob_schema_getter_guard.cpp:3904) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CD3-0-0] [lt=6][errcode=-5171] REFERENCES command denied to user 'root'@'%' for table 'DBA_OB_TENANTS' [2024-10-16 14:57:48.062969] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.064271] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=17) [2024-10-16 14:57:48.067745] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CDE-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.067814] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CDE-0-0] [lt=39][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"P", column:"PACKAGE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.068122] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CDE-0-0] [lt=16][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.068171] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CDE-0-0] [lt=24][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"R", column:"ROUTINE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x6bb441d 0x6bb441d 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.068679] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CDE-0-0] [lt=17][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.068728] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CDE-0-0] [lt=25][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"T", column:"TRIGGER_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6bb4b17 0x6bb4261 0x6bb44d2 0x6bb441d 0x115c9b1d 0x6886d2e 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.078648] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CCF-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178520, log_task={header:{begin_lsn:{lsn:14538495907}, end_lsn:{lsn:14538496029}, log_id:115178520, min_scn:{val:1729061867649318003, v:0}, max_scn:{val:1729061867649318003, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538495785}, prev_proposal_id:23, committed_end_lsn:{lsn:14538495907}, data_checksum:1715074492, accum_checksum:3068530880, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061867978570, freeze_ts:1729061867978570, submit_ts:1729061867978575, flushed_ts:1729061867979406, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:831}, fs_cb_begin_ts=1729061868078643, log_life_time=100073) [2024-10-16 14:57:48.079432] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.087236] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CE5-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"DATA_TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.087510] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997CE5-0-0] [lt=40][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"I", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x11488971 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.093305] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D7-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061868083586, keepalive_ts:-1}) [2024-10-16 14:57:48.093326] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D7-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061868083586, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:48.096141] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061868094611000, start_id=1729061868094611000, end_id=1729061868094611001) [2024-10-16 14:57:48.111052] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061868111051, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007613, add_bytes: 0 [2024-10-16 14:57:48.113401] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061868112049000, start_id=1729061868112049000, end_id=1729061868112049001) [2024-10-16 14:57:48.114658] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:48.114677] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:48.124050] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=14] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97915314176) [2024-10-16 14:57:48.127283] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:48.127298] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:48.136473] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:48.136691] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3206, mem_used=6239232) [2024-10-16 14:57:48.143787] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=17][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4002, dropped:262, tid:21226}]) [2024-10-16 14:57:48.145781] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:48.148759] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061868147631000, start_id=1729061868147631000, end_id=1729061868147631001) [2024-10-16 14:57:48.148814] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=21, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:48.148827] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:48.148944] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA0B-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061868148862000, start_id=1729061868148862000, end_id=1729061868148862001) [2024-10-16 14:57:48.148977] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA0F-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061868148862000, start_id=1729061868148862000, end_id=1729061868148862001) [2024-10-16 14:57:48.149018] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA0E-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061868148862000, start_id=1729061868148862000, end_id=1729061868148862001) [2024-10-16 14:57:48.149053] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA0D-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061868148862000, start_id=1729061868148862000, end_id=1729061868148862001) [2024-10-16 14:57:48.149120] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA0C-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061868148862000, start_id=1729061868148862000, end_id=1729061868148862001) [2024-10-16 14:57:48.149479] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F953F-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10152) [2024-10-16 14:57:48.154394] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=6] current monitor number(seq_=-1) [2024-10-16 14:57:48.155663] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061868149920000, start_id=1729061868149920000, end_id=1729061868149920001) [2024-10-16 14:57:48.157721] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=463, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:48.157737] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=576, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:48.161636] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:48.161647] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1906, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:48.161652] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=1566, avg_log_submit_to_slide_time=1876) [2024-10-16 14:57:48.161660] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:48.163013] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.164691] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109203143, max_lsn:17959706491, max_scn:{val:1729061868052931000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7385580775}, reuse_lsn:{lsn:17959706491}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17917763451}}, last_submit_lsn:{lsn:17959706369}, last_submit_end_lsn:{lsn:17959706491}, last_submit_log_id:109203143, last_submit_log_pid:246, max_flushed_lsn:{lsn:17959706369}, max_flushed_end_lsn:{lsn:17959706491}, max_flushed_log_pid:246, committed_end_lsn:{lsn:17959706491}, last_slide_log_id:109203143, last_slide_scn:{val:1729061868052931000, v:0}, last_slide_lsn:{lsn:17959706369}, last_slide_end_lsn:{lsn:17959706491}, last_slide_log_pid:246, last_slide_log_accum_checksum:252224885, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1d4530}, [StateMgr]={this:0x2b0e0f1d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109203144, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261839590, palf_role_change_cb:0x2b0e0f1df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, reconfig_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, persistent_config_version:{proposal_id:246, config_seq:121}, ms_ack_list:0, resend_config_version:{proposal_id:246, config_seq:121}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258131745, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258131745, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262039736, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728994126835081001, v:0}, min_block_id:265, min_block_min_scn:{val:1728952113212346001, v:0}, base_lsn_for_block_gc:{lsn:17849868288}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, log_config_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, log_snapshot_meta:{version:1, base_lsn:{lsn:17849868288}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1798144}, readable_log_tail:{lsn:1798144}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17984077824}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/meta", dir_fd:167, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65306624, block_header_serialize_buf_:0x2b0e0f1da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17959706491}, readable_log_tail:{lsn:17959706491}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17916973056}, min_scn:{val:1729035023717936004, v:0}, curr_block_id:267, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/log", dir_fd:169, min_block_id:265, max_block_id:268, curr_writable_block_id:267}, logical_block_size_:67104768, curr_block_writable_size_:24371333, block_header_serialize_buf_:0x2b0e0f1dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261838799, this:0x2b0e0f1d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:246, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:245, majority_max_lsn:{lsn:7385763025}, saved_end_lsn:{lsn:7385763025}, last_submit_prepare_req_time_us:1722494261839744, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1d89b0}) [2024-10-16 14:57:48.164842] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=147] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868142921, last_flushed_end_lsn:{lsn:17959706491}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868139967, last_flushed_end_lsn:{lsn:17959706491}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868139782, last_flushed_end_lsn:{lsn:17959706491}}]) [2024-10-16 14:57:48.164854] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=109203144, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868142921, last_flushed_end_lsn:{lsn:17959706491}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868139967, last_flushed_end_lsn:{lsn:17959706491}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868139782, last_flushed_end_lsn:{lsn:17959706491}}]) [2024-10-16 14:57:48.167669] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77EF-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1817, avg_group_log_size=181) [2024-10-16 14:57:48.172004] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172025] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=15][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172034] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172045] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172055] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172063] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172073] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172081] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172091] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172102] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172149] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172161] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172172] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172241] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.172304] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D1E-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.179496] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.183170] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183200] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=20][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183213] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183228] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183242] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183259] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183284] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183306] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183321] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183333] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183345] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183355] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183366] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183377] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183448] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183524] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183590] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183690] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.183701] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D22-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.191656] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5032-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:48.191676] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5032-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:48.192934] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85023-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=12, avg_log_batch_cnt=0, total_group_log_size=1454, avg_group_log_size=121) [2024-10-16 14:57:48.192954] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85023-0-0] [lt=18] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:48.193784] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=397, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:48.193799] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=137, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:48.193830] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:48.201948] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA886-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:48.201969] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA886-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061867648246005, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061867648246005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:48.205541] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA886-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061868205276000, start_id=1729061868205276000, end_id=1729061868205276001) [2024-10-16 14:57:48.205609] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA886-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10153, avg_group_log_size=597) [2024-10-16 14:57:48.205619] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA886-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10153) [2024-10-16 14:57:48.205793] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205812] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=13][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205840] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205848] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10153) [2024-10-16 14:57:48.205875] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205919] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=33][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205943] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205955] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205966] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205976] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205987] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.205995] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.206006] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.206021] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.206033] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.206043] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.206054] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D33-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.206199] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:48.215006] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:48.216156] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216196] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=18][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216228] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216245] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216262] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216282] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216305] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216326] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216347] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216369] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216392] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216414] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216436] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216458] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216479] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216502] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216529] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216730] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216744] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216755] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216766] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.216776] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D3B-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.218515] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:48.224820] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.224856] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=17][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.224944] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.224968] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225009] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225034] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225067] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225129] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225149] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225247] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225263] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225278] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225293] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225307] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225319] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225333] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225349] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225365] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225381] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225441] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225676] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225694] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=8][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225728] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.225749] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D40-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.229138] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=1][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:48.229178] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=18][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:48.229190] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=8][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:48.229199] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=7][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:48.229220] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:48.229241] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=6][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:48.229252] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=7][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:48.229258] WDIAG [COMMON] check_stack_overflow (ob_common_utility.cpp:98) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=4][errcode=0] stack possible overflow(cur_stack=0x2b0dceb20890, stack_eof=0x2b0dceb79c00, stack_start=0x2b0dceb02000, stack_size=490496, reserved_size=131072, cur_stack_used=365424) [2024-10-16 14:57:48.229486] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.229506] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D41-0-0] [lt=11][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232067] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232122] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=12][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232146] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232166] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x114887d9 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232197] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x114891d6 0x689e678 0x6887f1c 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232219] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232249] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232270] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232482] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232495] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232507] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232519] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232531] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232544] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232556] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232569] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232582] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232595] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232687] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232727] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D9C-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510336101122, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410609, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232983] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.232996] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.233027] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D42-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.238355] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:213058247091}, scn={val:1729061868148862002, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:48.238590] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060060198021000) [2024-10-16 14:57:48.238598] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=8] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:213058247091}, scn_:{val:1729061868148862002, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061868228576, first_handle_ts_:1729061868238579, replay_cost_:18, retry_cost_:-7369787846959263591, read_log_buf_:0x2b0f38a75c20, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061868238319, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058247147}, next_to_submit_scn_:{val:1729061868148862003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058247035}, end_lsn:{lsn:213058247147}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061868148862002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061868148862002, v:0}, data_checksum:122991068, flag:0}}, init_mode_version:0, accumulate_checksum:220620985, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061868148862002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561342, miss_cnt:1, cache_read_size:152590687129, hit_ratio:"9.999999928346920974e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}, replay_queue_idx=0, ret=0) [2024-10-16 14:57:48.239452] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:48.242521] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061868241603000, start_id=1729061868241603000, end_id=1729061868241603001) [2024-10-16 14:57:48.247093] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D48-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.247270] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D48-0-0] [lt=34][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.247384] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D48-0-0] [lt=21][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"trg", column:"package_exec_env", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.249925] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D49-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.249948] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D49-0-0] [lt=17][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.249957] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D49-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.249969] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D49-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.253576] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C109E-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061868253362000, start_id=1729061868253362000, end_id=1729061868253362001) [2024-10-16 14:57:48.258884] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D51-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.258933] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D51-0-0] [lt=14][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.263194] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.263465] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=427, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:48.263484] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=870, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:48.263503] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:48.263930] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C2C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=1483, avg_log_submit_to_slide_time=1762) [2024-10-16 14:57:48.278981] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CD3-0-0] [lt=25] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:48.279003] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CD3-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178522, log_task={header:{begin_lsn:{lsn:14538496151}, end_lsn:{lsn:14538496273}, log_id:115178522, min_scn:{val:1729061868148862001, v:0}, max_scn:{val:1729061868148862001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538496029}, prev_proposal_id:23, committed_end_lsn:{lsn:14538496151}, data_checksum:1715074492, accum_checksum:2124018985, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061868178641, freeze_ts:1729061868178641, submit_ts:1729061868178646, flushed_ts:1729061868179794, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:1148}, fs_cb_begin_ts=1729061868278994, log_life_time=100353) [2024-10-16 14:57:48.279045] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] succ to submit log task to replay service(to_submit_lsn={lsn:14538496207}, to_submit_scn={val:1729061868148862001, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061868278997, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538496273}, next_to_submit_scn_:{val:1729061868148862002, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538496151}, end_lsn:{lsn:14538496273}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061868148862001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061868148862001, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:2124018985, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061868148862001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650279, miss_cnt:32, cache_read_size:8050425358, hit_ratio:"9.999994892283934345e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:48.279274] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=248, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:48.279286] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=336, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:48.279307] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:48.279558] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.284788] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D66-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"a", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x6b22bf7 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.292932] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85025-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:48.292966] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85025-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497482, log_task={header:{begin_lsn:{lsn:207589555480}, end_lsn:{lsn:207589555602}, log_id:218497482, min_scn:{val:1729061868148862000, v:0}, max_scn:{val:1729061868148862000, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589555358}, prev_proposal_id:81, committed_end_lsn:{lsn:207589555480}, data_checksum:940009244, accum_checksum:1128840745, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061868192921, freeze_ts:1729061868192921, submit_ts:1729061868192928, flushed_ts:1729061868193813, gen_to_freeze cost time:0, gen_to_submit cost time:7, submit_to_flush cost time:885}, fs_cb_begin_ts=1729061868292952, log_life_time=100031) [2024-10-16 14:57:48.296253] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061868294829000, start_id=1729061868294829000, end_id=1729061868294829001) [2024-10-16 14:57:48.300286] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D73-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.307518] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:48.307564] WDIAG read_multi_row (ob_table_access_helper.h:238) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:48.307619] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D79-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.307659] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D79-0-0] [lt=10][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.308323] INFO delete_row (ob_table_access_helper.h:312) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1002, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061868307571) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061868307571) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:48.308389] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=10] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:48.308456] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=5] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:48.308511] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=3] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:48.312165] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:48.312179] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:48.312193] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373645231, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:48.313444] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061868312438000, start_id=1729061868312438000, end_id=1729061868312438001) [2024-10-16 14:57:48.314950] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:48.314983] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:48.317703] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:48.317722] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=15] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28020300, tenant_id=1002) [2024-10-16 14:57:48.317740] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=10] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:48.317794] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Slow write(duration=12139, write_len=4096) [2024-10-16 14:57:48.317804] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601430, offset=9633424}, end_cursor:ObLogCursor{file_id=192, log_id=47601431, offset=9633708}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72ca050, len:368, log_data_len:284, seq:47601430, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:48.319562] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4AA-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061868317863000, start_id=1729061868317863000, end_id=1729061868317863001) [2024-10-16 14:57:48.321651] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=6] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7856, mem_used=14558208) [2024-10-16 14:57:48.331618] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=11] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:48.333731] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D91-0-0] [lt=0][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.333794] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D91-0-0] [lt=36][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"B", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.333852] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D91-0-0] [lt=21][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.333884] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D91-0-0] [lt=15][errcode=-5217] resolve table column expr failed(ret=-5217, q_name={database_name:"", table_name:"D", column:"TABLE_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[], current_resolve_level:1, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x114be224 0x6a05cbe 0x69f96f4 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.336538] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:48.338241] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D94-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.338277] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D94-0-0] [lt=18][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6b22bf7 0x6b22948 0x6b22948 0x6a6bdb3 0x1144278c 0x68891d8 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.338327] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D94-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.338354] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D94-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.338403] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D94-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.338427] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D94-0-0] [lt=9][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x6a0994f 0x6a0930d 0x6a0908b 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x689a992 0x688bbcb 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.338526] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D94-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.338635] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Slow write(duration=17415, write_len=8192) [2024-10-16 14:57:48.338644] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=8] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601431, offset=9633708}, end_cursor:ObLogCursor{file_id=192, log_id=47601432, offset=9633982}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73ba050, len:4180, log_data_len:274, seq:47601431, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:48.338676] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:48.338688] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53383560, tenant_id=1003) [2024-10-16 14:57:48.338698] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:48.339833] WDIAG [SQL.RESV] check_table_column_namespace (ob_column_namespace_checker.cpp:124) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=0][errcode=-5207] column in all tables is ambiguous(q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:48.339886] WDIAG [SQL.RESV] resolve_table_column_expr (ob_dml_resolver.cpp:4587) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=52][errcode=-5207] column not found in table(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}) [2024-10-16 14:57:48.339912] WDIAG [SQL.RESV] resolve_table_column_ref (ob_select_resolver.cpp:5355) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=16][errcode=-5207] resolve table column expr failed(ret=-5207, q_name={database_name:"", table_name:"", column:"TENANT_ID", dblink_name:"", is_star:false, ref_expr:{item_type:"T_REF_COLUMN", result_type:{meta:{type:"", collation:"binary", coercibility:"NUMERIC"}, accuracy:{length:-1, precision:-1, scale:-1}, flag:0, calc_type:{type:"NULL", collation:"invalid_type", coercibility:"INVALID"}}, expr_info:["IS_COLUMN", "CNT_COLUMN"], rel_id:[], table_id:18446744073709551615, column_id:18446744073709551615, database_name:"", table_name:"", synonym_name:"", synonym_db_name:"", column_name:"", column_flags:0, enum_set_values:[], is_lob_column:false, is_joined_dup_column:false, is_unpivot_mocked_column:false, is_hidden:false, from_alias_table:false, is_rowkey_column:false, is_unique_key_column:false, is_mul_key_column:false, is_strict_json_column:0, srs_id:18446744073709551615, udt_set_id:0}, parents_expr_info:[], parent_aggr_level:-1, access_idents:[{access_name:"TENANT_ID", access_index:-1, type:-1, params:[], udf_info:{udf_name:"", udf_package:"", udf_database:"", dblink_name:"", param_names:[], param_exprs:[], udf_param_num:0, is_udt_udf:false, is_contain_self_param:false, is_udt_udf_inside_pkg:false, is_new_keyword_used:false, flag:0}}], current_resolve_level:0, is_access_root:true}, lbt()="0x1a5796b1 0x11606f4f 0x6a091cb 0x6a08b69 0x6a0812c 0x6a06a2e 0x69fabac 0x6c2c860 0x688c2c2 0x6886d47 0x11528b76 0x1149be62 0x116838a7 0x689f30f 0x6887f1c 0x6886d47 0x11528b76 0x6b3bd17 0x6b3b804 0x6b35e44 0x6b2e72c 0x689e814 0x6887f1c 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.340301] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=16][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.340313] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.340335] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.340344] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=4][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.340352] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.340370] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.340381] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.340390] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:864) [21226][RSInspectTask][T1004][YB4A0A000407-00061E9B3B997D95-0-0] [lt=3][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002, lbt()="0x1a5796b1 0xf8e05ab 0x6a0bc8e 0x69d5a28 0x69d498b 0x115e548c 0x689a8b2 0x688bbcb 0x6886d47 0x6884ba7 0x687aad3 0x6875185 0x6874e2e 0x68749de 0x687499c 0x6874b16 0x6874a50 0x687499c 0x6873d38 0x6873432 0x684b222 0x65ec87b 0x65ebab1 0x69ab625 0x69ab17a 0x69a84c9 0xc8cd329 0x69a72b0 0x69a702d 0x69a6ece 0x69a6d30 0x69a5e2e 0x69a532c 0xa61eb2f 0xa623155 0xa61b81b 0x6b435e6 0x6b41dd4 0x1a60fecc 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:48.346117] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:48.349220] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061868347858000, start_id=1729061868347858000, end_id=1729061868347858001) [2024-10-16 14:57:48.354048] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257778, offset=39037547}, end_cursor:ObLogCursor{file_id=197, log_id=49257779, offset=39037830}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7282050, len:1429, log_data_len:283, seq:49257778, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:48.355923] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061868355014000, start_id=1729061868355014000, end_id=1729061868355014001) [2024-10-16 14:57:48.356032] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1001, new_min_version={val:1729061868147631000, v:0}, new_max_version={val:1729061868147631000, v:0}, last_min_version={val:1729061867946952000, v:0}, last_max_version={val:1729061867946952000, v:0}, total_time_us=7541, delta_us=208400) [2024-10-16 14:57:48.356044] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1001, current_version={val:1729061868147631000, v:0}, min_version={val:1729061868147631000, v:0}, max_version={val:1729061868147631000, v:0}, in_service=true, leader_epoch=33705) [2024-10-16 14:57:48.356364] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=370, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:48.356382] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=950, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:48.356408] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:48.356746] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601432, offset=9633982}, end_cursor:ObLogCursor{file_id=192, log_id=47601433, offset=9634256}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db726a050, len:3906, log_data_len:274, seq:47601432, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:48.357006] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601433, offset=9634256}, end_cursor:ObLogCursor{file_id=192, log_id=47601434, offset=9634530}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d0c050, len:3632, log_data_len:274, seq:47601433, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:48.358057] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB375-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061868357313000, start_id=1729061868357313000, end_id=1729061868357313001) [2024-10-16 14:57:48.358142] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561342, miss_cnt:1, cache_read_size:152590687129, hit_ratio:"9.999999928346920974e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:48.358164] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=38] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598930, miss_cnt:9, cache_read_size:149777920165, hit_ratio:"9.999999345925189020e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:48.358321] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2D8-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:48.358330] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2D8-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=35996, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:48.358355] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=41] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657019, miss_cnt:17, cache_read_size:8050172974, hit_ratio:"9.999997286817078335e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:48.363246] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.367766] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77F3-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:48.367796] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77F3-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:48.367801] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77F3-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=33607, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:48.370337] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Slow write(duration=10772, write_len=4096) [2024-10-16 14:57:48.370351] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257779, offset=39037830}, end_cursor:ObLogCursor{file_id=197, log_id=49257780, offset=39038112}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e4c050, len:1146, log_data_len:282, seq:49257779, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:48.374495] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257780, offset=39038112}, end_cursor:ObLogCursor{file_id=197, log_id=49257781, offset=39038394}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e38050, len:864, log_data_len:282, seq:49257780, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:48.378768] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CD5-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:48.378791] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CD5-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:48.378795] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CD5-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40515, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:48.378799] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CD5-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=601, avg_log_submit_to_slide_time=40509) [2024-10-16 14:57:48.378852] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:48.378864] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650280, miss_cnt:32, cache_read_size:8050425480, hit_ratio:"9.999994892284016501e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:48.379626] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.382177] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=14] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:48.385645] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Slow write(duration=11079, write_len=4096) [2024-10-16 14:57:48.385656] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257781, offset=39038394}, end_cursor:ObLogCursor{file_id=197, log_id=49257782, offset=39038676}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd200c050, len:582, log_data_len:282, seq:49257781, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:48.393514] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=52] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=87, stat_interval=1000000, avg (count/cost)=196, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:48.393526] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=87, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:48.394833] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:48.396539] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=18] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:48.412832] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:48.414870] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:48.414885] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:48.414915] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:48.414921] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:48.415252] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:48.418844] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:48.439389] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:48.439406] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:48.439671] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:48.442929] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061868442724000, start_id=1729061868442724000, end_id=1729061868442724001) [2024-10-16 14:57:48.453737] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91AB-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=2062, avg_log_submit_to_slide_time=27023) [2024-10-16 14:57:48.453750] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:48.463322] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:48.463343] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=3423, avg_group_log_size=380) [2024-10-16 14:57:48.463354] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:48.463487] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.468020] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=241, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:48.468039] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=380, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:48.468389] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=2173, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:48.477022] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A86-0-0] [lt=9] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:48.477044] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A86-0-0] [lt=21] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:48.479685] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.491779] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=4] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:48.492385] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=19] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:48.492435] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=17] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061858725452000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:48.493553] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5433-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:48.493565] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5433-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=29619, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:48.493593] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1454, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:48.496424] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061868495752000, start_id=1729061868495752000, end_id=1729061868495752001) [2024-10-16 14:57:48.496518] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8F4-0-0] [lt=7] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:48.496532] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8F4-0-0] [lt=11] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:48.496548] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8F4-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:48.508675] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] ====== checkpoint timer task ====== [2024-10-16 14:57:48.509414] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=15] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:48.509444] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=29] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:48.509471] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=23] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:48.509543] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061875590307000, latest_log_ts:{val:1729061858932674001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:48.509593] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=47] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:48.509626] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:48.509637] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=0, rec_log_ts={val:1729061858932674001, v:0}) [2024-10-16 14:57:48.509644] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:48.509647] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:48.509657] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:48.509671] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:48.509683] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:48.509697] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=13] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:48.513789] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061868512919000, start_id=1729061868512919000, end_id=1729061868512919001) [2024-10-16 14:57:48.533662] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:48.533682] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:48.533699] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444560951, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:48.537966] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2233447, replace_num=31457) [2024-10-16 14:57:48.537981] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:48.539411] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:48.539428] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:48.539688] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:48.540071] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41D1-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=1045, avg_log_submit_to_slide_time=1752) [2024-10-16 14:57:48.540256] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41D2-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:48.540263] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41D2-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1812, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:48.541985] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=16] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:48.542007] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:14538496639}, scn={val:1729061868148862005, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061868479324, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538496639}, next_to_submit_scn_:{val:1729061868148862005, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538496517}, end_lsn:{lsn:14538496639}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061868148862004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061868148862004, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:3122819238, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061868148862004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650282, miss_cnt:32, cache_read_size:8050425724, hit_ratio:"9.999994892284178594e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:48.542053] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=42] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:48.542647] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=911101952, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=162639872, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:48.542699] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=19] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:48.542708] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:48.542739] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=969699328, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=104042496, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:48.542772] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:48.542776] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:48.542811] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=940462080, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=133279744, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:48.542846] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:48.542850] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011866, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:48.542882] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=958939136, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=114802688, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:48.542913] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:48.542917] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:48.543400] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=356, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:48.543414] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=365, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:48.543429] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10718) [2024-10-16 14:57:48.543436] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:48.549369] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061868548869000, start_id=1729061868548869000, end_id=1729061868548869001) [2024-10-16 14:57:48.554364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061868554150000, start_id=1729061868554150000, end_id=1729061868554150001) [2024-10-16 14:57:48.554388] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061868554384], gts:1729061868554150000, latest_srr:[mts=1729061868554384]}) [2024-10-16 14:57:48.556123] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061868555211000, start_id=1729061868555211000, end_id=1729061868555211001) [2024-10-16 14:57:48.556609] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1AD-0-0] [lt=32] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:48.556625] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1AD-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1492, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:48.556630] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1AD-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=26, avg_log_submit_to_flush_time=987, avg_log_submit_to_slide_time=1465) [2024-10-16 14:57:48.563550] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.564748] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115178525, max_lsn:14538496639, max_scn:{val:1729061868148862004, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6487749074}, reuse_lsn:{lsn:14538496639}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14496553599}}, last_submit_lsn:{lsn:14538496517}, last_submit_end_lsn:{lsn:14538496639}, last_submit_log_id:115178525, last_submit_log_pid:23, max_flushed_lsn:{lsn:14538496517}, max_flushed_end_lsn:{lsn:14538496639}, max_flushed_log_pid:23, committed_end_lsn:{lsn:14538496639}, last_slide_log_id:115178525, last_slide_scn:{val:1729061868148862004, v:0}, last_slide_lsn:{lsn:14538496517}, last_slide_end_lsn:{lsn:14538496639}, last_slide_log_pid:23, last_slide_log_accum_checksum:3122819238, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43cc0530}, [StateMgr]={this:0x2b0e43cc4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, leader:"10.0.4.8:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6488080380}, scan_disk_log_finished:true, last_check_start_id:115178526, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261653045, palf_role_change_cb:0x2b0e43ccb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, reconfig_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, persistent_config_version:{proposal_id:23, config_seq:34}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061865851103, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061865851103, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43cc1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, accepted_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1728259580444569001, v:0}, min_block_id:201, min_block_min_scn:{val:1728204893605525001, v:0}, base_lsn_for_block_gc:{lsn:14427525120}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, log_config_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, log_snapshot_meta:{version:1, base_lsn:{lsn:14427525120}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}}, log_meta_storage:{log_tail:{lsn:1134592}, readable_log_tail:{lsn:1134592}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14561734656}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/meta", dir_fd:166, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65970176, block_header_serialize_buf_:0x2b0e43cc69f8, flashback_version:0}, log_storage:{log_tail:{lsn:14538496639}, readable_log_tail:{lsn:14538496639}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14494629888}, min_scn:{val:1729026040007117003, v:0}, curr_block_id:216, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/log", dir_fd:201, min_block_id:201, max_block_id:217, curr_writable_block_id:216}, logical_block_size_:67104768, curr_block_writable_size_:23238017, block_header_serialize_buf_:0x2b0e43cc86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261652788, this:0x2b0e43cc4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:22, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:21, majority_max_lsn:{lsn:6487893593}, saved_end_lsn:{lsn:6487893593}, last_submit_prepare_req_time_us:1722494261653224, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43cc49b0}) [2024-10-16 14:57:48.564888] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=137] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:48.564903] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=12] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=115178526, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:48.564932] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:220938338, max_lsn:213058247513, max_scn:{val:1729061868148862005, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:60464363030}, reuse_lsn:{lsn:213058247513}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:213016304473}}, last_submit_lsn:{lsn:213058247391}, last_submit_end_lsn:{lsn:213058247513}, last_submit_log_id:220938338, last_submit_log_pid:88, max_flushed_lsn:{lsn:213058247391}, max_flushed_end_lsn:{lsn:213058247513}, max_flushed_log_pid:88, committed_end_lsn:{lsn:213058247513}, last_slide_log_id:220938338, last_slide_scn:{val:1729061868148862005, v:0}, last_slide_lsn:{lsn:213058247391}, last_slide_end_lsn:{lsn:213058247513}, last_slide_log_pid:88, last_slide_log_accum_checksum:3386034572, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0dac0530}, [StateMgr]={this:0x2b0e0dac4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, leader:"10.0.4.8:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:60468525403}, scan_disk_log_finished:true, last_check_start_id:220938339, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261639483, palf_role_change_cb:0x2b0e0dacb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, reconfig_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, persistent_config_version:{proposal_id:88, config_seq:21}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061867964260, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061867964260, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0dac1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, accepted_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1729009096591759001, v:0}, min_block_id:3154, min_block_min_scn:{val:1729006455168279002, v:0}, base_lsn_for_block_gc:{lsn:212587905024}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, log_config_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, log_snapshot_meta:{version:1, base_lsn:{lsn:212587905024}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}}, log_meta_storage:{log_tail:{lsn:5767168}, readable_log_tail:{lsn:5767168}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213124743168}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/meta", dir_fd:175, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61337600, block_header_serialize_buf_:0x2b0e0dac69f8, flashback_version:0}, log_storage:{log_tail:{lsn:213058247513}, readable_log_tail:{lsn:213058247513}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213057638400}, min_scn:{val:1729061839666529002, v:0}, curr_block_id:3175, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/log", dir_fd:177, min_block_id:3154, max_block_id:3176, curr_writable_block_id:3175}, logical_block_size_:67104768, curr_block_writable_size_:66495655, block_header_serialize_buf_:0x2b0e0dac86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261637707, this:0x2b0e0dac4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:87, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:86, majority_max_lsn:{lsn:60466038076}, saved_end_lsn:{lsn:60466038076}, last_submit_prepare_req_time_us:1722494261639634, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0dac49b0}) [2024-10-16 14:57:48.565065] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=130] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:48.565078] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=12] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=220938339, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:48.565100] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:211147363, max_lsn:204023108975, max_scn:{val:1729061868253362002, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57922170954}, reuse_lsn:{lsn:204023108975}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:203981165935}}, last_submit_lsn:{lsn:204023108853}, last_submit_end_lsn:{lsn:204023108975}, last_submit_log_id:211147363, last_submit_log_pid:130, max_flushed_lsn:{lsn:204023108853}, max_flushed_end_lsn:{lsn:204023108975}, max_flushed_log_pid:130, committed_end_lsn:{lsn:204023108975}, last_slide_log_id:211147363, last_slide_scn:{val:1729061868253362002, v:0}, last_slide_lsn:{lsn:204023108853}, last_slide_end_lsn:{lsn:204023108975}, last_slide_log_pid:130, last_slide_log_accum_checksum:1239968475, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1ec530}, [StateMgr]={this:0x2b0e0f1f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:211147364, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261539368, palf_role_change_cb:0x2b0e0f1f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, reconfig_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, persistent_config_version:{proposal_id:130, config_seq:17}, ms_ack_list:0, resend_config_version:{proposal_id:130, config_seq:17}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258231773, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258231773, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, accepted_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261739861, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1729008439619722001, v:0}, min_block_id:3020, min_block_min_scn:{val:1729005678665693002, v:0}, base_lsn_for_block_gc:{lsn:203595866112}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, log_config_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, log_snapshot_meta:{version:1, base_lsn:{lsn:203595866112}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}}, log_meta_storage:{log_tail:{lsn:6397952}, readable_log_tail:{lsn:6397952}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:204065599488}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/meta", dir_fd:145, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:60706816, block_header_serialize_buf_:0x2b0e0f1f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:204023108975}, readable_log_tail:{lsn:204023108975}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:203998494720}, min_scn:{val:1729060878194301003, v:0}, curr_block_id:3040, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/log", dir_fd:173, min_block_id:3020, max_block_id:3041, curr_writable_block_id:3040}, logical_block_size_:67104768, curr_block_writable_size_:42490513, block_header_serialize_buf_:0x2b0e0f1f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261539352, this:0x2b0e0f1f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:130, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:129, majority_max_lsn:{lsn:57923794664}, saved_end_lsn:{lsn:57923794664}, last_submit_prepare_req_time_us:1722494261539574, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1f09b0}) [2024-10-16 14:57:48.565205] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=104] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868539701, last_flushed_end_lsn:{lsn:204023108975}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868540276, last_flushed_end_lsn:{lsn:204023108975}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868540231, last_flushed_end_lsn:{lsn:204023108975}}]) [2024-10-16 14:57:48.565213] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=211147364, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868539701, last_flushed_end_lsn:{lsn:204023108975}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868540276, last_flushed_end_lsn:{lsn:204023108975}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868540231, last_flushed_end_lsn:{lsn:204023108975}}]) [2024-10-16 14:57:48.565231] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:218497485, max_lsn:207589555968, max_scn:{val:1729061868148862003, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57807896114}, reuse_lsn:{lsn:207589555968}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:207547612928}}, last_submit_lsn:{lsn:207589555846}, last_submit_end_lsn:{lsn:207589555968}, last_submit_log_id:218497485, last_submit_log_pid:81, max_flushed_lsn:{lsn:207589555846}, max_flushed_end_lsn:{lsn:207589555968}, max_flushed_log_pid:81, committed_end_lsn:{lsn:207589555968}, last_slide_log_id:218497485, last_slide_scn:{val:1729061868148862003, v:0}, last_slide_lsn:{lsn:207589555846}, last_slide_end_lsn:{lsn:207589555968}, last_slide_log_pid:81, last_slide_log_accum_checksum:1969609116, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0db32530}, [StateMgr]={this:0x2b0e0db36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, leader:"10.0.4.9:2890", leader_epoch:7, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:57814812323}, scan_disk_log_finished:true, last_check_start_id:218497486, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433991656, palf_role_change_cb:0x2b0e0db3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, reconfig_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, persistent_config_version:{proposal_id:81, config_seq:20}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061864939057, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061864939057, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0db33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, accepted_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1729009518284639001, v:0}, min_block_id:3073, min_block_min_scn:{val:1729006819670466001, v:0}, base_lsn_for_block_gc:{lsn:207152418816}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, log_config_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, log_snapshot_meta:{version:1, base_lsn:{lsn:207152418816}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}}, log_meta_storage:{log_tail:{lsn:5660672}, readable_log_tail:{lsn:5660672}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207622152192}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/meta", dir_fd:180, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61444096, block_header_serialize_buf_:0x2b0e0db389f8, flashback_version:0}, log_storage:{log_tail:{lsn:207589555968}, readable_log_tail:{lsn:207589555968}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207555047424}, min_scn:{val:1729060516440786001, v:0}, curr_block_id:3093, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/log", dir_fd:182, min_block_id:3073, max_block_id:3094, curr_writable_block_id:3093}, logical_block_size_:67104768, curr_block_writable_size_:32596224, block_header_serialize_buf_:0x2b0e0db3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433990608, this:0x2b0e0db36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:80, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:79, majority_max_lsn:{lsn:57811903042}, saved_end_lsn:{lsn:57811903042}, last_submit_prepare_req_time_us:1722494433991756, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0db369b0}) [2024-10-16 14:57:48.565325] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=92] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:48.565332] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=218497486, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:48.568618] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.568719] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.569273] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.569294] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.574407] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22405][T1003_DupTbLeas][T1003][Y0-0000000000000000-0-0] [lt=12][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:48.574475] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22405][T1003_DupTbLeas][T1003][Y0-0000000000000000-0-0] [lt=13][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1003 and flag like "%DUPLICATE%", tenant_id=1003, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:48.579748] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.596118] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1}, {id:1002}]) [2024-10-16 14:57:48.596337] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=2][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:48.596350] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:48.609827] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5478, mem_used=10398720) [2024-10-16 14:57:48.615508] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2DF-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:48.615531] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2DF-0-0] [lt=22] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=749, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:48.615535] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107145][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2DF-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=13, avg_log_submit_to_flush_time=320, avg_log_submit_to_slide_time=736) [2024-10-16 14:57:48.619140] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:48.629040] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CA-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:48.629068] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CA-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061868148862004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061868148862004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:48.630167] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9CA-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061868629284000, start_id=1729061868629284000, end_id=1729061868629284001) [2024-10-16 14:57:48.632819] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CA-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061868632560000, start_id=1729061868632560000, end_id=1729061868632560001) [2024-10-16 14:57:48.632892] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CA-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10123, avg_group_log_size=595) [2024-10-16 14:57:48.632915] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CA-0-0] [lt=20] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10123) [2024-10-16 14:57:48.633125] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10123) [2024-10-16 14:57:48.633525] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:48.638026] INFO [LIB] runTimerTask (ob_work_queue.cpp:24) [21220][RSAsyncTask][T0][Y0-0000000000000000-0-0] [lt=4] add async task(this=tasktype:N9oceanbase10rootserver23ObUpdateRsListTimerTaskE) [2024-10-16 14:57:48.638042] INFO [LIB] runTimerTask (ob_work_queue.cpp:24) [21220][RSAsyncTask][T0][Y0-0000000000000000-0-0] [lt=12] add async task(this=tasktype:N9oceanbase10rootserver13ObRootService27ObUpdateAllServerConfigTaskE) [2024-10-16 14:57:48.638093] INFO [SHARE] store (ob_rs_mgr.cpp:128) [21221][RSAsyncTask0][T0][Y0-00061E997F998436-0-0] [lt=6] store rs list succeed(agent=0, addr_list=[{server:"10.0.4.7:2890", role:1, sql_port:2889}, {server:"10.0.4.9:2890", role:2, sql_port:2889}, {server:"10.0.4.8:2890", role:2, sql_port:2889}], force=true) [2024-10-16 14:57:48.642846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061868641008000, start_id=1729061868641008000, end_id=1729061868641008001) [2024-10-16 14:57:48.649143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA11-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061868648187000, start_id=1729061868648187000, end_id=1729061868648187001) [2024-10-16 14:57:48.649467] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA12-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061868649245000, start_id=1729061868649245000, end_id=1729061868649245001) [2024-10-16 14:57:48.649475] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA10-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061868649245000, start_id=1729061868649245000, end_id=1729061868649245001) [2024-10-16 14:57:48.649495] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA14-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061868649245000, start_id=1729061868649245000, end_id=1729061868649245001) [2024-10-16 14:57:48.663255] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:48.663274] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=17] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:57:48.663281] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:57:48.663286] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:48.665453] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=108, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}, {obj_id:616254, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616255, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616256, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:616257, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}]) [2024-10-16 14:57:48.665605] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=144] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:48.665626] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.665663] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109593790, max_lsn:13645427147, max_scn:{val:1729061868554150000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6198843846}, reuse_lsn:{lsn:13645427147}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:13603484107}}, last_submit_lsn:{lsn:13645427025}, last_submit_end_lsn:{lsn:13645427147}, last_submit_log_id:109593790, last_submit_log_pid:17, max_flushed_lsn:{lsn:13645427025}, max_flushed_end_lsn:{lsn:13645427147}, max_flushed_log_pid:17, committed_end_lsn:{lsn:13645427147}, last_slide_log_id:109593790, last_slide_scn:{val:1729061868554150000, v:0}, last_slide_lsn:{lsn:13645427025}, last_slide_end_lsn:{lsn:13645427147}, last_slide_log_pid:17, last_slide_log_accum_checksum:4168561909, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431ec530}, [StateMgr]={this:0x2b0e431f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109593791, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261552988, palf_role_change_cb:0x2b0e431f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:14, lock_type:0, added_list:0, removed_list:0, type:"UNLOCK_CONFIG_CHANGE"}, state:0, checking_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:30}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:30}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494257649102, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494257649102, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261753456, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1728296971089337004, v:0}, min_block_id:189, min_block_min_scn:{val:1728240898542283004, v:0}, base_lsn_for_block_gc:{lsn:13555163136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:13555163136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}}, log_meta_storage:{log_tail:{lsn:1011712}, readable_log_tail:{lsn:1011712}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13689372672}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/meta", dir_fd:198, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66093056, block_header_serialize_buf_:0x2b0e431f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:13645427147}, readable_log_tail:{lsn:13645427147}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13622267904}, min_scn:{val:1729042537979987002, v:0}, curr_block_id:203, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/log", dir_fd:200, min_block_id:189, max_block_id:204, curr_writable_block_id:203}, logical_block_size_:67104768, curr_block_writable_size_:43945525, block_header_serialize_buf_:0x2b0e431f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261552391, this:0x2b0e431f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:6198988121}, saved_end_lsn:{lsn:6198988121}, last_submit_prepare_req_time_us:1722494261553147, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431f09b0}) [2024-10-16 14:57:48.665802] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=135] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868615293, last_flushed_end_lsn:{lsn:13645427147}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868615466, last_flushed_end_lsn:{lsn:13645427147}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868615623, last_flushed_end_lsn:{lsn:13645427147}}]) [2024-10-16 14:57:48.665812] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=109593791, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868615293, last_flushed_end_lsn:{lsn:13645427147}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868615466, last_flushed_end_lsn:{lsn:13645427147}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061868615623, last_flushed_end_lsn:{lsn:13645427147}}]) [2024-10-16 14:57:48.671719] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:48.679812] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:48.679833] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=20] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:48.679840] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.683483] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C57-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:48.683512] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C57-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061868649245001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061868649245001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:48.687466] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=10] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:48.687830] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C57-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061868686399000, start_id=1729061868686399001, end_id=1729061868686399002) [2024-10-16 14:57:48.688181] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061868686399000, start_id=1729061868686399000, end_id=1729061868686399001) [2024-10-16 14:57:48.697315] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061868696811000, start_id=1729061868696811000, end_id=1729061868696811001) [2024-10-16 14:57:48.697392] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3545) [2024-10-16 14:57:48.697689] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3545) [2024-10-16 14:57:48.697721] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3545) [2024-10-16 14:57:48.697842] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1068-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:48.697860] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1068-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1529, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:48.699108] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:48.713841] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061868713794000, start_id=1729061868713794000, end_id=1729061868713794001) [2024-10-16 14:57:48.714998] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:48.715014] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] try gc retain ctx [2024-10-16 14:57:48.715017] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:48.715036] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=2] get_max_applied_scn(scn={val:1729061868554150000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493479446}, palf_committed_end_scn_:{val:1729061868686399000, v:0}, last_check_scn_:{val:1729061868686399000, v:0}, max_applied_cb_scn_:{val:1729061868554150000, v:0}}) [2024-10-16 14:57:48.715060] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:48.715738] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=48] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880721265000, latest_log_ts:{val:1729061864144741001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:48.715834] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=43] get_max_applied_scn(scn={val:1729061868554150000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645427147}, palf_committed_end_scn_:{val:1729061868554150000, v:0}, last_check_scn_:{val:1729061868554150000, v:0}, max_applied_cb_scn_:{val:1729061868554150000, v:0}}) [2024-10-16 14:57:48.715845] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:48.716498] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:48.716779] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=11] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:48.717158] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=37] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:48.717200] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=35] get_min_unreplayed_log_info(lsn={lsn:14352383119}, scn={val:1729061868649245001, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061868652801, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352383119}, next_to_submit_scn_:{val:1729061868649245001, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352382997}, end_lsn:{lsn:14352383119}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061868649245000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061868649245000, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:1815076148, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061868649245000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657022, miss_cnt:17, cache_read_size:8050173340, hit_ratio:"9.999997286817208231e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:48.717852] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=37] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:48.726692] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:48.731710] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=3650007, this={dio_aligned_buf:{buf_write_offset:1289, buf_padding_size:2807, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80962793, truncate_used_ts:29163537}, log_block_size:67108864, dir_fd:148, io_fd:171}, aligned_buf_len=4096, aligned_block_offset=3649536, buf_len=818, total_write_size_=101909, total_write_size_after_dio_=815104, ob_pwrite_used_ts=55693, count=174, each_pwrite_cost=320) [2024-10-16 14:57:48.738069] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:48.739771] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=285, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:48.739796] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=725, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:48.739816] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:48.744107] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=19] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:48.748619] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:48.749488] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061868749002000, start_id=1729061868749002000, end_id=1729061868749002001) [2024-10-16 14:57:48.752423] INFO [LIB] stat (utility.h:1165) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E5E-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:48.752442] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107145][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E5E-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:57:48.752606] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1333) [2024-10-16 14:57:48.756233] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061868755321000, start_id=1729061868755321000, end_id=1729061868755321001) [2024-10-16 14:57:48.757924] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:48.757949] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:48.757959] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:48.765689] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.772114] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:48.779869] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.782174] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=14] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:48.788572] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.788598] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.788618] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.788618] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.788634] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.788642] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.788657] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.788699] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=3] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.789640] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.789642] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.789658] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:48.796544] INFO [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:194) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get wrs ts(ls_id={id:1}, delta=98787, log_service_ts=1729061868696811, min_tx_service_ts=4611686018427387, timestamp={val:1729061868696811000, v:0}) [2024-10-16 14:57:48.796562] INFO [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:214) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=0, Near_To_GTS_Cnt=60, Other_Error_Cnt=0, Submit_Succ_Cnt=40, last_scn="{val:1729061865590789001, v:0}", last_lsn={lsn:66390139254}, last_gts={val:1729061868696811000, v:0}, min_start_scn="{val:0, v:0}", min_start_status=1, sys_ls_end_scn={val:0, v:0}) [2024-10-16 14:57:48.819422] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:48.830812] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1003}, {id:1}]) [2024-10-16 14:57:48.843028] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061868842355000, start_id=1729061868842355000, end_id=1729061868842355001) [2024-10-16 14:57:48.843745] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:48.859733] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5288, mem_used=10398720) [2024-10-16 14:57:48.866092] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061864849472, last_first_register_time_us=1722494157071428, register_time_us=1729061864849472, curr_time_us=1729061868866090) [2024-10-16 14:57:48.866124] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061868866109, last_submit_register_req_time_us=1729061868866109, register_parent_reason=3) [2024-10-16 14:57:48.866133] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:48.867517] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=21] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:57:48.867616] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.869756] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:48.872748] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=11] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:48.873065] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107145][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=5] worker thread exit(tenant_->id()=1004, workers_.get_size()=2) [2024-10-16 14:57:48.873077] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107145][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=9] tenant thread end_run(id_=1004, ret=0, thread_count_=191) [2024-10-16 14:57:48.879064] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CDF-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:48.879570] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:48.879589] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:48.879908] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.886705] INFO [ARCHIVE] do_thread_task_ (ob_archive_timer.cpp:130) [21905][T1001_ArcTimer][T1001][YB4A0A000407-00061E996DC960C1-0-0] [lt=16] archive round not in doing status, just skip(key={incarnation:-1, dest_id:-1, round:-1}, state={status:"INVALID"}) [2024-10-16 14:57:48.887287] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D74-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:48.887314] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D74-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061868649245003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061868649245003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:48.888271] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D74-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061868887367000, start_id=1729061868887367000, end_id=1729061868887367001) [2024-10-16 14:57:48.890623] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D74-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061868890531000, start_id=1729061868890531000, end_id=1729061868890531001) [2024-10-16 14:57:48.890689] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D74-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:48.890705] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D74-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:48.897318] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061868895788000, start_id=1729061868895788000, end_id=1729061868895788001) [2024-10-16 14:57:48.901854] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:48.913852] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061868913372000, start_id=1729061868913372000, end_id=1729061868913372001) [2024-10-16 14:57:48.915187] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=36] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:48.915202] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:48.915379] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:48.915386] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=222, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:48.915399] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=262, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:48.915421] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:48.915637] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2E9-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:48.915644] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2E9-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=792, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:48.923792] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:48.925433] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15DE-0-0] [lt=20] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:48.925495] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15E0-0-0] [lt=11] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:48.933247] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE3-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061868900988, keepalive_ts:-1}) [2024-10-16 14:57:48.933261] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE3-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061868900988, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:48.938146] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:48.939880] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:48.940184] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41DD-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:48.940194] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41DD-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1314, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:48.943597] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=8] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3466, mem_used=8318976) [2024-10-16 14:57:48.949723] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] submit log success(service_type_=0, last_id=1729061885590307000, limited_id=1729061885590307000) [2024-10-16 14:57:48.949742] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] submit log with lock success(service_type_=0, last_id=1729061885590307000, limited_id=1729061885590307000) [2024-10-16 14:57:48.949747] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061868948338000, start_id=1729061868948338000, end_id=1729061868948338001) [2024-10-16 14:57:48.950248] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] update id meta(ret=0, service_type=0, limited_id=1729061885590307000, latest_scn={val:1729061868948338001, v:0}, *this={tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061885590307000, latest_log_ts:{val:1729061868948338001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:48.950282] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=34] handle submit callback(service_type_=0, success=true, used_ts=560, log_ts={val:1729061868948338001, v:0}, limited_id=1729061885590307000) [2024-10-16 14:57:48.950289] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:48.950296] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:48.950407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061868949888000, start_id=1729061868949888000, end_id=1729061868949888001) [2024-10-16 14:57:48.952543] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E62-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:48.956256] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22455][T1003_TenantWea][T1003][YB4A0A000407-00061E9B3ABCFD21-0-0] [lt=19] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.103558874130249023e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.008835578279190031e+03) [2024-10-16 14:57:48.956404] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061868956233000, start_id=1729061868956233000, end_id=1729061868956233001) [2024-10-16 14:57:48.963718] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=3] ====== tenant freeze timer task ====== [2024-10-16 14:57:48.963732] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:57:48.963742] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:48.965184] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061864939057, last_first_register_time_us=1722494150998907, register_time_us=1729061864939057, curr_time_us=1729061868965182) [2024-10-16 14:57:48.965220] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061868965202, last_submit_register_req_time_us=1729061868965202, register_parent_reason=3) [2024-10-16 14:57:48.965230] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:48.966147] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=6] ====== tenant freeze timer task ====== [2024-10-16 14:57:48.966154] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115173223, max_lsn:14352383485, max_scn:{val:1729061868649245003, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6301879685}, reuse_lsn:{lsn:14352383485}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14310440445}}, last_submit_lsn:{lsn:14352383363}, last_submit_end_lsn:{lsn:14352383485}, last_submit_log_id:115173223, last_submit_log_pid:17, max_flushed_lsn:{lsn:14352383363}, max_flushed_end_lsn:{lsn:14352383485}, max_flushed_log_pid:17, committed_end_lsn:{lsn:14352383485}, last_slide_log_id:115173223, last_slide_scn:{val:1729061868649245003, v:0}, last_slide_lsn:{lsn:14352383363}, last_slide_end_lsn:{lsn:14352383485}, last_slide_log_pid:17, last_slide_log_accum_checksum:4157701670, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43d32530}, [StateMgr]={this:0x2b0e43d36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.9:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6302436797}, scan_disk_log_finished:true, last_check_start_id:115173224, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433837169, palf_role_change_cb:0x2b0e43d3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, reconfig_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, persistent_config_version:{proposal_id:17, config_seq:26}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061868866109, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061868866109, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43d33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1728301961314447004, v:0}, min_block_id:199, min_block_min_scn:{val:1728247158357854003, v:0}, base_lsn_for_block_gc:{lsn:14293315584}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:14293315584}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}}, log_meta_storage:{log_tail:{lsn:1064960}, readable_log_tail:{lsn:1064960}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14360420352}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/meta", dir_fd:192, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66039808, block_header_serialize_buf_:0x2b0e43d389f8, flashback_version:0}, log_storage:{log_tail:{lsn:14352383485}, readable_log_tail:{lsn:14352383485}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14293315584}, min_scn:{val:1729013741169122001, v:0}, curr_block_id:213, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/log", dir_fd:207, min_block_id:199, max_block_id:214, curr_writable_block_id:213}, logical_block_size_:67104768, curr_block_writable_size_:8036867, block_header_serialize_buf_:0x2b0e43d3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433836510, this:0x2b0e43d36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:16, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:15, majority_max_lsn:{lsn:6302238429}, saved_end_lsn:{lsn:6302238429}, last_submit_prepare_req_time_us:1722494433837290, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43d369b0}) [2024-10-16 14:57:48.966305] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=147] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:48.966316] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=115173224, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:48.967643] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.968118] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE77FF-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1454) [2024-10-16 14:57:48.979334] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:48.979972] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:48.992972] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:48.992995] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=213) [2024-10-16 14:57:49.006070] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:49.006089] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151784220, tenant_id=1) [2024-10-16 14:57:49.006102] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:49.014793] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061869014792, bytes: 4446816987062, bw: 0.345812 MB/s, add_ts: 1002450, add_bytes: 363499 [2024-10-16 14:57:49.015487] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:49.015503] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:49.015958] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF4A-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=507, avg_log_submit_to_slide_time=775) [2024-10-16 14:57:49.017007] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 8094.75; ) [2024-10-16 14:57:49.017025] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:49.017064] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 7653.75; ) [2024-10-16 14:57:49.017072] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:49.017083] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:49.019715] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:49.022499] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Cache wash clean map node details(ret=0, clean_node_count=15, clean_time=0, clean_start_pos=1258280, clean_num=62914) [2024-10-16 14:57:49.028925] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:57:49.029653] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.029681] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.029702] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:49.029843] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=16] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880098662000, latest_log_ts:{val:1729061863549302001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:49.029892] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=46] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880098662000, latest_log_ts:{val:1729061863549302001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:49.029925] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:49.029935] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061863549302001, v:0}) [2024-10-16 14:57:49.029942] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:49.029950] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:49.029963] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:49.029974] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:49.030660] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.030675] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.030685] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:49.030721] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:49.030750] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:49.030777] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:49.030782] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:49.030791] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:49.030799] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:49.031476] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.031492] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.031502] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:49.031536] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:49.031566] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=29] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:49.031598] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:49.031608] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:49.031618] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:49.031627] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:49.032312] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.032331] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=19] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.032342] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:49.032379] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:49.032411] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:49.032438] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:49.032444] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:49.032450] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:49.032456] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:49.032459] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:49.039701] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:49.039718] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:49.039762] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:49.039772] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:49.040320] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41E0-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=874, avg_log_submit_to_slide_time=1211) [2024-10-16 14:57:49.041000] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:49.042691] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:49.042703] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:49.042715] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291954599, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:49.042953] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061869042358000, start_id=1729061869042358000, end_id=1729061869042358001) [2024-10-16 14:57:49.045434] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=13][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5171, dropped:6902, tid:21226}]) [2024-10-16 14:57:49.054771] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061869053945000, start_id=1729061869053945000, end_id=1729061869053945001) [2024-10-16 14:57:49.054792] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:49.054811] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061869054807], gts:1729061869053945000, latest_srr:[mts=1729061869054807]}) [2024-10-16 14:57:49.067718] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.068393] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1454) [2024-10-16 14:57:49.080040] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.093562] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:49.097590] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061869096304000, start_id=1729061869096304000, end_id=1729061869096304001) [2024-10-16 14:57:49.114161] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061869113504000, start_id=1729061869113504000, end_id=1729061869113504001) [2024-10-16 14:57:49.115296] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:49.115329] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:49.118675] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061869118673, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007622, add_bytes: 0 [2024-10-16 14:57:49.127531] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:49.127545] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:49.136153] INFO [STORAGE] runTimerTask (ob_ddl_heart_beat_task.cpp:49) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8] send to rs all task status succeed [2024-10-16 14:57:49.136781] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3243, mem_used=6239232) [2024-10-16 14:57:49.138221] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:49.140054] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=277, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:49.140070] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=525, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:49.140092] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:49.145840] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:49.149513] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA17-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061869149380000, start_id=1729061869149380000, end_id=1729061869149380001) [2024-10-16 14:57:49.149622] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=16][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:49.149645] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA16-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061869149380000, start_id=1729061869149380000, end_id=1729061869149380001) [2024-10-16 14:57:49.149672] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA15-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061869149380000, start_id=1729061869149380000, end_id=1729061869149380001) [2024-10-16 14:57:49.149792] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA19-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061869149380000, start_id=1729061869149380000, end_id=1729061869149380001) [2024-10-16 14:57:49.149904] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061869149380000, start_id=1729061869149380001, end_id=1729061869149380002) [2024-10-16 14:57:49.149971] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=25, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:49.150000] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:49.150248] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11280) [2024-10-16 14:57:49.150361] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA18-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061869149380000, start_id=1729061869149380000, end_id=1729061869149380001) [2024-10-16 14:57:49.150959] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:49.154436] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=9] current monitor number(seq_=-1) [2024-10-16 14:57:49.156384] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061869154611000, start_id=1729061869154611000, end_id=1729061869154611001) [2024-10-16 14:57:49.167730] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.168214] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7803-0-0] [lt=19] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:49.180072] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.192416] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5033-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:49.192430] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5033-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:49.193427] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85037-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:49.193441] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85037-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:49.194260] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=257, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:49.194284] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=343, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:49.194335] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:49.208444] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EAD-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061869176159, keepalive_ts:-1}) [2024-10-16 14:57:49.208459] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EAD-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061869176159, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:49.208480] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EAE-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061869176218, keepalive_ts:-1}) [2024-10-16 14:57:49.208487] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EAE-0-0] [lt=8] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061869176218, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:49.208648] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA887-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:49.208667] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA887-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061869149380000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061869149380000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:49.212656] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21667][T1_L0_G0][T1][YB4A0A000408-00061E9B592E587C-0-0] [lt=13] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:49.212772] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21667][T1_L0_G0][T1][YB4A0A000408-00061E9B592E587C-0-0] [lt=18][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:49.215586] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=37] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1758) [2024-10-16 14:57:49.216085] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA887-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061869215409000, start_id=1729061869215409000, end_id=1729061869215409001) [2024-10-16 14:57:49.216165] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA887-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=21, avg_log_batch_cnt=1, total_group_log_size=11280, avg_group_log_size=537) [2024-10-16 14:57:49.216180] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA887-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11280) [2024-10-16 14:57:49.216393] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=241, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:49.216411] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=18] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11280) [2024-10-16 14:57:49.216422] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=69, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:49.216440] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:49.216670] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9555-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:49.216676] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9555-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=818, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:49.216686] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9555-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=22, avg_log_gen_to_freeze_time=11, avg_log_gen_to_submit_time=41, avg_log_submit_to_flush_time=393, avg_log_submit_to_slide_time=776) [2024-10-16 14:57:49.216766] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:49.222817] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:49.226389] INFO [SERVER] refresh_avail_memory (ob_table_load_assigned_memory_manager.cpp:117) [21531][T1_TntSharedTim][T1][Y0-0000000000000000-0-0] [lt=11] ObTableLoadAssignedMemoryManager::refresh_avail_memory(MTL_ID()=1, avail_memory=107374182, chunk_count_=0, avail_sort_memory_=107374182, avail_memory_=107374182) [2024-10-16 14:57:49.237147] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3D9E-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510337105542, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410610, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:49.240687] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:49.243156] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061869242115000, start_id=1729061869242115000, end_id=1729061869242115001) [2024-10-16 14:57:49.244293] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:57:49.245070] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.245099] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=28] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.245122] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=18] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:49.245246] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880721265000, latest_log_ts:{val:1729061864144741001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:49.245292] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=42] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:49.245312] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=15] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:49.245322] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061864144741001, v:0}) [2024-10-16 14:57:49.245327] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.245345] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:49.245370] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:49.245378] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:49.246208] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.246232] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=23] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.246247] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:49.246321] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:49.246372] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=48] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:49.246410] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=29] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:49.246420] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:49.246429] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:49.246436] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:49.246550] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD0D-0-0] [lt=5][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:49.247261] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.247288] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=26] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.247303] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:49.247383] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:49.247444] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=59] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:49.247496] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=49] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:49.247507] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:49.247520] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:49.247529] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:49.248256] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.248283] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:49.248297] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:49.248378] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:49.248413] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=34] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061866050909001, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:49.248444] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:49.248452] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:49.248460] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:49.248467] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:49.248471] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:49.250700] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E0D-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061869249780000, start_id=1729061869249780000, end_id=1729061869249780001) [2024-10-16 14:57:49.264019] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C10B0-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061869263880000, start_id=1729061869263880000, end_id=1729061869263880001) [2024-10-16 14:57:49.267805] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.278836] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB9B-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12239, add_timestamp:1729061869278825}) [2024-10-16 14:57:49.278865] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E48-0-0] [lt=7] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12239, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061869278851, sql_renew_time:1729061869278851, is_mark_fail:false}]) [2024-10-16 14:57:49.278891] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E48-0-0] [lt=26] success to process renew task(task={tenant_id:1, table_id:12239, add_timestamp:1729061869278825}, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061869278851, sql_renew_time:1729061869278851, is_mark_fail:false}]) [2024-10-16 14:57:49.280136] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.280274] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CE7-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:49.280295] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CE7-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178532, log_task={header:{begin_lsn:{lsn:14538497371}, end_lsn:{lsn:14538497493}, log_id:115178532, min_scn:{val:1729061869149380001, v:0}, max_scn:{val:1729061869149380001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538497249}, prev_proposal_id:23, committed_end_lsn:{lsn:14538497371}, data_checksum:1715074492, accum_checksum:3060600279, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061869179259, freeze_ts:1729061869179259, submit_ts:1729061869179264, flushed_ts:1729061869179472, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:208}, fs_cb_begin_ts=1729061869280287, log_life_time=101028) [2024-10-16 14:57:49.280641] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=224, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:49.280656] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:49.280686] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:49.282906] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B453E1E02-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061869281954000, start_id=1729061869281954000, end_id=1729061869281954001) [2024-10-16 14:57:49.284233] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB9B-0-0] [lt=14][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:49.287146] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=16] start to print blacklist info [2024-10-16 14:57:49.287296] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=11] ls blacklist refresh finish(cost_time=9137) [2024-10-16 14:57:49.292110] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=16] dump tenant info(tenant={id:1, tenant_meta:{unit:{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, super_block:{tenant_id:1, replay_start_point:ObLogCursor{file_id=12, log_id=1497048, offset=40358755}, ls_meta_entry:[2738](ver=0,mode=0,seq=7639021), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"3.000000000000000000e+00", unit_max_cpu:"3.000000000000000000e+00", total_worker_cnt:54, min_worker_cnt:32, max_worker_cnt:150, stopped:0, worker_us:1096710310883, recv_hp_rpc_cnt:5717689, recv_np_rpc_cnt:154801664, recv_lp_rpc_cnt:0, recv_mysql_cnt:62154046, recv_task_cnt:6664584, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:19623895265, workers:32, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=7 cnt[3]=0 cnt[4]=0 cnt[5]=228765035 cnt[6]=10623 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541081, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143148686}{group_id:10, queue_size:0, recv_req_cnt:19682095, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143048442}{group_id:20, queue_size:0, recv_req_cnt:401, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015260372185}{group_id:15, queue_size:0, recv_req_cnt:5821, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723554329}{group_id:5, queue_size:0, recv_req_cnt:35746161, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494262337428}{group_id:19, queue_size:0, recv_req_cnt:7245692, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061834968674}{group_id:9, queue_size:0, recv_req_cnt:15758102, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143222379}{group_id:1, queue_size:0, recv_req_cnt:153511144, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061854965187}, rpc_stat_info: pcode=0x51c:cnt=312 pcode=0x1502:cnt=198 pcode=0x717:cnt=178 pcode=0x150b:cnt=60 pcode=0x710:cnt=50, token_change_ts:1722494237022007, tenant_role:1}) [2024-10-16 14:57:49.292641] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=529] dump throttled time info(id_=1, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 10, group: OBCG_LOC_CACHE, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 9, group: OBCG_DETECT_RS, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1, tenant_throttled_time: 0;) [2024-10-16 14:57:49.292657] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=16] dump tenant info(tenant={id:508, tenant_meta:{unit:{tenant_id:508, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140910089, is_removed:false}, super_block:{tenant_id:508, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:15574442863, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:51691822, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6536490694, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:15, queue_size:0, recv_req_cnt:53884, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060722522359}, rpc_stat_info: pcode=0x5c0:cnt=72, token_change_ts:1722494140914497, tenant_role:0}) [2024-10-16 14:57:49.293072] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=414] dump throttled time info(id_=508, throttled_time_log=group_id: 15, group: OBCG_WR, throttled_time: 0;tenant_id: 508, tenant_throttled_time: 0;) [2024-10-16 14:57:49.293077] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=6] dump tenant info(tenant={id:509, tenant_meta:{unit:{tenant_id:509, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140901337, is_removed:false}, super_block:{tenant_id:509, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:12838923959, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6441829099, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1722494140916657, tenant_role:0}) [2024-10-16 14:57:49.293505] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=427] dump throttled time info(id_=509, throttled_time_log=tenant_id: 509, tenant_throttled_time: 0;) [2024-10-16 14:57:49.293744] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85039-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:49.293511] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=6] dump tenant info(tenant={id:1001, tenant_meta:{unit:{tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1001, replay_start_point:ObLogCursor{file_id=11, log_id=1059563, offset=59267180}, ls_meta_entry:[7452](ver=0,mode=0,seq=7431948), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:468365124682, recv_hp_rpc_cnt:13589294, recv_np_rpc_cnt:126529614, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2784375840, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=9 cnt[3]=0 cnt[4]=0 cnt[5]=167011650 cnt[6]=3244 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541080, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494145075365}{group_id:20, queue_size:0, recv_req_cnt:306, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015263681355}{group_id:5, queue_size:0, recv_req_cnt:58923612, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726724812431820}{group_id:19, queue_size:0, recv_req_cnt:7662201, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061844973518}{group_id:1, queue_size:0, recv_req_cnt:227493458, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061856295889}, rpc_stat_info: pcode=0x1502:cnt=346 pcode=0x51c:cnt=224 pcode=0x717:cnt=173 pcode=0x710:cnt=90 pcode=0x150b:cnt=60, token_change_ts:1722494144838773, tenant_role:1}) [2024-10-16 14:57:49.293973] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=461] dump throttled time info(id_=1001, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1001, tenant_throttled_time: 0;) [2024-10-16 14:57:49.293986] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] dump tenant info(tenant={id:1002, tenant_meta:{unit:{tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1002, replay_start_point:ObLogCursor{file_id=191, log_id=47529733, offset=57901112}, ls_meta_entry:[10485](ver=0,mode=0,seq=7657455), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:998833526154, recv_hp_rpc_cnt:198478869, recv_np_rpc_cnt:379058798, recv_lp_rpc_cnt:0, recv_mysql_cnt:280947772, recv_task_cnt:146658, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:4103771744, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=673 cnt[3]=0 cnt[4]=0 cnt[5]=64428979 cnt[6]=226112 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131355053, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1728209719937557}{group_id:20, queue_size:0, recv_req_cnt:1673, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264869418}{group_id:15, queue_size:0, recv_req_cnt:5733, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723564477}{group_id:5, queue_size:0, recv_req_cnt:401894113, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1727167492226332}{group_id:19, queue_size:0, recv_req_cnt:1399887, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061856166239}{group_id:1, queue_size:0, recv_req_cnt:977160560, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061846446256}, rpc_stat_info: pcode=0x1502:cnt=406 pcode=0x1501:cnt=201 pcode=0x1516:cnt=197 pcode=0x717:cnt=185 pcode=0x150b:cnt=120, token_change_ts:1729004298741539, tenant_role:1}) [2024-10-16 14:57:49.294473] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=485] dump throttled time info(id_=1002, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1002, tenant_throttled_time: 0;) [2024-10-16 14:57:49.294483] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=10] dump tenant info(tenant={id:1003, tenant_meta:{unit:{tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1003, replay_start_point:ObLogCursor{file_id=11, log_id=1067656, offset=59494798}, ls_meta_entry:[3704](ver=0,mode=0,seq=7380334), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:503358074473, recv_hp_rpc_cnt:13609686, recv_np_rpc_cnt:126132333, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2783972636, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=4 cnt[3]=0 cnt[4]=0 cnt[5]=196318044 cnt[6]=6245 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541045, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722517512781421}{group_id:20, queue_size:0, recv_req_cnt:308, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015261556381}{group_id:5, queue_size:0, recv_req_cnt:63971845, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726083164255294}{group_id:19, queue_size:0, recv_req_cnt:7660901, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061862930562}{group_id:1, queue_size:0, recv_req_cnt:228854505, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061867552475}, rpc_stat_info: pcode=0x1502:cnt=344 pcode=0x51c:cnt=262 pcode=0x717:cnt=181 pcode=0x710:cnt=98 pcode=0x150b:cnt=60, token_change_ts:1722494150252707, tenant_role:1}) [2024-10-16 14:57:49.294926] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=442] dump throttled time info(id_=1003, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1003, tenant_throttled_time: 0;) [2024-10-16 14:57:49.294933] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=7] dump tenant info(tenant={id:1004, tenant_meta:{unit:{tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1004, replay_start_point:ObLogCursor{file_id=197, log_id=49173916, offset=15367427}, ls_meta_entry:[12150](ver=0,mode=0,seq=7655944), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:603327758687, recv_hp_rpc_cnt:12341784, recv_np_rpc_cnt:113755609, recv_lp_rpc_cnt:0, recv_mysql_cnt:11227, recv_task_cnt:1052, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2741357195, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=99 cnt[3]=0 cnt[4]=0 cnt[5]=72426490 cnt[6]=155203 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131354911, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494154950936}{group_id:20, queue_size:0, recv_req_cnt:798, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264132685}{group_id:15, queue_size:0, recv_req_cnt:5423, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723614999}{group_id:21, queue_size:0, recv_req_cnt:15, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269726417109}{group_id:5, queue_size:0, recv_req_cnt:31029637, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1724705543331737}{group_id:19, queue_size:0, recv_req_cnt:1325447, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061845553743}{group_id:16, queue_size:0, recv_req_cnt:18, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269721623294}{group_id:1, queue_size:0, recv_req_cnt:509700251, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061868867515}, rpc_stat_info: pcode=0x1502:cnt=396 pcode=0x1501:cnt=201 pcode=0x1516:cnt=199 pcode=0x717:cnt=183 pcode=0x150b:cnt=120, token_change_ts:1725266206494273, tenant_role:1}) [2024-10-16 14:57:49.295388] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=454] dump throttled time info(id_=1004, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 21, group: OBCG_UNLOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 16, group: OBCG_TRANSFER, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1004, tenant_throttled_time: 0;) [2024-10-16 14:57:49.297610] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061869295868000, start_id=1729061869295868000, end_id=1729061869295868001) [2024-10-16 14:57:49.299131] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21657][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE893-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510337167526, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410611, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:49.299713] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD103A-0-0] [lt=8] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:49.299881] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=482, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:49.299899] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=166, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:49.300977] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:49.301100] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C37-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=866, avg_log_submit_to_slide_time=1144) [2024-10-16 14:57:49.306418] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:49.307146] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985D8-0-0] [lt=14] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=188) [2024-10-16 14:57:49.307247] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061869306266000, start_id=1729061869306266000, end_id=1729061869306266001) [2024-10-16 14:57:49.314258] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061869312663000, start_id=1729061869312663000, end_id=1729061869312663001) [2024-10-16 14:57:49.317924] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:49.317958] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1758) [2024-10-16 14:57:49.323429] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7926, mem_used=14558208) [2024-10-16 14:57:49.326861] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B24-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061869326144000, start_id=1729061869326144000, end_id=1729061869326144001) [2024-10-16 14:57:49.332156] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=12] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:49.338287] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:49.349887] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061869349792000, start_id=1729061869349792000, end_id=1729061869349792001) [2024-10-16 14:57:49.356545] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061869352946000, start_id=1729061869352946000, end_id=1729061869352946001) [2024-10-16 14:57:49.358226] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601434, offset=9634530}, end_cursor:ObLogCursor{file_id=192, log_id=47601435, offset=9634814}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db726e050, len:3358, log_data_len:284, seq:47601434, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:49.359855] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4AB-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061869359275000, start_id=1729061869359275000, end_id=1729061869359275001) [2024-10-16 14:57:49.361212] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601435, offset=9634814}, end_cursor:ObLogCursor{file_id=192, log_id=47601436, offset=9635088}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db734e050, len:3074, log_data_len:274, seq:47601435, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:49.363152] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601436, offset=9635088}, end_cursor:ObLogCursor{file_id=192, log_id=47601437, offset=9635362}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d3c050, len:2800, log_data_len:274, seq:47601436, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:49.363297] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601437, offset=9635362}, end_cursor:ObLogCursor{file_id=192, log_id=47601438, offset=9635636}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db736e050, len:2526, log_data_len:274, seq:47601437, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:49.363338] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561352, miss_cnt:1, cache_read_size:152590688349, hit_ratio:"9.999999928346925415e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:49.363364] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598940, miss_cnt:9, cache_read_size:149777921385, hit_ratio:"9.999999345925236760e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:49.367866] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.368329] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7807-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:49.368355] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7807-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:49.368361] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7807-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20575, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:49.370569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789DD-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061869370280000, start_id=1729061869370280000, end_id=1729061869370280001) [2024-10-16 14:57:49.371174] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB1-0-0] [lt=17] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:49.371188] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB1-0-0] [lt=10] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=14) [2024-10-16 14:57:49.380196] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.380311] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CE9-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:49.380733] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91BE-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:49.380742] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91BE-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9667, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:49.380745] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:49.380747] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91BE-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=264, avg_log_submit_to_slide_time=9659) [2024-10-16 14:57:49.380759] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650291, miss_cnt:32, cache_read_size:8050426822, hit_ratio:"9.999994892284912451e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:49.383138] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=15] blacklist_loop exec finished(cost_time=15, is_enabled=true, send_cnt=0) [2024-10-16 14:57:49.387736] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257782, offset=39038676}, end_cursor:ObLogCursor{file_id=197, log_id=49257783, offset=39038959}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7306050, len:4396, log_data_len:283, seq:49257782, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:49.389760] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB376-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061869389700000, start_id=1729061869389700000, end_id=1729061869389700001) [2024-10-16 14:57:49.390589] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257783, offset=39038959}, end_cursor:ObLogCursor{file_id=197, log_id=49257784, offset=39039241}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c98050, len:4113, log_data_len:282, seq:49257783, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:49.392521] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257784, offset=39039241}, end_cursor:ObLogCursor{file_id=197, log_id=49257785, offset=39039523}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dcc050, len:3831, log_data_len:282, seq:49257784, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:49.392656] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257785, offset=39039523}, end_cursor:ObLogCursor{file_id=197, log_id=49257786, offset=39039805}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72e2050, len:3549, log_data_len:282, seq:49257785, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:49.392711] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657029, miss_cnt:17, cache_read_size:8050174194, hit_ratio:"9.999997286817511322e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:49.394892] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:49.396726] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:49.396740] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:49.417698] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:49.417714] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:49.417992] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:49.418211] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=22, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:57:49.418219] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=8] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=965, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:57:49.418223] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:57:49.418231] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=8] [APPLY STAT CB EXECUTE TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=-559, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:57:49.418237] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB TOTAL TIME](cur_stat_count=20, stat_interval=5000000, avg (count/cost)=427, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:57:49.423107] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:49.440431] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:49.440460] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:49.440678] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:49.440941] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB APPEND COST TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=43, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:57:49.440967] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=25] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=1937, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:57:49.440971] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:57:49.440975] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=-552, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:57:49.440978] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=1428, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:57:49.443313] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061869442839000, start_id=1729061869442839000, end_id=1729061869442839001) [2024-10-16 14:57:49.444046] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1729061869053945001, v:0}, new_max_version={val:1729061869053945001, v:0}, last_min_version={val:1729061868649245003, v:0}, last_max_version={val:1729061868649245003, v:0}, total_time_us=1688, delta_us=390100) [2024-10-16 14:57:49.444077] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=30] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1729061869053945001, v:0}, min_version={val:1729061869053945001, v:0}, max_version={val:1729061869053945001, v:0}, in_service=true, leader_epoch=17) [2024-10-16 14:57:49.453094] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=183, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:49.453115] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:49.453159] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:49.453445] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91BF-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:49.453454] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91BF-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=625, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:49.453458] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91BF-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=214, avg_log_submit_to_slide_time=531) [2024-10-16 14:57:49.453459] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:49.458101] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061859457199, end_time=1729061869458099, dag_status={}) [2024-10-16 14:57:49.458117] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=14] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:49.458124] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:49.458128] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:49.458138] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:49.458146] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:49.458155] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:49.458158] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:49.458165] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:49.458168] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:49.458174] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458180] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458184] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458191] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458196] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458203] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458206] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458213] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458217] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458224] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458228] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458234] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458239] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458246] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458249] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458256] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458260] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458267] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458271] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458277] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458281] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458288] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458292] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458298] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458302] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458309] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458313] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458320] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458323] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458330] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458334] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458341] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458345] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458350] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458357] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458361] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458367] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458371] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458378] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458382] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458389] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458392] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458404] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458410] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458414] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458421] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458424] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458431] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458435] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458441] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458445] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458452] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458458] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458462] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458468] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458472] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:49.458480] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:49.458484] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:49.458490] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:49.458493] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:49.458498] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:49.458501] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:49.458503] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:49.458508] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:49.458510] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:49.458515] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:49.458518] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:49.464521] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:49.464536] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3968, avg_group_log_size=396) [2024-10-16 14:57:49.464546] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:49.466052] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21810][T1001_TntShared][T1001][Y0-0000000000000000-0-0] [lt=10] clear dtl interm result cost(us)(clear_cost=14, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:49.467912] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.468683] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=232, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:49.468703] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=110, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:49.468941] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:49.477875] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A87-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:49.477890] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A87-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:49.480257] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.483159] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8CA-0-0] [lt=7] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:49.483182] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8CA-0-0] [lt=18] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:49.483203] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8CA-0-0] [lt=18] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:49.490346] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:49.490363] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:49.490375] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70759853, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:49.490911] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:49.490920] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:49.490925] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A26-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:49.490935] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:49.490936] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A26-0-0] [lt=9] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:49.490941] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:49.490945] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:49.490947] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:49.490950] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:49.490953] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:49.490955] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:49.490958] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:49.490960] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:49.494049] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5447-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:49.494062] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5447-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=963, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:49.494061] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:49.497691] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061869496762000, start_id=1729061869496762000, end_id=1729061869496762001) [2024-10-16 14:57:49.503308] INFO [STORAGE.TRANS] refresh_elr_tenant_config_ (ob_tx_elr_util.cpp:47) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C87-0-0] [lt=11] refresh tenant config success(tenant_id=1004, *this={last_refresh_ts:1729061869503131, can_tenant_elr:true}) [2024-10-16 14:57:49.503467] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C87-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061869503131000, start_id=1729061869503131000, end_id=1729061869503131001) [2024-10-16 14:57:49.503537] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C87-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=2730, avg_group_log_size=227) [2024-10-16 14:57:49.503558] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C87-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2305) [2024-10-16 14:57:49.503979] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061869503131000, start_id=1729061869503131001, end_id=1729061869503131002) [2024-10-16 14:57:49.514231] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061869513683000, start_id=1729061869513683000, end_id=1729061869513683001) [2024-10-16 14:57:49.523053] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:7385763025}, base_lsn={lsn:7314419712}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:49.523108] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=54] get_replay_process success(id={id:1}, replayed_log_size=71343313, unreplayed_log_size=0) [2024-10-16 14:57:49.523116] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:57923794664}, base_lsn={lsn:57911414784}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:49.523143] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=26] get_replay_process success(id={id:1001}, replayed_log_size=12379880, unreplayed_log_size=0) [2024-10-16 14:57:49.523150] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=152596852765, unreplayed_log_size=0) [2024-10-16 14:57:49.523155] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1003}, replayed_log_size=149812351931, unreplayed_log_size=0) [2024-10-16 14:57:49.523173] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=6] dump tenant replay process(tenant_id=1002, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=288479, last_replayed_log_size(MB)=288479, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:49.538364] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:49.540455] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:49.540476] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:49.540876] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2194) [2024-10-16 14:57:49.541044] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C43D2-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:49.541053] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C43D2-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=550, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:49.541121] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C43D3-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=326, avg_log_submit_to_slide_time=681) [2024-10-16 14:57:49.543961] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=217, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:49.543982] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=222, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:49.543998] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10216) [2024-10-16 14:57:49.544009] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:49.549986] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061869549644000, start_id=1729061869549644000, end_id=1729061869549644001) [2024-10-16 14:57:49.552107] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=25] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:49.555118] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061869554926000, start_id=1729061869554926000, end_id=1729061869554926001) [2024-10-16 14:57:49.555143] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=507, get_gts_with_stc_cnt=78, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=76, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:49.555309] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=528, get_gts_with_stc_cnt=6, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:49.555323] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061869555304], gts:1729061869554926000, latest_srr:[mts=1729061869555304]}) [2024-10-16 14:57:49.556781] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061869555985000, start_id=1729061869555985000, end_id=1729061869555985001) [2024-10-16 14:57:49.557258] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1BE-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:49.557269] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1BE-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=676, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:49.557274] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1BE-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=463, avg_log_submit_to_slide_time=646) [2024-10-16 14:57:49.565311] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=17][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:49.567970] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.568741] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.569289] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.569311] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.569517] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.580309] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=43][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.596213] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=9] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1002}, {id:1001}]) [2024-10-16 14:57:49.596715] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:49.596737] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:49.596912] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=12] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:57:49.600553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B2E-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061869599429000, start_id=1729061869599429000, end_id=1729061869599429001) [2024-10-16 14:57:49.601577] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=42, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:49.601598] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=20] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=1692, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:49.601623] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=24] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:49.601629] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=5] [APPLY STAT CB EXECUTE TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=-620, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:49.601635] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=5] [APPLY STAT CB TOTAL TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=1114, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:49.603063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1E01-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061869602833000, start_id=1729061869602833000, end_id=1729061869602833001) [2024-10-16 14:57:49.606943] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D8-0-0] [lt=13] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061869574632, keepalive_ts:-1}) [2024-10-16 14:57:49.606979] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D8-0-0] [lt=35] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061869574632, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:49.609936] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5495, mem_used=10398720) [2024-10-16 14:57:49.623397] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:49.626754] WDIAG [OCCAM] runTimerTask (ob_occam_timer.h:163) [22261][T1002_MultiVers][T1002][Y0-0000000000000000-0-0] [lt=35][errcode=0] task scheduled too delay(scheduled_delay=1382638, expected_run_ts_us_=1729061868244114, ObClockGenerator::getRealClock()=1729061869626753, ret=0, ret="OB_SUCCESS", *this={this:0x2b0df5bf7910, caller:ob_multi_version_garbage_collector.cpp:start:115, function_type:ZN9oceanbase19concurrency_control30ObMultiVersionGarbageCollector5startEvE5$_938, timer_running_flag:1, total_running_count:1, func_shared_ptr_.ptr:0x2b0df5bf7bf0, time_wheel:0x2b0e0bafd990, thread_pool:0x2b0e0bafd170, is_running:1, time_interval:60.00s, expected_run_time:14:57:48.244, task_priority:1, with_handle_protected:1}) [2024-10-16 14:57:49.626841] INFO [MVCC] repeat_study (ob_multi_version_garbage_collector.cpp:196) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=5] skip repeat study(ret=0, this={this:0x2b0e0c3fcf30, last_study_timestamp:1729061329514560, last_refresh_timestamp:1729061809589879, last_reclaim_timestamp:1729061329516245, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061256125681004, v:0}, is_inited:true}, current_timestamp=1729061869626840, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:49.627586] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=16] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=1729061329511206000, snapshot_type=1, create_time=1729061329512245, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:49.627616] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=23] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=1729061693311908000, snapshot_type=1, create_time=1729061693314238, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:49.627624] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=1729061856983632000, snapshot_type=1, create_time=1729061856962275, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:49.627630] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=1729061328670161003, snapshot_type=2, create_time=1729061329512245, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:49.627636] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=1729061692497236003, snapshot_type=2, create_time=1729061693314238, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:49.627646] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=8] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=1729061856138655005, snapshot_type=2, create_time=1729061856962275, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:49.627651] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=1729061329302047000, snapshot_type=3, create_time=1729061329512245, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:49.627660] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=6] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=1729061692998783000, snapshot_type=3, create_time=1729061693314238, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:49.627666] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=1729061856983632000, snapshot_type=3, create_time=1729061856962275, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:49.627675] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=7] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061329512245, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:49.627680] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061693314238, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:49.627686] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1002 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061856962275, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:49.627803] INFO [MVCC] decide_reserved_snapshot_version_ (ob_multi_version_garbage_collector.cpp:618) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to update global reserved snapshot(ret=0, this={this:0x2b0e0c3fcf30, last_study_timestamp:1729061329514560, last_refresh_timestamp:1729061809589879, last_reclaim_timestamp:1729061329516245, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061256125681004, v:0}, is_inited:true}, global_reserved_snapshot_={val:1729061256125681004, v:0}, reserved_snapshot={val:1729061328670161003, v:0}) [2024-10-16 14:57:49.627824] INFO [MVCC] refresh_ (ob_multi_version_garbage_collector.cpp:552) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=21] multi-version garbage collector refresh successfully(this={this:0x2b0e0c3fcf30, last_study_timestamp:1729061329514560, last_refresh_timestamp:1729061809589879, last_reclaim_timestamp:1729061329516245, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061328670161003, v:0}, is_inited:true}, collector={reserved_snapshot_version:{val:1729061328670161003, v:0}, reserved_snapshot_type:2, reserved_status:-1, reserved_create_time:1729061329512245, reserved_addr:"10.0.4.7:2890", is_this_server_disabled:false, status:0}) [2024-10-16 14:57:49.627834] INFO [MVCC] repeat_refresh (ob_multi_version_garbage_collector.cpp:231) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=9] repeat refresh successfully(ret=0, this={this:0x2b0e0c3fcf30, last_study_timestamp:1729061329514560, last_refresh_timestamp:1729061869627833, last_reclaim_timestamp:1729061329516245, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061328670161003, v:0}, is_inited:true}, current_timestamp=1729061869626858, GARBAGE_COLLECT_RETRY_INTERVAL=60000000) [2024-10-16 14:57:49.627843] INFO [MVCC] repeat_reclaim (ob_multi_version_garbage_collector.cpp:266) [22260][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=8] skip repeat reclaim(ret=0, this={this:0x2b0e0c3fcf30, last_study_timestamp:1729061329514560, last_refresh_timestamp:1729061869627833, last_reclaim_timestamp:1729061329516245, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061328670161003, v:0}, is_inited:true}, current_timestamp=1729061869627842, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:49.631002] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=23] BlockGCTimerTask success(ret=0, cost_time_us=8, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:49.635101] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CB-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:49.635137] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CB-0-0] [lt=23] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061869149380004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061869149380004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:49.636204] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9CB-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061869636034000, start_id=1729061869636034000, end_id=1729061869636034001) [2024-10-16 14:57:49.638597] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CB-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061869638192000, start_id=1729061869638192000, end_id=1729061869638192001) [2024-10-16 14:57:49.638680] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CB-0-0] [lt=20] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10767, avg_group_log_size=598) [2024-10-16 14:57:49.638691] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CB-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10767) [2024-10-16 14:57:49.638936] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10767) [2024-10-16 14:57:49.639273] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:49.643467] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061869643466000, start_id=1729061869643466000, end_id=1729061869643466001) [2024-10-16 14:57:49.649899] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA1A-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061869649834000, start_id=1729061869649834000, end_id=1729061869649834001) [2024-10-16 14:57:49.649910] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA1C-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061869649834000, start_id=1729061869649834000, end_id=1729061869649834001) [2024-10-16 14:57:49.650069] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA1E-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061869649834000, start_id=1729061869649834000, end_id=1729061869649834001) [2024-10-16 14:57:49.650206] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA1B-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061869649834000, start_id=1729061869649834000, end_id=1729061869649834001) [2024-10-16 14:57:49.660760] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=16] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:49.667998] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.671820] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:49.680362] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.690764] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C58-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:49.690788] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C58-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061869150443004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061869150443004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:49.693468] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D7-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061869684716, keepalive_ts:-1}) [2024-10-16 14:57:49.693481] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D7-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061869684716, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:49.694386] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C58-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061869694141000, start_id=1729061869694141000, end_id=1729061869694141001) [2024-10-16 14:57:49.698553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061869698371000, start_id=1729061869698371000, end_id=1729061869698371001) [2024-10-16 14:57:49.698618] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4392) [2024-10-16 14:57:49.698818] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4392) [2024-10-16 14:57:49.698842] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4392) [2024-10-16 14:57:49.699014] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1072-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:49.699022] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1072-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=929, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:49.699177] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:49.702990] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:49.703685] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061869702582000, start_id=1729061869702582000, end_id=1729061869702582001) [2024-10-16 14:57:49.710431] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15E4-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:49.714291] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061869713301000, start_id=1729061869713301000, end_id=1729061869713301001) [2024-10-16 14:57:49.716013] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:49.716031] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=214) [2024-10-16 14:57:49.718400] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2FF-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:49.718429] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2FF-0-0] [lt=27] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=742, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:49.718434] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD2FF-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=235, avg_log_submit_to_slide_time=722) [2024-10-16 14:57:49.720049] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B21-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061869719446000, start_id=1729061869719446000, end_id=1729061869719446001) [2024-10-16 14:57:49.726794] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:49.738419] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:49.749999] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061869749192000, start_id=1729061869749192000, end_id=1729061869749192001) [2024-10-16 14:57:49.753015] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E72-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:49.753037] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E72-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:49.753238] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:49.756769] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061869756612000, start_id=1729061869756612000, end_id=1729061869756612001) [2024-10-16 14:57:49.757939] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:49.757963] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=12] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:49.757973] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=8] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:49.762133] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=15] LogLoopThread round_cost_time(us)(round_cost_time=12) [2024-10-16 14:57:49.768058] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.772147] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=15] gc stale ls task succ [2024-10-16 14:57:49.780422] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.782080] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=20] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:49.788160] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.788178] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.788187] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.788548] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.788571] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.788674] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.788683] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.789461] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.789620] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.789642] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.789660] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:49.797864] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061869496762000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=301096, in_cluster_service=true, cluster_version={val:1729061869496762000, v:0}, min_cluster_version={val:1729061869496762000, v:0}, max_cluster_version={val:1729061869496762000, v:0}, get_cluster_version_err=0, cluster_version_delta=301096, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65248979, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=367070573, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:49.823701] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:49.828852] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:57:49.828871] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=15] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:49.828888] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] cannot_recycle_log_size statistics(cannot_recycle_log_size=52503672, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:49.830923] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=20] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1003}, {id:1002}]) [2024-10-16 14:57:49.840954] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=240, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:49.840968] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=55, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:49.840978] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:49.843506] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061869842777000, start_id=1729061869842777000, end_id=1729061869842777001) [2024-10-16 14:57:49.855677] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=25] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}) [2024-10-16 14:57:49.859846] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5350, mem_used=10398720) [2024-10-16 14:57:49.867148] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061865851103, last_first_register_time_us=1722494157071428, register_time_us=1729061865851103, curr_time_us=1729061869867147) [2024-10-16 14:57:49.867177] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061869867164, last_submit_register_req_time_us=1729061869867164, register_parent_reason=3) [2024-10-16 14:57:49.867185] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:49.868520] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.870413] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:49.879371] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:49.880482] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.880635] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CF3-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:49.881142] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:49.881155] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:49.892301] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [21891][T1001_ObXAHbWor][T1001][Y0-0000000000000000-0-0] [lt=7] XA scheduler heartbeat task statistics(avg_time=1) [2024-10-16 14:57:49.893041] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D75-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:49.893073] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D75-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061869649834003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061869649834003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:49.894018] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=34514217, this={dio_aligned_buf:{buf_write_offset:1443, buf_padding_size:2653, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111299689, truncate_used_ts:40614834}, log_block_size:67108864, dir_fd:182, io_fd:127}, aligned_buf_len=4096, aligned_block_offset=34512896, buf_len=122, total_write_size_=12172, total_write_size_after_dio_=421888, ob_pwrite_used_ts=42216, count=100, each_pwrite_cost=422) [2024-10-16 14:57:49.894173] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D75-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061869894101000, start_id=1729061869894101000, end_id=1729061869894101001) [2024-10-16 14:57:49.898632] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061869897309000, start_id=1729061869897309000, end_id=1729061869897309001) [2024-10-16 14:57:49.898845] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:49.899547] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22483][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D75-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061869898762000, start_id=1729061869898762000, end_id=1729061869898762001) [2024-10-16 14:57:49.899591] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:49.899610] INFO [LIB] stat (utility.h:1165) [22483][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D75-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:49.899623] INFO [LIB] stat (utility.h:1165) [22483][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D75-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:49.899643] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=16] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061867249743000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:49.901915] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:49.914461] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061869913564000, start_id=1729061869913564000, end_id=1729061869913564001) [2024-10-16 14:57:49.918016] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:49.918031] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:49.918198] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=201, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:49.918199] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:49.918214] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=49, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:49.918230] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:49.918475] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD306-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:49.918482] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD306-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1117, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:49.923879] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:49.939164] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:49.939192] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=17] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:49.940984] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2620) [2024-10-16 14:57:49.941344] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41FC-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:49.941355] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F41FC-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=710, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:49.943694] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3500, mem_used=8318976) [2024-10-16 14:57:49.944773] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26142][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB25CC6BA-0-0] [lt=25] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:57:49.950144] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061869949327000, start_id=1729061869949327000, end_id=1729061869949327001) [2024-10-16 14:57:49.953175] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E76-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:49.953275] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:49.956895] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061869956738000, start_id=1729061869956738000, end_id=1729061869956738001) [2024-10-16 14:57:49.968597] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=46][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.968696] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7813-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:49.980541] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:49.980829] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:49.985955] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC735-0-0] [lt=14] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:49.985976] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC735-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:50.016656] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:50.016759] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAD-0-0] [lt=14] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061869949327000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061885590307000, latest_log_ts:{val:1729061868948338001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:50.016818] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAD-0-0] [lt=56] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:50.018127] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061870018125, bytes: 4446817433025, bw: 0.423891 MB/s, add_ts: 1003333, add_bytes: 445963 [2024-10-16 14:57:50.018916] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 237.50; ) [2024-10-16 14:57:50.018933] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:50.018971] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 6144.00, iops: 3.99, rt: 137.75; ) [2024-10-16 14:57:50.018976] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:50.018985] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:50.019431] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=305, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:50.019449] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=31, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:50.020022] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=445, avg_log_submit_to_slide_time=1221) [2024-10-16 14:57:50.023953] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:50.040798] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2620, avg_group_log_size=238) [2024-10-16 14:57:50.040825] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2620) [2024-10-16 14:57:50.040855] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:50.040865] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:50.041469] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C43E2-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=21, avg_log_submit_to_flush_time=279, avg_log_submit_to_slide_time=532) [2024-10-16 14:57:50.042732] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1004, min_version={val:1729061869554926001, v:0}, base_version={val:1729061869149380004, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061869554926001, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061870042717, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061869554926002, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061869995469, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061869554926002, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061870020399, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:50.043569] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22795][T1004_TenantWea][T1003][YB4A0A000407-00061E9B3AFB96BB-0-0] [lt=24] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.103558874130249023e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.008835578279190031e+03) [2024-10-16 14:57:50.043741] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061870042804000, start_id=1729061870042804000, end_id=1729061870042804001) [2024-10-16 14:57:50.045169] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=8] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:50.055472] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061870054950000, start_id=1729061870054950000, end_id=1729061870054950001) [2024-10-16 14:57:50.055495] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061870055467], gts:1729061870054950000, latest_srr:[mts=1729061870055467]}) [2024-10-16 14:57:50.055505] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:50.055525] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=543, get_gts_with_stc_cnt=82, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=78, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:50.068637] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:50.068665] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=27] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:50.068672] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.068998] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:50.080582] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.087188] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=70] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:50.087790] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:50.087839] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061860198021000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:50.090342] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:57:50.091106] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:50.091131] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:50.091149] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:50.091271] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061877008855000, latest_log_ts:{val:1729061860546474001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:50.091316] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=42] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:50.091329] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:57:50.091338] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061860546474001, v:0}) [2024-10-16 14:57:50.091342] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:50.091346] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:50.091359] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:50.091369] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:57:50.091377] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:50.091384] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:50.094147] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=25] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:50.098769] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061870097887000, start_id=1729061870097887000, end_id=1729061870097887001) [2024-10-16 14:57:50.114564] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061870113607000, start_id=1729061870113607000, end_id=1729061870113607001) [2024-10-16 14:57:50.118146] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:50.118186] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:50.126288] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=16] PNIO [ratelimit] time: 1729061870126286, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007613, add_bytes: 0 [2024-10-16 14:57:50.127501] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:50.127515] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:50.136871] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3318, mem_used=6239232) [2024-10-16 14:57:50.139253] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:50.144046] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15DC-0-0] [lt=9] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:50.144469] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=223, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:50.144487] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=363, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:50.145714] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:50.145878] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:50.146283] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15E2-0-0] [lt=6] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:50.150364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061870150206000, start_id=1729061870150206000, end_id=1729061870150206001) [2024-10-16 14:57:50.150428] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:50.150441] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:50.150479] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA23-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061870150206000, start_id=1729061870150206000, end_id=1729061870150206001) [2024-10-16 14:57:50.150490] INFO [STORAGE.TRANS] handle_request (ob_timestamp_service.cpp:175) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA23-0-0] [lt=10] handle gts request statistics(total_rt=2186, total_cnt=440, avg_rt=4.956916099773242657e+00, avg_cnt=4.400000000000000000e+01) [2024-10-16 14:57:50.150519] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA1F-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061870150206000, start_id=1729061870150206000, end_id=1729061870150206001) [2024-10-16 14:57:50.150643] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA20-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061870150206000, start_id=1729061870150206001, end_id=1729061870150206002) [2024-10-16 14:57:50.150746] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=34] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10362) [2024-10-16 14:57:50.150867] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA22-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061870150206000, start_id=1729061870150206000, end_id=1729061870150206001) [2024-10-16 14:57:50.150888] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0CAC-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061870150206000, start_id=1729061870150206000, end_id=1729061870150206001) [2024-10-16 14:57:50.151148] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get_max_applied_scn(scn={val:1729061869949327000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618604686}, palf_committed_end_scn_:{val:1729061870150206000, v:0}, last_check_scn_:{val:1729061870150206000, v:0}, max_applied_cb_scn_:{val:1729061869949327000, v:0}}) [2024-10-16 14:57:50.151167] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:50.154492] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=12] current monitor number(seq_=-1) [2024-10-16 14:57:50.157011] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061870156567000, start_id=1729061870156567000, end_id=1729061870156567001) [2024-10-16 14:57:50.168675] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.170817] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7817-0-0] [lt=33] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:50.175021] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:50.180730] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.193220] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5034-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:50.193237] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5034-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:50.193986] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8504B-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:50.194003] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8504B-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:50.218434] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1948) [2024-10-16 14:57:50.218546] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA888-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:50.218567] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA888-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061870150206001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061870150206001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:50.222255] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA888-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061870221282000, start_id=1729061870221282000, end_id=1729061870221282001) [2024-10-16 14:57:50.222314] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA888-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=10881, avg_group_log_size=544) [2024-10-16 14:57:50.222331] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA888-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10881) [2024-10-16 14:57:50.222573] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=203, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:50.222588] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10881) [2024-10-16 14:57:50.222610] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=489, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:50.222620] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=10, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:50.222713] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9569-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:50.222719] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9569-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=1001, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:50.222728] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9569-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=746, avg_log_submit_to_slide_time=970) [2024-10-16 14:57:50.222750] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=29] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:50.224253] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:50.240186] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DA0-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510338108579, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410612, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:50.241211] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:50.243654] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061870243453000, start_id=1729061870243453000, end_id=1729061870243453001) [2024-10-16 14:57:50.256231] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061870042804000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=213422, in_cluster_service=true, cluster_version={val:1729061869898762000, v:0}, min_cluster_version={val:1729061869898762000, v:0}, max_cluster_version={val:1729061869898762000, v:0}, get_cluster_version_err=0, cluster_version_delta=357464, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244295, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=479326029, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:50.269072] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.269194] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=47] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:57:50.269203] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=9] try gc retain ctx [2024-10-16 14:57:50.269207] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:50.269861] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061885590307000, latest_log_ts:{val:1729061868948338001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:50.272072] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C10C2-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061870271377000, start_id=1729061870271377000, end_id=1729061870271377001) [2024-10-16 14:57:50.280746] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CFB-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:50.280802] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.280980] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:50.280998] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:50.281031] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=35] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:50.294000] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8504D-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:50.294272] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=229, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:50.294293] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=93, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:50.294330] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:50.299023] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061870298011000, start_id=1729061870298011000, end_id=1729061870298011001) [2024-10-16 14:57:50.312264] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:50.312283] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:50.312298] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373656230, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:50.314697] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061870313876000, start_id=1729061870313876000, end_id=1729061870313876001) [2024-10-16 14:57:50.318418] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1948) [2024-10-16 14:57:50.318446] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:50.318895] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=6] ====== tenant freeze timer task ====== [2024-10-16 14:57:50.318908] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28020300, tenant_id=1002) [2024-10-16 14:57:50.318924] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:50.323524] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=7987, mem_used=14558208) [2024-10-16 14:57:50.332232] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=12] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:50.339334] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:50.344272] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:50.344284] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=53173860, tenant_id=1003) [2024-10-16 14:57:50.344305] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=17] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:50.349618] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=40] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061870150206000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199405, in_cluster_service=true, cluster_version={val:1729061869749192001, v:0}, min_cluster_version={val:1729061869749192001, v:0}, max_cluster_version={val:1729061869749192001, v:0}, get_cluster_version_err=0, cluster_version_delta=600419, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244542, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=244745782, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:50.350488] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061870349808000, start_id=1729061870349808000, end_id=1729061870349808001) [2024-10-16 14:57:50.354761] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:50.357168] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061870356181000, start_id=1729061870356181000, end_id=1729061870356181001) [2024-10-16 14:57:50.364473] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601438, offset=9635636}, end_cursor:ObLogCursor{file_id=192, log_id=47601439, offset=9635920}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dd0050, len:2252, log_data_len:284, seq:47601438, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:50.366080] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22067][T1002_TeRec][T1002][YB4A0A000407-00061E9B225FA4AC-0-0] [lt=18] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=155822996, MAX_GAP=900000000000, new_sync_scn={val:1729061869306266000, v:0}, new_replayable_scn={val:1729061869306266000, v:0}, new_readable_scn={val:1729061869150443004, v:0}, old_tenant_info={tenant_id:1002, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061868253362000, v:0}, replayable_scn:{val:1729061868253362000, v:0}, standby_scn:{val:1729061868052931001, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"NOARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:57:50.366264] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4AC-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061870364666000, start_id=1729061870364666000, end_id=1729061870364666001) [2024-10-16 14:57:50.367161] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=28] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601439, offset=9635920}, end_cursor:ObLogCursor{file_id=192, log_id=47601440, offset=9636194}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e5c050, len:1968, log_data_len:274, seq:47601439, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:50.368936] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE781B-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:50.368959] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE781B-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:50.368964] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE781B-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20410, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:50.368999] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561363, miss_cnt:1, cache_read_size:152590689682, hit_ratio:"9.999999928346930966e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:50.369125] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601440, offset=9636194}, end_cursor:ObLogCursor{file_id=192, log_id=47601441, offset=9636468}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20ac050, len:1694, log_data_len:274, seq:47601440, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:50.369167] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.369339] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601441, offset=9636468}, end_cursor:ObLogCursor{file_id=192, log_id=47601442, offset=9636742}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72ce050, len:1420, log_data_len:274, seq:47601441, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:50.369402] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598950, miss_cnt:9, cache_read_size:149777922596, hit_ratio:"9.999999345925284500e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:50.377356] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22442][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=16][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:50.377417] WDIAG read_multi_row (ob_table_access_helper.h:238) [22442][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=21][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:50.378239] INFO delete_row (ob_table_access_helper.h:312) [22442][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1003, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061870377427) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061870377427) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:50.380800] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2CFD-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:50.380867] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.381012] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:50.381028] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=10] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=216) [2024-10-16 14:57:50.381248] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91D2-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:50.381257] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91D2-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10211, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:50.381261] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91D2-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=238, avg_log_submit_to_slide_time=10205) [2024-10-16 14:57:50.381259] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:50.381271] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650301, miss_cnt:32, cache_read_size:8050428033, hit_ratio:"9.999994892285728465e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:50.393582] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257786, offset=39039805}, end_cursor:ObLogCursor{file_id=197, log_id=49257787, offset=39040088}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e28050, len:3267, log_data_len:283, seq:49257786, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:50.394953] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:50.395712] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB377-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061870393856000, start_id=1729061870393856000, end_id=1729061870393856001) [2024-10-16 14:57:50.396641] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=26] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257787, offset=39040088}, end_cursor:ObLogCursor{file_id=197, log_id=49257788, offset=39040370}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e7c050, len:2984, log_data_len:282, seq:49257787, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:50.398552] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257788, offset=39040370}, end_cursor:ObLogCursor{file_id=197, log_id=49257789, offset=39040652}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e04050, len:2702, log_data_len:282, seq:49257788, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:50.398789] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257789, offset=39040652}, end_cursor:ObLogCursor{file_id=197, log_id=49257790, offset=39040934}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db737a050, len:2420, log_data_len:282, seq:49257789, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:50.398840] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657039, miss_cnt:17, cache_read_size:8050175414, hit_ratio:"9.999997286817944309e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:50.418310] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:50.418330] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:50.418510] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:50.418670] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:50.424515] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:50.441036] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:50.441053] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:57:50.441309] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:57:50.443843] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061870442183000, start_id=1729061870442183000, end_id=1729061870442183001) [2024-10-16 14:57:50.446132] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1729061870054950001, v:0}, new_max_version={val:1729061870054950001, v:0}, last_min_version={val:1729061869554926003, v:0}, last_max_version={val:1729061869554926003, v:0}, total_time_us=3190, delta_us=391181) [2024-10-16 14:57:50.446154] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1729061870054950001, v:0}, min_version={val:1729061870054950001, v:0}, max_version={val:1729061870054950001, v:0}, in_service=true, leader_epoch=17) [2024-10-16 14:57:50.453806] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=191, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:50.453827] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:50.453859] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:50.454080] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD316-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:50.454092] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD316-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=467, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:50.454092] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:50.454097] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD316-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=227, avg_log_submit_to_slide_time=459) [2024-10-16 14:57:50.466621] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:50.466645] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=4515, avg_group_log_size=410) [2024-10-16 14:57:50.466669] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:50.466895] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=221, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:50.466914] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=35, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:50.466933] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:50.467377] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C107A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=345, avg_log_submit_to_slide_time=632) [2024-10-16 14:57:50.469220] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.469264] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=251, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:50.469278] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=117, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:50.469472] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:50.469505] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=37, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:50.469510] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:50.479279] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A88-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:50.479306] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A88-0-0] [lt=25] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:50.480931] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.489415] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD011-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061870480483, keepalive_ts:-1}) [2024-10-16 14:57:50.489442] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD011-0-0] [lt=26] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061870480483, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:50.494167] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85051-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:50.494183] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85051-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22412, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:50.494216] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:50.499261] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] submit log success(service_type_=0, last_id=1729061887008855000, limited_id=1729061887008855000) [2024-10-16 14:57:50.499273] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] submit log with lock success(service_type_=0, last_id=1729061887008855000, limited_id=1729061887008855000) [2024-10-16 14:57:50.499277] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061870498146000, start_id=1729061870498146000, end_id=1729061870498146001) [2024-10-16 14:57:50.499791] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=6] update id meta(ret=0, service_type=0, limited_id=1729061887008855000, latest_scn={val:1729061870498146001, v:0}, *this={tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061887008855000, latest_log_ts:{val:1729061870498146001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:50.499860] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=68] handle submit callback(service_type_=0, success=true, used_ts=600, log_ts={val:1729061870498146001, v:0}, limited_id=1729061887008855000) [2024-10-16 14:57:50.499878] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=17] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:50.499889] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=11] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:50.500880] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3D99-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061870500292000, start_id=1729061870500292000, end_id=1729061870500292001) [2024-10-16 14:57:50.500978] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get_max_applied_scn(scn={val:1729061870298011000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390158270}, palf_committed_end_scn_:{val:1729061870498146002, v:0}, last_check_scn_:{val:1729061870498146002, v:0}, max_applied_cb_scn_:{val:1729061870298011000, v:0}}) [2024-10-16 14:57:50.501000] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:50.514946] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061870513101000, start_id=1729061870513101000, end_id=1729061870513101001) [2024-10-16 14:57:50.518346] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:57:50.518371] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1523) [2024-10-16 14:57:50.533721] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:57:50.533738] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:50.533760] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=10] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444572542, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:50.539409] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:50.541055] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:50.541080] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:50.541383] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2217) [2024-10-16 14:57:50.541692] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C43EE-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:50.541700] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C43EE-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1120, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:50.541747] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F420D-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=819, avg_log_submit_to_slide_time=1047) [2024-10-16 14:57:50.542992] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=7] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061870054950002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=488037, in_cluster_service=true, cluster_version={val:1729061870054950001, v:0}, min_cluster_version={val:1729061870054950001, v:0}, max_cluster_version={val:1729061870054950001, v:0}, get_cluster_version_err=0, cluster_version_delta=488037, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228322, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=415934508, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:50.545982] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=208, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:50.546000] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=162, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:50.546024] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10673) [2024-10-16 14:57:50.546057] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=30] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:50.550668] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061870548905000, start_id=1729061870548905000, end_id=1729061870548905001) [2024-10-16 14:57:50.555859] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061870555071000, start_id=1729061870555071000, end_id=1729061870555071001) [2024-10-16 14:57:50.555878] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=16] ts response task statistics(alloc_count=261364790, free_count=130682394) [2024-10-16 14:57:50.557443] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061870557230000, start_id=1729061870557230000, end_id=1729061870557230001) [2024-10-16 14:57:50.558138] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1D0-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:50.558154] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1D0-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=724, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:50.558159] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1D0-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=405, avg_log_submit_to_slide_time=694) [2024-10-16 14:57:50.568683] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.569294] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.569323] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.569345] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.569541] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.581044] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.589512] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD012-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061870580536, keepalive_ts:-1}) [2024-10-16 14:57:50.589532] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD012-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061870580536, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:50.596262] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1}, {id:1003}]) [2024-10-16 14:57:50.596671] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:50.596688] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:50.608668] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=21] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:50.610076] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=5511, mem_used=10398720) [2024-10-16 14:57:50.614034] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061870150206003, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=463821, in_cluster_service=true, cluster_version={val:1729061870150206001, v:0}, min_cluster_version={val:1729061870150206001, v:0}, max_cluster_version={val:1729061870150206001, v:0}, get_cluster_version_err=0, cluster_version_delta=463821, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244244, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=541943866, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:50.624802] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:50.626864] INFO [CLOG] update_upstream (ob_remote_location_adaptor.cpp:91) [22156][T1002_LogRessvr][T1002][YB4A0A000407-00061E996E7960C1-0-0] [lt=9] primary tenant, just skip [2024-10-16 14:57:50.642542] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CC-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:50.642572] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CC-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061870150206004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061870150206004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:50.644041] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061870642614000, start_id=1729061870642614000, end_id=1729061870642614001) [2024-10-16 14:57:50.644112] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9962, avg_group_log_size=586) [2024-10-16 14:57:50.644131] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10124) [2024-10-16 14:57:50.644169] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9CC-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061870642614000, start_id=1729061870642614001, end_id=1729061870642614002) [2024-10-16 14:57:50.644333] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9962) [2024-10-16 14:57:50.649676] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CC-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061870649442000, start_id=1729061870649442000, end_id=1729061870649442001) [2024-10-16 14:57:50.650278] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:50.650793] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA24-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061870650472000, start_id=1729061870650472000, end_id=1729061870650472001) [2024-10-16 14:57:50.650798] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA26-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061870650472000, start_id=1729061870650472000, end_id=1729061870650472001) [2024-10-16 14:57:50.650949] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA28-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061870650472000, start_id=1729061870650472000, end_id=1729061870650472001) [2024-10-16 14:57:50.651127] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA25-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061870650472000, start_id=1729061870650472000, end_id=1729061870650472001) [2024-10-16 14:57:50.666669] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:57:50.666685] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=15] try gc retain ctx [2024-10-16 14:57:50.666688] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:50.667305] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061887008855000, latest_log_ts:{val:1729061870498146001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:50.669390] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.671922] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:50.681110] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.696869] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DA2-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510338565262, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410613, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:50.697057] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C59-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:50.697083] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C59-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061870650472001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061870650472001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:50.697723] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DA6-0-0] [lt=15][errcode=-4201] hashtable not init [2024-10-16 14:57:50.697740] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DA6-0-0] [lt=17][errcode=-4016] get cursor info failed(cursor_id=1116, get_sessid()=3221565766) [2024-10-16 14:57:50.697830] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DA7-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061870697343000, start_id=1729061870697343000, end_id=1729061870697343001) [2024-10-16 14:57:50.699183] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1729061870697343000, start_id=1729061870697343001, end_id=1729061870697343002) [2024-10-16 14:57:50.699283] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C47-0-0] [lt=19] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:50.699292] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C47-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=810, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:50.699484] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:50.699589] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D48-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061870690273, keepalive_ts:-1}) [2024-10-16 14:57:50.699610] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D48-0-0] [lt=21] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061870690273, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:50.699629] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4764) [2024-10-16 14:57:50.699653] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4764) [2024-10-16 14:57:50.699745] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D47-0-0] [lt=10] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061870690233, keepalive_ts:-1}) [2024-10-16 14:57:50.699754] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D47-0-0] [lt=8] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061870690233, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:50.702497] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C59-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061870702459000, start_id=1729061870702459000, end_id=1729061870702459001) [2024-10-16 14:57:50.708689] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22141][T1002_TntShared][T1002][Y0-0000000000000000-0-0] [lt=19] clear dtl interm result cost(us)(clear_cost=922, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:50.714870] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061870714055000, start_id=1729061870714055000, end_id=1729061870714055001) [2024-10-16 14:57:50.718933] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD31E-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:50.718958] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD31E-0-0] [lt=25] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=614, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:50.726895] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:50.736143] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=5] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:50.736168] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=16] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=216) [2024-10-16 14:57:50.740915] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2264904, replace_num=31457) [2024-10-16 14:57:50.740932] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:50.750753] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061870749847000, start_id=1729061870749847000, end_id=1729061870749847001) [2024-10-16 14:57:50.753575] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E86-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:50.753606] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E86-0-0] [lt=19] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:50.753798] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:50.754273] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15E6-0-0] [lt=5] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:50.755869] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:50.757360] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061870757250000, start_id=1729061870757250000, end_id=1729061870757250001) [2024-10-16 14:57:50.758074] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:50.758093] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:50.758103] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:50.758211] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get_max_applied_scn(scn={val:1729061870649442000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264829025}, palf_committed_end_scn_:{val:1729061870757250000, v:0}, last_check_scn_:{val:1729061870757250000, v:0}, max_applied_cb_scn_:{val:1729061870649442000, v:0}}) [2024-10-16 14:57:50.758235] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:50.764509] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488F-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061870732266, keepalive_ts:-1}) [2024-10-16 14:57:50.764525] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C488F-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061870732266, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:50.769443] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.772189] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:50.781173] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.782178] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:50.788072] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.788090] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.788535] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.788558] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.788575] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.788576] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.788586] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.789179] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.789400] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.789596] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.789617] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:50.819126] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=27] [APPLY STAT CB APPEND COST TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=45, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:57:50.819142] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=15] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=749, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:57:50.819147] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:57:50.819150] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=-623, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:57:50.819154] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=171, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:57:50.819154] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF7E-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=348, avg_log_submit_to_slide_time=598) [2024-10-16 14:57:50.825104] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:50.831006] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1001}, {id:1002}]) [2024-10-16 14:57:50.841517] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=397, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:50.841535] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=419, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:50.841616] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:50.844141] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061870843453000, start_id=1729061870843453000, end_id=1729061870843453001) [2024-10-16 14:57:50.849975] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=37] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:50.850591] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:50.850647] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=18] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061864434291000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:50.859981] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5406, mem_used=10398720) [2024-10-16 14:57:50.866806] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=56] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=5351) [2024-10-16 14:57:50.869502] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.881110] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D07-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:50.881216] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.881669] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:50.881682] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:50.883316] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=10] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:50.889736] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:50.889919] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D49-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061870555071001, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061868253362000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880098662000, latest_log_ts:{val:1729061863549302001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:50.890055] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D49-0-0] [lt=92] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061870555071001, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061868253362000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:50.890125] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D49-0-0] [lt=35] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061870650472003, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061868253362000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:50.890201] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D49-0-0] [lt=42] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061870650472001, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061868253362000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:50.890234] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D49-0-0] [lt=32] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:50.897665] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=42] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:50.898254] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=13] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:50.898297] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061861837115000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:50.899285] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061870898528000, start_id=1729061870898528000, end_id=1729061870898528001) [2024-10-16 14:57:50.901990] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:50.902152] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D76-0-0] [lt=26] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:50.902174] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D76-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061870651542002, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061870651542002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:50.903177] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D76-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061870902783000, start_id=1729061870902783000, end_id=1729061870902783001) [2024-10-16 14:57:50.905551] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D76-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061870904868000, start_id=1729061870904868000, end_id=1729061870904868001) [2024-10-16 14:57:50.905618] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D76-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:50.905641] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D76-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:50.915009] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061870914897000, start_id=1729061870914897000, end_id=1729061870914897001) [2024-10-16 14:57:50.918541] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:50.918555] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:50.918804] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=403, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:50.918816] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:50.918809] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:50.918861] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:50.919071] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF82-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:50.919078] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF82-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=721, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:50.923938] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:50.934433] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE5-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061870902136, keepalive_ts:-1}) [2024-10-16 14:57:50.934451] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE5-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061870902136, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:50.941004] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=9] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:50.941902] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4218-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:50.941916] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4218-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1116, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:50.943787] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3539, mem_used=8318976) [2024-10-16 14:57:50.949689] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:50.949707] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:50.949715] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=151) [2024-10-16 14:57:50.950712] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061870950241000, start_id=1729061870950241000, end_id=1729061870950241001) [2024-10-16 14:57:50.953846] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E8A-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:50.957504] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=50] get number(ret=0, service_type_=0, range=1, base_id=1729061870955585000, start_id=1729061870955585000, end_id=1729061870955585001) [2024-10-16 14:57:50.958043] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB APPEND COST TIME](cur_stat_count=35, stat_interval=5000000, avg (count/cost)=40, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:57:50.958058] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=15] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=35, stat_interval=5000000, avg (count/cost)=1100, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:57:50.958062] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=35, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:57:50.958065] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=35, stat_interval=5000000, avg (count/cost)=-742, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:57:50.958068] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=35, stat_interval=5000000, avg (count/cost)=398, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:57:50.964897] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:50.964913] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:57:50.964923] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:50.968241] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:50.969194] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7827-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:50.969462] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:50.969616] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.976976] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=8] LogLoopThread round_cost_time(us)(round_cost_time=9) [2024-10-16 14:57:50.981287] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:50.981629] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:51.007175] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:57:51.007189] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151784220, tenant_id=1) [2024-10-16 14:57:51.007202] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:51.018722] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061871018721, bytes: 4446817777210, bw: 0.328045 MB/s, add_ts: 1000596, add_bytes: 344185 [2024-10-16 14:57:51.020682] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 156.75; ) [2024-10-16 14:57:51.020711] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=26] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:51.020745] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 156.75; ) [2024-10-16 14:57:51.020754] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:51.020764] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:51.024280] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061871023972000, start_id=1729061871023972000, end_id=1729061871023972001) [2024-10-16 14:57:51.025439] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:51.028415] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Cache wash clean map node details(ret=0, clean_node_count=15, clean_time=0, clean_start_pos=1321194, clean_num=62914) [2024-10-16 14:57:51.041269] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1791, avg_group_log_size=199) [2024-10-16 14:57:51.041286] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1791) [2024-10-16 14:57:51.041310] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:51.041320] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:51.041709] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1913) [2024-10-16 14:57:51.041888] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F421B-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=823, avg_log_submit_to_slide_time=1133) [2024-10-16 14:57:51.042769] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:51.042784] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:51.042797] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=291963755, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:51.044653] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061871043039000, start_id=1729061871043039000, end_id=1729061871043039001) [2024-10-16 14:57:51.056264] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061871055408000, start_id=1729061871055408000, end_id=1729061871055408001) [2024-10-16 14:57:51.056287] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:51.056304] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061871056301], gts:1729061871055408000, latest_srr:[mts=1729061871056301]}) [2024-10-16 14:57:51.062425] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:468) [21535][T1_SSTableDefra][T1][Y0-0000000000000000-0-0] [lt=6] skip defragment task(ret=0, need_defragment=false) [2024-10-16 14:57:51.062447] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [21535][T1_SSTableDefra][T1][Y0-0000000000000000-0-0] [lt=22] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=15) [2024-10-16 14:57:51.062453] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [21535][T1_SSTableDefra][T1][Y0-0000000000000000-0-0] [lt=4] successfully defragment data blocks(rewrite_cnt=0) [2024-10-16 14:57:51.062458] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [21535][T1_SSTableDefra][T1][Y0-0000000000000000-0-0] [lt=4] tablet skip filter destructed(total_cnt=0, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105368325, total_skip_cnt=310297, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:57:51.069428] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:51.069689] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.081352] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.093191] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:57:51.093370] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D47-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061870677946003, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061868052931002, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880721265000, latest_log_ts:{val:1729061864144741001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:51.093543] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D47-0-0] [lt=88] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061870677946003, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061868052931002, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:51.093691] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D47-0-0] [lt=74] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061870651542004, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061868052931002, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:51.093830] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D47-0-0] [lt=67] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061870650472004, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061868052931002, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:51.093892] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D47-0-0] [lt=61] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:51.094608] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:51.098554] INFO [SQL.PC] need_check_schema_version (ob_plan_cache_value.cpp:2237) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B710F1615-0-0] [lt=11] need check schema(new_schema_version=1724725354537200, cached_tenant_schema_version=1724725354537200, is_contain_synonym()=false, contain_sys_name_table_=true, is_contain_tmp_tbl()=false, is_contain_sys_pl_object()=false, pc_ctx.sql_ctx_.session_info_->get_has_temp_table_flag()=false, need_check=true, constructed_sql_=SELECT * FROM __all_ls_meta_table WHERE tenant_id = ? ORDER BY tenant_id, ls_id, svr_ip, svr_port) [2024-10-16 14:57:51.099413] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3AD0B0-0-0] [lt=20] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.736266231536865234e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=868, Throttle Trigger(MB)=520, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.132845141145328682e+01) [2024-10-16 14:57:51.099569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061871098151000, start_id=1729061871098151000, end_id=1729061871098151001) [2024-10-16 14:57:51.115183] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061871114567000, start_id=1729061871114567000, end_id=1729061871114567001) [2024-10-16 14:57:51.118649] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:51.118669] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:51.118884] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:51.118924] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=37] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:51.119153] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD329-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=281, avg_log_submit_to_slide_time=552) [2024-10-16 14:57:51.127634] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:51.127648] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:51.136962] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3361, mem_used=6239232) [2024-10-16 14:57:51.139079] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061871139077, bytes: 0, bw: 0.000000 MB/s, add_ts: 1012791, add_bytes: 0 [2024-10-16 14:57:51.141077] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:51.145936] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:51.151138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA2A-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061871150609000, start_id=1729061871150609000, end_id=1729061871150609001) [2024-10-16 14:57:51.151192] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA2B-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061871150609000, start_id=1729061871150609000, end_id=1729061871150609001) [2024-10-16 14:57:51.151193] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061871150609000, start_id=1729061871150609001, end_id=1729061871150609002) [2024-10-16 14:57:51.151254] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:51.151270] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:51.151351] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA2C-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061871150609000, start_id=1729061871150609000, end_id=1729061871150609001) [2024-10-16 14:57:51.151441] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA29-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061871150609000, start_id=1729061871150609000, end_id=1729061871150609001) [2024-10-16 14:57:51.151471] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA2D-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061871150609000, start_id=1729061871150609000, end_id=1729061871150609001) [2024-10-16 14:57:51.151597] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10447) [2024-10-16 14:57:51.157515] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061871156910000, start_id=1729061871156910000, end_id=1729061871156910001) [2024-10-16 14:57:51.158430] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:51.169750] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.170210] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=4] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:51.170225] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=8] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:51.170232] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:51.170235] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:51.170238] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:51.170242] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16727) [2024-10-16 14:57:51.181416] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.194008] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5035-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:51.194026] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5035-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:51.194444] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8505F-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:51.194456] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8505F-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:51.196232] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=13] set tenant work area memory(tenant_id=1001, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:51.196288] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=13] set tenant work area memory(tenant_id=1002, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:51.196329] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7] set tenant work area memory(tenant_id=1003, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:51.196367] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6] set tenant work area memory(tenant_id=1004, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:57:51.196552] EDIAG [SHARE.CONFIG] check_500_tenant_hold (ob_server_config.cpp:438) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] the hold memory of tenant_500 is over the reserved memory(hold=1082318848, reserved=1073741824) BACKTRACE:0x1a5796b1 0x6847325 0x6a4dfba 0x6a4d9a4 0x6a4d8e0 0x6a4d707 0x1479497f 0x11dea49b 0x6809ea1 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d [2024-10-16 14:57:51.196570] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:32) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=17] === Run print tenant memory usage task === [2024-10-16 14:57:51.196613] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:102) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4] ====== tenants memory info ====== === TENANTS MEMORY INFO === divisive_memory_used= 220,811,264 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,082,318,848 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 19,554,304 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 17,457,152 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 1 now= 1,729,061,871,195,969 active_memstore_used= 247,463,936 total_memstore_used= 245,366,784 total_memstore_hold= 245,366,784 memstore_freeze_trigger_limit= 151,784,220 memstore_limit= 858,993,440 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 1,982,304,256 max_mem_memstore_can_get_now= 758,921,184 memstore_alloc_pos= 20,002,635,776 memstore_frozen_pos= 19,755,171,840 memstore_reclaimed_pos= 19,757,268,992 [TENANT_MEMORY] tenant_id= 1,001 now= 1,729,061,871,195,969 active_memstore_used= 50,331,648 total_memstore_used= 46,137,344 total_memstore_hold= 46,137,344 memstore_freeze_trigger_limit= 54,222,420 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 909,004,800 max_mem_memstore_can_get_now= 271,112,144 memstore_alloc_pos= 32,719,765,504 memstore_frozen_pos= 32,669,433,856 memstore_reclaimed_pos= 32,673,628,160 [TENANT_MEMORY] tenant_id= 1,002 now= 1,729,061,871,195,969 active_memstore_used= 44,040,192 total_memstore_used= 44,040,192 total_memstore_hold= 44,040,192 memstore_freeze_trigger_limit= 28,020,300 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 969,699,328 max_mem_memstore_can_get_now= 140,101,592 memstore_alloc_pos= 465,951,522,816 memstore_frozen_pos= 465,907,482,624 memstore_reclaimed_pos= 465,907,482,624 [TENANT_MEMORY] tenant_id= 1,003 now= 1,729,061,871,195,969 active_memstore_used= 60,817,408 total_memstore_used= 60,817,408 total_memstore_hold= 60,817,408 memstore_freeze_trigger_limit= 53,173,860 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 942,559,232 max_mem_memstore_can_get_now= 265,869,308 memstore_alloc_pos= 34,273,755,136 memstore_frozen_pos= 34,212,937,728 memstore_reclaimed_pos= 34,212,937,728 [TENANT_MEMORY] tenant_id= 1,004 now= 1,729,061,871,195,969 active_memstore_used= 39,845,888 total_memstore_used= 39,845,888 total_memstore_hold= 39,845,888 memstore_freeze_trigger_limit= 27,418,600 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 958,939,136 max_mem_memstore_can_get_now= 137,093,088 memstore_alloc_pos= 1,602,224,128 memstore_frozen_pos= 1,562,378,240 memstore_reclaimed_pos= 1,562,378,240 [2024-10-16 14:57:51.196672] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:114) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=10] [CHUNK_MGR] limit= 12,884,901,888 hold= 7,747,866,624 total_hold= 8,638,169,088 used= 6,881,837,056 freelists_hold= 866,029,568 total_maps= 388,959 total_unmaps= 385,794 large_maps= 301 large_unmaps= 0 huge_maps= 200 huge_unmaps= 187 memalign=0 resident_size= 8,395,403,264 virtual_memory_used= 9,720,012,800 [CHUNK_MGR] 2 MB_CACHE: hold= 125,829,120 free= 60 pushes= 6,196,535,238 pops= 6,196,535,178 maps= 388,458 unmaps= 385,607 [CHUNK_MGR] 4 MB_CACHE: hold= 167,227,392 free= 73 pushes= 6,821,144 pops= 6,821,071 maps= 172 unmaps= 0 [CHUNK_MGR] 6 MB_CACHE: hold= 193,880,064 free= 46 pushes= 6,920 pops= 6,874 maps= 59 unmaps= 0 [CHUNK_MGR] 8 MB_CACHE: hold= 25,247,744 free= 4 pushes= 16,248,833 pops= 16,248,829 maps= 11 unmaps= 0 [CHUNK_MGR] 10 MB_CACHE: hold= 109,318,144 free= 13 pushes= 123 pops= 110 maps= 30 unmaps= 0 [CHUNK_MGR] 12 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [CHUNK_MGR] 14 MB_CACHE: hold= 26,435,584 free= 2 pushes= 4 pops= 2 maps= 9 unmaps= 0 [CHUNK_MGR] 16 MB_CACHE: hold= 33,316,864 free= 2 pushes= 4 pops= 2 maps= 2 unmaps= 0 [CHUNK_MGR] 18 MB_CACHE: hold= 184,774,656 free= 11 pushes= 993 pops= 982 maps= 18 unmaps= 0 [CHUNK_MGR] 20 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [2024-10-16 14:57:51.196711] INFO print (ob_malloc_time_monitor.cpp:39) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8] [MALLOC_TIME_MONITOR] show the distribution of ob_malloc's cost_time [MALLOC_TIME_MONITOR] [ 0, 10): delta_total_cost_time= 126691, delta_count= 666296, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 10, 100): delta_total_cost_time= 3186, delta_count= 207, avg_cost_time= 15 [MALLOC_TIME_MONITOR] [ 100, 1000): delta_total_cost_time= 1674, delta_count= 6, avg_cost_time= 279 [MALLOC_TIME_MONITOR] [ 1000, 10000): delta_total_cost_time= 3235, delta_count= 2, avg_cost_time= 1617 [MALLOC_TIME_MONITOR] [ 10000, 100000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 100000, 1000000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 1000000, 9223372036854775807): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [2024-10-16 14:57:51.219344] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD32C-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:51.224511] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA889-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:51.224532] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA889-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061871150609000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061871150609000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:51.228744] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1171185664, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:51.228851] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA889-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061871227918000, start_id=1729061871227918000, end_id=1729061871227918001) [2024-10-16 14:57:51.228918] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA889-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=9928, avg_group_log_size=522) [2024-10-16 14:57:51.228927] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA889-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9928) [2024-10-16 14:57:51.229837] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=225, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:51.229849] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9928) [2024-10-16 14:57:51.229859] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=80, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:51.229881] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:51.229886] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=563, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:51.229889] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=324, avg_log_submit_to_slide_time=532) [2024-10-16 14:57:51.229901] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:51.229942] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=14] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:51.235715] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:57:51.235728] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=11] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:57:51.235742] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:57:51.235747] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:51.237807] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=86, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}, {obj_id:78893917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:78893918, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893919, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893920, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}]) [2024-10-16 14:57:51.237930] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=121] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:51.243828] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DA8-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510339112215, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410614, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:51.244863] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061871241265000, start_id=1729061871241265000, end_id=1729061871241265001) [2024-10-16 14:57:51.247596] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4403-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:51.247819] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=9] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:57:51.248651] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061871247628000, start_id=1729061871247628000, end_id=1729061871247628001) [2024-10-16 14:57:51.252680] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=5] no need to update min/max version(new_version={val:1729061870950241000, v:0}, min_version={val:1729061870950241000, v:0}, max_version={val:1729061870950241000, v:0}) [2024-10-16 14:57:51.256161] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1203, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:51.256179] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=573, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:51.256207] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=39] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:51.256204] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:17959712122}], scn_array=[cnt:1, {val:1729061871055408001, v:0}], curr_size=122, accum_size=244, time_cost=10099) [2024-10-16 14:57:51.256328] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=23] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:51.264293] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=25] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:51.269630] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE782D-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:51.269654] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE782D-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938365, log_task={header:{begin_lsn:{lsn:213058250676}, end_lsn:{lsn:213058250798}, log_id:220938365, min_scn:{val:1729061871151670000, v:0}, max_scn:{val:1729061871151670000, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058250554}, prev_proposal_id:88, committed_end_lsn:{lsn:213058250676}, data_checksum:1715074492, accum_checksum:1822754201, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061871169290, freeze_ts:1729061871169290, submit_ts:1729061871169294, flushed_ts:1729061871173891, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:4597}, fs_cb_begin_ts=1729061871269647, log_life_time=100357) [2024-10-16 14:57:51.269807] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.280014] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C10D4-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061871279755000, start_id=1729061871279755000, end_id=1729061871279755001) [2024-10-16 14:57:51.281351] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D0F-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:51.281370] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D0F-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178552, log_task={header:{begin_lsn:{lsn:14538499802}, end_lsn:{lsn:14538499924}, log_id:115178552, min_scn:{val:1729061871151670000, v:0}, max_scn:{val:1729061871151670000, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538499680}, prev_proposal_id:23, committed_end_lsn:{lsn:14538499802}, data_checksum:1715074492, accum_checksum:1046302967, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061871181267, freeze_ts:1729061871181267, submit_ts:1729061871181270, flushed_ts:1729061871181814, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:544}, fs_cb_begin_ts=1729061871281363, log_life_time=100096) [2024-10-16 14:57:51.281508] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.281786] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=243, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:51.281813] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=57, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:51.281842] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:51.288639] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:45338084276}, base_lsn={lsn:45161508864}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:51.288687] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=49] get_replay_process success(id={id:1}, replayed_log_size=176575412, unreplayed_log_size=0) [2024-10-16 14:57:51.288705] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1003, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=168, last_replayed_log_size(MB)=168, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:51.294552] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85061-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:51.295811] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:51.295826] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=121, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:51.295863] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:51.299590] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061871298094000, start_id=1729061871298094000, end_id=1729061871298094001) [2024-10-16 14:57:51.302494] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD103B-0-0] [lt=17] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:51.312347] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:51.312432] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAC-0-0] [lt=11] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061879029444000, latest_log_ts:{val:1729061862417342001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:51.312484] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAC-0-0] [lt=50] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:51.316896] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21656][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3DFA-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510339185292, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410615, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:51.317673] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061871313802000, start_id=1729061871313802000, end_id=1729061871313802001) [2024-10-16 14:57:51.320696] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD330-0-0] [lt=17] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:51.327241] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:185) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=39] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=16106127, size_used=8085, mem_used=16637952) [2024-10-16 14:57:51.328090] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:191) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=12] release old cannot free more memory [2024-10-16 14:57:51.328099] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=8] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=3, elapse_time=858, size_used=0, mem_used=2079744) [2024-10-16 14:57:51.331211] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E49-0-0] [lt=8] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12274, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:51.331345] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E49-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12274, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871331333, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871331333, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871331333, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.331364] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E49-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:12274, add_timestamp:1729061871331184}, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871331333, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871331333, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871331333, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.332358] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:51.332378] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB284-0-0] [lt=16][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:51.337670] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB284-0-0] [lt=10][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:57:51.341150] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:51.344196] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=13] [WRS] update tenant weak read server version(tenant_id=1004, server_version={version:{val:1729061871055408001, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3, epoch_tstamp:1729061871344151}, version_delta=288787) [2024-10-16 14:57:51.353569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061871352718000, start_id=1729061871352718000, end_id=1729061871352718001) [2024-10-16 14:57:51.353625] WDIAG [OCCAM] runTimerTask (ob_occam_timer.h:163) [22477][T1003_MultiVers][T1003][Y0-0000000000000000-0-0] [lt=28][errcode=0] task scheduled too delay(scheduled_delay=1103732, expected_run_ts_us_=1729061870249890, ObClockGenerator::getRealClock()=1729061871353623, ret=0, ret="OB_SUCCESS", *this={this:0x2b0df39ff910, caller:ob_multi_version_garbage_collector.cpp:start:115, function_type:ZN9oceanbase19concurrency_control30ObMultiVersionGarbageCollector5startEvE5$_938, timer_running_flag:1, total_running_count:1, func_shared_ptr_.ptr:0x2b0df5bf5e70, time_wheel:0x2b0e0bb49990, thread_pool:0x2b0e0bb49170, is_running:1, time_interval:60.00s, expected_run_time:14:57:50.249, task_priority:1, with_handle_protected:1}) [2024-10-16 14:57:51.353675] INFO [MVCC] repeat_study (ob_multi_version_garbage_collector.cpp:196) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=5] skip repeat study(ret=0, this={this:0x2b0e2c7fcf30, last_study_timestamp:1729061391284615, last_refresh_timestamp:1729061811341257, last_reclaim_timestamp:1729061391286492, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061390930197000, v:0}, is_inited:true}, current_timestamp=1729061871353674, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:51.354441] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=13] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1729061391281289000, snapshot_type=1, create_time=1729061391281937, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:51.354459] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=13] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1729061755229180000, snapshot_type=1, create_time=1729061755231265, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:51.354466] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1729061678815723000, snapshot_type=1, create_time=1729061678794616, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:51.354473] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1729061390930197000, snapshot_type=2, create_time=1729061391281937, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:51.354479] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1729061754761303001, snapshot_type=2, create_time=1729061755231265, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:51.354484] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1729061678388962000, snapshot_type=2, create_time=1729061678794616, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:51.354490] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1729061391268611000, snapshot_type=3, create_time=1729061391281937, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:51.354495] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1729061755123973000, snapshot_type=3, create_time=1729061755231265, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:51.354500] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1729061678815723000, snapshot_type=3, create_time=1729061678794616, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:51.354506] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061391281937, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:51.354511] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061755231265, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:51.354516] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061678794616, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:51.354646] INFO [MVCC] decide_reserved_snapshot_version_ (ob_multi_version_garbage_collector.cpp:618) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=4] succeed to update global reserved snapshot(ret=0, this={this:0x2b0e2c7fcf30, last_study_timestamp:1729061391284615, last_refresh_timestamp:1729061811341257, last_reclaim_timestamp:1729061391286492, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061390930197000, v:0}, is_inited:true}, global_reserved_snapshot_={val:1729061390930197000, v:0}, reserved_snapshot={val:1729061390930197000, v:0}) [2024-10-16 14:57:51.354657] INFO [MVCC] refresh_ (ob_multi_version_garbage_collector.cpp:552) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=10] multi-version garbage collector refresh successfully(this={this:0x2b0e2c7fcf30, last_study_timestamp:1729061391284615, last_refresh_timestamp:1729061811341257, last_reclaim_timestamp:1729061391286492, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061390930197000, v:0}, is_inited:true}, collector={reserved_snapshot_version:{val:1729061390930197000, v:0}, reserved_snapshot_type:2, reserved_status:-1, reserved_create_time:1729061391281937, reserved_addr:"10.0.4.7:2890", is_this_server_disabled:false, status:0}) [2024-10-16 14:57:51.354667] INFO [MVCC] repeat_refresh (ob_multi_version_garbage_collector.cpp:231) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=9] repeat refresh successfully(ret=0, this={this:0x2b0e2c7fcf30, last_study_timestamp:1729061391284615, last_refresh_timestamp:1729061871354666, last_reclaim_timestamp:1729061391286492, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061390930197000, v:0}, is_inited:true}, current_timestamp=1729061871353688, GARBAGE_COLLECT_RETRY_INTERVAL=60000000) [2024-10-16 14:57:51.354673] INFO [MVCC] repeat_reclaim (ob_multi_version_garbage_collector.cpp:266) [22476][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=6] skip repeat reclaim(ret=0, this={this:0x2b0e2c7fcf30, last_study_timestamp:1729061391284615, last_refresh_timestamp:1729061871354666, last_reclaim_timestamp:1729061391286492, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061390930197000, v:0}, is_inited:true}, current_timestamp=1729061871354672, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:51.358147] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061871356806000, start_id=1729061871356806000, end_id=1729061871356806001) [2024-10-16 14:57:51.360313] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E07-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061871359416000, start_id=1729061871359416000, end_id=1729061871359416001) [2024-10-16 14:57:51.369396] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE782F-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:51.369864] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.369927] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4226-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:51.369935] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4226-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9556, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:51.369947] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561374, miss_cnt:1, cache_read_size:152590691024, hit_ratio:"9.999999928346936517e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:51.370248] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601442, offset=9636742}, end_cursor:ObLogCursor{file_id=192, log_id=47601443, offset=9637026}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72f6050, len:1146, log_data_len:284, seq:47601442, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:51.371954] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4AD-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061871370923000, start_id=1729061871370923000, end_id=1729061871370923001) [2024-10-16 14:57:51.372201] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB2-0-0] [lt=13] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:51.372213] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB2-0-0] [lt=8] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=12) [2024-10-16 14:57:51.373057] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601443, offset=9637026}, end_cursor:ObLogCursor{file_id=192, log_id=47601444, offset=9637300}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d50050, len:862, log_data_len:274, seq:47601443, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:51.376717] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601444, offset=9637300}, end_cursor:ObLogCursor{file_id=192, log_id=47601445, offset=9637574}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d04050, len:588, log_data_len:274, seq:47601444, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:51.377002] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=30] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601445, offset=9637574}, end_cursor:ObLogCursor{file_id=192, log_id=47601446, offset=9637848}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2df8050, len:4410, log_data_len:274, seq:47601445, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:51.377053] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598959, miss_cnt:9, cache_read_size:149777923694, hit_ratio:"9.999999345925326688e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:51.381392] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D11-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:51.381569] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.381875] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF8F-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:51.381886] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FF8F-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10647, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:51.393326] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:51.394610] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85063-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497513, log_task={header:{begin_lsn:{lsn:207589559253}, end_lsn:{lsn:207589559375}, log_id:218497513, min_scn:{val:1729061871150609001, v:0}, max_scn:{val:1729061871150609001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589559131}, prev_proposal_id:81, committed_end_lsn:{lsn:207589559253}, data_checksum:940009244, accum_checksum:3377827954, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061871294546, freeze_ts:1729061871294546, submit_ts:1729061871294564, flushed_ts:1729061871295838, gen_to_freeze cost time:0, gen_to_submit cost time:18, submit_to_flush cost time:1274}, fs_cb_begin_ts=1729061871394596, log_life_time=100050) [2024-10-16 14:57:51.395021] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:51.396615] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=11] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:51.397787] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DA9-0-0] [lt=13] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12276, add_timestamp:1729061871397780}) [2024-10-16 14:57:51.397989] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4A-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12276, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:51.398025] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=13] new mysql sessid created(conn.sessid_=3221737633, support_ssl=false) [2024-10-16 14:57:51.398052] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] sm conn init succ(conn.sessid_=3221737633, sess.client_addr_="127.0.0.1:33574") [2024-10-16 14:57:51.398059] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] accept one succ(*s={this:0x2b0e5cdcc030, session_id:3221737633, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398073] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=11] new mysql sessid created(conn.sessid_=3221719884, support_ssl=false) [2024-10-16 14:57:51.398085] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221719884, sess.client_addr_="127.0.0.1:33580") [2024-10-16 14:57:51.398089] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e2917e030, session_id:3221719884, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398098] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221739503, support_ssl=false) [2024-10-16 14:57:51.398109] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221739503, sess.client_addr_="127.0.0.1:33582") [2024-10-16 14:57:51.398112] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdfcdf0, session_id:3221739503, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:214}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398121] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221737317, support_ssl=false) [2024-10-16 14:57:51.398132] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221737317, sess.client_addr_="127.0.0.1:33584") [2024-10-16 14:57:51.398135] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdbadf0, session_id:3221737317, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:215}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398144] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221736902, support_ssl=false) [2024-10-16 14:57:51.398154] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221736902, sess.client_addr_="127.0.0.1:33588") [2024-10-16 14:57:51.398158] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] accept one succ(*s={this:0x2b0e5cdf8df0, session_id:3221736902, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:218}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398194] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221738549, support_ssl=false) [2024-10-16 14:57:51.398205] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221738549, sess.client_addr_="127.0.0.1:33594") [2024-10-16 14:57:51.398209] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdb6df0, session_id:3221738549, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:221}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398329] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=82] new mysql sessid created(conn.sessid_=3221637205, support_ssl=false) [2024-10-16 14:57:51.398348] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] sm conn init succ(conn.sessid_=3221637205, sess.client_addr_="127.0.0.1:33598") [2024-10-16 14:57:51.398353] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f27406030, session_id:3221637205, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:223}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398384] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.398396] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=5] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.398404] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.398410] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.398473] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA0-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.398534] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=38] new mysql sessid created(conn.sessid_=3221728504, support_ssl=false) [2024-10-16 14:57:51.398563] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=16] sm conn init succ(conn.sessid_=3221728504, sess.client_addr_="127.0.0.1:33576") [2024-10-16 14:57:51.398570] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0f27404030, session_id:3221728504, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:224}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398585] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=10] new mysql sessid created(conn.sessid_=3221740609, support_ssl=false) [2024-10-16 14:57:51.398595] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221740609, sess.client_addr_="127.0.0.1:33578") [2024-10-16 14:57:51.398609] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f27414df0, session_id:3221740609, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:225}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398620] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221740449, support_ssl=false) [2024-10-16 14:57:51.398630] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221740449, sess.client_addr_="127.0.0.1:33586") [2024-10-16 14:57:51.398634] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f2742c030, session_id:3221740449, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:227}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398642] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221646562, support_ssl=false) [2024-10-16 14:57:51.398652] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221646562, sess.client_addr_="127.0.0.1:33592") [2024-10-16 14:57:51.398659] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f2742cdf0, session_id:3221646562, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:229}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398668] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221720148, support_ssl=false) [2024-10-16 14:57:51.398678] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221720148, sess.client_addr_="127.0.0.1:33596") [2024-10-16 14:57:51.398681] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f2749c030, session_id:3221720148, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:230}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398704] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=20] new mysql sessid created(conn.sessid_=3221738848, support_ssl=false) [2024-10-16 14:57:51.398716] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=4] sm conn init succ(conn.sessid_=3221738848, sess.client_addr_="127.0.0.1:33590") [2024-10-16 14:57:51.398720] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdb8df0, session_id:3221738848, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:232}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.398793] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBF-0-0] [lt=12] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.398934] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA0-0-0] [lt=5] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221738549, proxy_sessid=0, client_sessid=3221738549, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.399009] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.399020] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=4] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.399031] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.399037] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.399043] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.399047] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.399052] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.399056] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.399062] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.399066] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.399118] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA1-0-0] [lt=15] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.399206] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DBF-0-0] [lt=6] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221736902, proxy_sessid=0, client_sessid=3221736902, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.399254] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC0-0-0] [lt=13] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.399481] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA1-0-0] [lt=3] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221637205, proxy_sessid=0, client_sessid=3221637205, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.399529] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA2-0-0] [lt=16] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.399643] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC0-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737317, proxy_sessid=0, client_sessid=3221737317, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.399690] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC1-0-0] [lt=12] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.399894] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA2-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221739503, proxy_sessid=0, client_sessid=3221739503, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.399956] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA3-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.400038] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC1-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221719884, proxy_sessid=0, client_sessid=3221719884, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.400317] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA3-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221737633, proxy_sessid=0, client_sessid=3221737633, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.400479] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=10] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.400491] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=5] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.400556] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.400563] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=4] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.400570] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.400575] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.400580] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.400583] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.400589] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.400593] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.400598] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:57:51.400609] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=9] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:51.400652] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B1-0-0] [lt=17] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.401032] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B1-0-0] [lt=3] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221738848, proxy_sessid=0, client_sessid=3221738848, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.401059] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257790, offset=39040934}, end_cursor:ObLogCursor{file_id=197, log_id=49257791, offset=39041217}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2d9c050, len:2138, log_data_len:283, seq:49257790, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:51.401081] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B2-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.401411] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B2-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221720148, proxy_sessid=0, client_sessid=3221720148, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.401453] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B3-0-0] [lt=12] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.401456] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=7] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12358, add_timestamp:1729061871401451}) [2024-10-16 14:57:51.401471] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12396, add_timestamp:1729061871401471}) [2024-10-16 14:57:51.401787] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B3-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221646562, proxy_sessid=0, client_sessid=3221646562, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.401831] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B4-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.402167] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B4-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221740449, proxy_sessid=0, client_sessid=3221740449, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.402204] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B5-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.402286] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.402295] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.402557] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B5-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221740609, proxy_sessid=0, client_sessid=3221740609, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.402598] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B6-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:57:51.402943] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.402945] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B6-0-0] [lt=14] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221728504, proxy_sessid=0, client_sessid=3221728504, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:51.402965] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.402972] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.402976] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.402976] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=7] LogLoopThread round_cost_time(us)(round_cost_time=9) [2024-10-16 14:57:51.402981] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.403007] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:51.403010] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.403019] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.403022] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.403030] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:51.403033] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.403037] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.403040] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9977EE668C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.403463] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED797-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1729061871403460}) [2024-10-16 14:57:51.403582] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4A-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12276, locations=[{table_id:12276, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403569, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403569, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403569, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.403612] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4A-0-0] [lt=29] success to process renew task(task={tenant_id:1, table_id:12276, add_timestamp:1729061871397780}, locations=[{table_id:12276, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403569, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403569, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403569, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.403635] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4B-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12358, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403632, sql_renew_time:1729061871403632, is_mark_fail:false}]) [2024-10-16 14:57:51.403641] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4B-0-0] [lt=6] success to process renew task(task={tenant_id:1, table_id:12358, add_timestamp:1729061871401451}, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403632, sql_renew_time:1729061871403632, is_mark_fail:false}]) [2024-10-16 14:57:51.403651] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4C-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12396, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403648, sql_renew_time:1729061871403648, is_mark_fail:false}]) [2024-10-16 14:57:51.403656] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4C-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12396, add_timestamp:1729061871401471}, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403648, sql_renew_time:1729061871403648, is_mark_fail:false}]) [2024-10-16 14:57:51.403664] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4D-0-0] [lt=5] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:51.404386] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=16] get_stable_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:51.404412] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1001][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=7] get_stable_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:51.404424] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1002][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=4] get_stable_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:51.404431] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1003][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=2] get_stable_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:51.404440] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21681][T1_L0_G0][T1004][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=3] get_stable_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:51.405207] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=2] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.405216] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.405863] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.405884] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.405890] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.405895] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.405899] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.405937] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:51.405940] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.405949] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.405952] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.405961] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:51.405964] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.405968] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.405971] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4E9F-0-0] [lt=2] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.406232] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4EA0-0-0] [lt=3] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11008, add_timestamp:1729061871406228}) [2024-10-16 14:57:51.408757] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4EA1-0-0] [lt=7] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12142, add_timestamp:1729061871408755}) [2024-10-16 14:57:51.408770] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4EA1-0-0] [lt=8] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12217, add_timestamp:1729061871408769}) [2024-10-16 14:57:51.408776] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4EA1-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12068, add_timestamp:1729061871408775}) [2024-10-16 14:57:51.408781] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4EA1-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11035, add_timestamp:1729061871408781}) [2024-10-16 14:57:51.408788] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4EA1-0-0] [lt=3] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12072, add_timestamp:1729061871408788}) [2024-10-16 14:57:51.411231] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED797-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:51.411412] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1729061871411409}) [2024-10-16 14:57:51.412346] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.412355] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.412979] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.412999] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.413006] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.413011] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.413014] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.413068] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:51.413072] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.413081] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.413084] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.413092] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:51.413095] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.413099] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.413102] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED798-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.413315] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=3][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:10008, add_timestamp:1729061871413313}) [2024-10-16 14:57:51.413323] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=8][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:10008, add_timestamp:1729061871413313}) [2024-10-16 14:57:51.413327] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=3][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=10008, vtable_locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433051, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433051, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061856433051, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.413745] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21667][T1_L0_G0][T1][YB4A0A000407-00061E99796CFF96-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1729061871413743}) [2024-10-16 14:57:51.416591] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=23] no need to update min/max version(new_version={val:1729061871098151000, v:0}, min_version={val:1729061871098151000, v:0}, max_version={val:1729061871098151000, v:0}) [2024-10-16 14:57:51.417045] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] get_max_applied_scn(scn={val:1729061871279755000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959712366}, palf_committed_end_scn_:{val:1729061871279755000, v:0}, last_check_scn_:{val:1729061871279755000, v:0}, max_applied_cb_scn_:{val:1729061871279755000, v:0}}) [2024-10-16 14:57:51.417065] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:51.417114] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=40] get_max_applied_scn(scn={val:1729061871279755000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023112269}, palf_committed_end_scn_:{val:1729061871279755000, v:0}, last_check_scn_:{val:1729061871279755000, v:0}, max_applied_cb_scn_:{val:1729061871279755000, v:0}}) [2024-10-16 14:57:51.417128] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:51.417152] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:57:51.417161] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] get_min_unreplayed_log_info(lsn={lsn:213058251042}, scn={val:1729061871151670003, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061871377033, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058251042}, next_to_submit_scn_:{val:1729061871151670003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058250920}, end_lsn:{lsn:213058251042}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061871151670002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061871151670002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:2541612890, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061871151670002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561374, miss_cnt:1, cache_read_size:152590691024, hit_ratio:"9.999999928346936517e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:51.420218] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [21667][T1_L0_G0][T1][YB4A0A000407-00061E99796CFF96-0-0] [lt=8][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:51.420771] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55CD-0-0] [lt=7] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11021, add_timestamp:1729061871420768}) [2024-10-16 14:57:51.426747] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:51.426777] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:51.427386] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF722-0-0] [lt=8] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11003, add_timestamp:1729061871427383}) [2024-10-16 14:57:51.427917] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:290) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55CD-0-0] [lt=10][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:51.428529] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.428539] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.429221] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=25] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.429243] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.429249] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.429254] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.429258] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.429412] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:51.429416] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.429425] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.429428] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.429438] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:51.429442] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.429446] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.429449] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21672][T1_L0_G0][T1][YB4A0A000407-00061E99799CF724-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.429595] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdb8df0, session_id:3221738848, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:232}, err:5, last_decode_time:1729061871401171, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:654, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429626] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=27] kill and revert session(conn.sessid_=3221738848, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.429632] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] sock ref clean, do destroy(*s={this:0x2b0e5cdb8df0, session_id:3221738848, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:232}, err:5, last_decode_time:1729061871401171, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:654, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429646] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221738848, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.429651] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] connection close(sessid=3221738848, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED797-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:549, rec_:549, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:25, seq_:32}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:33}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.429716] INFO [SERVER] run (obmp_disconnect.cpp:82) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=10] free session successfully(sessid=3221738848) [2024-10-16 14:57:51.429721] INFO [SERVER] run (obmp_disconnect.cpp:89) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221738848) [2024-10-16 14:57:51.429731] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27404030, session_id:3221728504, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:224}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:793, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429741] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=8] kill and revert session(conn.sessid_=3221728504, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.429744] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2742cdf0, session_id:3221646562, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:229}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:466, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429751] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] kill and revert session(conn.sessid_=3221646562, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.429754] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2742c030, session_id:3221740449, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:227}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:203, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429760] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] kill and revert session(conn.sessid_=3221740449, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.429763] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27414df0, session_id:3221740609, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:225}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:239, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429769] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221740609, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.429786] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0f27404030, session_id:3221728504, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:224}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:793, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429795] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221728504, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.429799] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221728504, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B3A0A55CD-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:688, rec_:688, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:370, seq_:120}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:121}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.429814] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=14] sock ref clean, do destroy(*s={this:0x2b0f2742cdf0, session_id:3221646562, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:229}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:466, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429822] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221646562, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.429825] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221646562, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E99799CF723-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:361, rec_:361, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:7, seq_:9}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.429837] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=11] sock ref clean, do destroy(*s={this:0x2b0f2742c030, session_id:3221740449, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:227}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:203, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429845] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221740449, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.429846] INFO [SERVER] run (obmp_disconnect.cpp:82) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=7] free session successfully(sessid=3221728504) [2024-10-16 14:57:51.429855] INFO [SERVER] run (obmp_disconnect.cpp:89) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=8] mark session id unused(sessid=3221728504) [2024-10-16 14:57:51.429848] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221740449, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E99799CF722-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:98, rec_:98, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.429860] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0f27414df0, session_id:3221740609, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:225}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:239, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.429868] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221740609, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.429880] INFO [SERVER] run (obmp_disconnect.cpp:82) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221646562) [2024-10-16 14:57:51.429871] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221740609, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E99799CF721-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:134, rec_:134, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:7}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:8}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.429883] INFO [SERVER] run (obmp_disconnect.cpp:89) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221646562) [2024-10-16 14:57:51.429903] INFO [SERVER] run (obmp_disconnect.cpp:82) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221740449) [2024-10-16 14:57:51.429905] INFO [SERVER] run (obmp_disconnect.cpp:89) [21669][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221740449) [2024-10-16 14:57:51.429915] INFO [SERVER] run (obmp_disconnect.cpp:82) [25332][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=15] free session successfully(sessid=3221740609) [2024-10-16 14:57:51.429919] INFO [SERVER] run (obmp_disconnect.cpp:89) [25332][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] mark session id unused(sessid=3221740609) [2024-10-16 14:57:51.430409] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=11][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2749c030, session_id:3221720148, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:230}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.430426] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=14] kill and revert session(conn.sessid_=3221720148, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.430480] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] sock ref clean, do destroy(*s={this:0x2b0f2749c030, session_id:3221720148, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:230}, err:5, last_decode_time:1729061871417193, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.430504] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=7] unlock session of tenant(conn.sessid_=3221720148, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.430508] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221720148, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E99799CF724-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:452, rec_:452, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.430528] INFO [SERVER] run (obmp_disconnect.cpp:82) [21658][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=11] free session successfully(sessid=3221720148) [2024-10-16 14:57:51.430535] INFO [SERVER] run (obmp_disconnect.cpp:89) [21658][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=7] mark session id unused(sessid=3221720148) [2024-10-16 14:57:51.430879] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:51.430918] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27406030, session_id:3221637205, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:223}, err:5, last_decode_time:1729061871401171, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:667, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.430934] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12] kill and revert session(conn.sessid_=3221637205, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.430939] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] sock ref clean, do destroy(*s={this:0x2b0f27406030, session_id:3221637205, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:223}, err:5, last_decode_time:1729061871401171, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:667, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.430950] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] unlock session of tenant(conn.sessid_=3221637205, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.430954] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221637205, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA26D4E9F-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:562, rec_:562, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:10}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:11}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.431034] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4D-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11017, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871431022, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871431022, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871431022, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.431054] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4D-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:11017, add_timestamp:1729061871403460}, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871431022, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871431022, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871431022, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.431073] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4E-0-0] [lt=11] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11008, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:51.431570] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=17] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.431580] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.432226] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:57:51.432246] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.432253] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.432257] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.432261] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.432300] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=2] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:57:51.432303] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.432312] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:57:51.432315] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.432323] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:57:51.432326] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.432331] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:57:51.432334] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED799-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:57:51.432483] INFO [SERVER] run (obmp_disconnect.cpp:82) [21678][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221637205) [2024-10-16 14:57:51.432489] INFO [SERVER] run (obmp_disconnect.cpp:89) [21678][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=6] mark session id unused(sessid=3221637205) [2024-10-16 14:57:51.432730] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB378-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061871426656000, start_id=1729061871426656000, end_id=1729061871426656001) [2024-10-16 14:57:51.433163] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdf8df0, session_id:3221736902, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:218}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:346, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433185] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=17] kill and revert session(conn.sessid_=3221736902, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.433191] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdfcdf0, session_id:3221739503, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:214}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433199] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=7] kill and revert session(conn.sessid_=3221739503, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.433205] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdcc030, session_id:3221737633, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:231, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433216] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=9] kill and revert session(conn.sessid_=3221737633, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.433219] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdb6df0, session_id:3221738549, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:221}, err:5, last_decode_time:1729061871401171, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:738, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433228] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=8] kill and revert session(conn.sessid_=3221738549, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.433259] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=17] sock ref clean, do destroy(*s={this:0x2b0e5cdf8df0, session_id:3221736902, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:218}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:346, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433270] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221736902, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.433274] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221736902, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E99796CFF96-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:241, rec_:241, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.433300] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=24] sock ref clean, do destroy(*s={this:0x2b0e5cdfcdf0, session_id:3221739503, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:214}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433308] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221739503, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.433311] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221739503, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED798-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:452, rec_:452, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.433324] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=13] sock ref clean, do destroy(*s={this:0x2b0e5cdcc030, session_id:3221737633, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:231, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433332] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221737633, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.433335] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221737633, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA26D4EA0-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:126, rec_:126, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:64, seq_:70}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:71}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.433358] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=22] sock ref clean, do destroy(*s={this:0x2b0e5cdb6df0, session_id:3221738549, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:221}, err:5, last_decode_time:1729061871401171, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:738, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433366] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221738549, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.433373] INFO [SERVER] run (obmp_disconnect.cpp:82) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221736902) [2024-10-16 14:57:51.433382] INFO [SERVER] run (obmp_disconnect.cpp:89) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=8] mark session id unused(sessid=3221736902) [2024-10-16 14:57:51.433368] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] connection close(sessid=3221738549, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9977EE668C-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:633, rec_:633, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.433387] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=17][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdbadf0, session_id:3221737317, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:215}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:534, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433394] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221737317, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.433408] INFO [SERVER] run (obmp_disconnect.cpp:82) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221739503) [2024-10-16 14:57:51.433410] INFO [SERVER] run (obmp_disconnect.cpp:89) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221739503) [2024-10-16 14:57:51.433421] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0e5cdbadf0, session_id:3221737317, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:215}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:534, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.433430] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221737317, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.433433] INFO [SERVER] run (obmp_disconnect.cpp:82) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221737633) [2024-10-16 14:57:51.433436] INFO [SERVER] run (obmp_disconnect.cpp:89) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221737633) [2024-10-16 14:57:51.433432] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221737317, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED799-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:429, rec_:429, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.433458] INFO [SERVER] run (obmp_disconnect.cpp:82) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221738549) [2024-10-16 14:57:51.433461] INFO [SERVER] run (obmp_disconnect.cpp:89) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221738549) [2024-10-16 14:57:51.433490] INFO [SERVER] run (obmp_disconnect.cpp:82) [21654][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] free session successfully(sessid=3221737317) [2024-10-16 14:57:51.433494] INFO [SERVER] run (obmp_disconnect.cpp:89) [21654][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] mark session id unused(sessid=3221737317) [2024-10-16 14:57:51.433833] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:51.434055] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:51.434071] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4E-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11008, locations=[{table_id:11008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434060, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434060, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434060, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.434083] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257791, offset=39041217}, end_cursor:ObLogCursor{file_id=197, log_id=49257792, offset=39041499}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7356050, len:1855, log_data_len:282, seq:49257791, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:51.434095] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4E-0-0] [lt=23] success to process renew task(task={tenant_id:1, table_id:11008, add_timestamp:1729061871406228}, locations=[{table_id:11008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434060, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434060, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434060, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.434126] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4F-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12142, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434122, sql_renew_time:1729061871434122, is_mark_fail:false}]) [2024-10-16 14:57:51.434135] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E4F-0-0] [lt=9] success to process renew task(task={tenant_id:1, table_id:12142, add_timestamp:1729061871408755}, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434122, sql_renew_time:1729061871434122, is_mark_fail:false}]) [2024-10-16 14:57:51.434145] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E50-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12217, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434143, sql_renew_time:1729061871434143, is_mark_fail:false}]) [2024-10-16 14:57:51.434155] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E50-0-0] [lt=10] success to process renew task(task={tenant_id:1, table_id:12217, add_timestamp:1729061871408769}, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434143, sql_renew_time:1729061871434143, is_mark_fail:false}]) [2024-10-16 14:57:51.434166] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E51-0-0] [lt=7] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12068, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434164, sql_renew_time:1729061871434164, is_mark_fail:false}]) [2024-10-16 14:57:51.434171] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E51-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12068, add_timestamp:1729061871408775}, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434164, sql_renew_time:1729061871434164, is_mark_fail:false}]) [2024-10-16 14:57:51.434187] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E52-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11035, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434184, sql_renew_time:1729061871434184, is_mark_fail:false}]) [2024-10-16 14:57:51.434200] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E52-0-0] [lt=13] success to process renew task(task={tenant_id:1, table_id:11035, add_timestamp:1729061871408781}, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434184, sql_renew_time:1729061871434184, is_mark_fail:false}]) [2024-10-16 14:57:51.434210] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E53-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12072, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434208, sql_renew_time:1729061871434208, is_mark_fail:false}]) [2024-10-16 14:57:51.434218] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E53-0-0] [lt=8] success to process renew task(task={tenant_id:1, table_id:12072, add_timestamp:1729061871408788}, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434208, sql_renew_time:1729061871434208, is_mark_fail:false}]) [2024-10-16 14:57:51.434228] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E54-0-0] [lt=8] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:51.434968] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E54-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10008, locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434957, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434957, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434957, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.434989] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E54-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:10008, add_timestamp:1729061871411409}, locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434957, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434957, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434957, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.435006] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E55-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:51.435638] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E55-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11017, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871435629, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871435629, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871435629, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.435658] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E55-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:11017, add_timestamp:1729061871413743}, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871435629, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871435629, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871435629, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.435676] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E56-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11021, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:51.438131] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E56-0-0] [lt=4] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11021, locations=[{table_id:11021, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438124, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438124, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438124, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.438147] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E56-0-0] [lt=15] success to process renew task(task={tenant_id:1, table_id:11021, add_timestamp:1729061871420768}, locations=[{table_id:11021, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438124, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438124, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438124, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.438165] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E57-0-0] [lt=11] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11003, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:51.438474] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E57-0-0] [lt=4] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11003, locations=[{table_id:11003, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438468, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438468, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438468, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.438489] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E57-0-0] [lt=14] success to process renew task(task={tenant_id:1, table_id:11003, add_timestamp:1729061871427383}, locations=[{table_id:11003, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438468, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438468, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871438468, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:51.439474] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107515][][T1003][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1003, ret=0, thread_count_=173) [2024-10-16 14:57:51.439499] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] worker thread created(tenant_->id()=1003, group_id_=1, token=1) [2024-10-16 14:57:51.439523] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107515][][T1003][Y0-0000000000000000-0-0] [lt=47] Init thread local success [2024-10-16 14:57:51.439548] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107515][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797CF2-0-0] [lt=4] init point thread id with(&point=0x55e3435cad40, idx_=1019, point=[thread id=107515, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107515) [2024-10-16 14:57:51.441543] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:51.441558] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:51.441847] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:51.444921] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061871443540000, start_id=1729061871443540000, end_id=1729061871443540001) [2024-10-16 14:57:51.448615] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=45] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:57:51.448625] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:57:51.448629] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:51.449206] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061879029444000, latest_log_ts:{val:1729061862417342001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:51.455366] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257792, offset=39041499}, end_cursor:ObLogCursor{file_id=197, log_id=49257793, offset=39041781}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db729a050, len:1573, log_data_len:282, seq:49257792, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:51.455794] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=273, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:51.455809] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=241, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:51.455850] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:51.455870] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257793, offset=39041781}, end_cursor:ObLogCursor{file_id=197, log_id=49257794, offset=39042063}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72d6050, len:1291, log_data_len:282, seq:49257793, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:51.456617] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650311, miss_cnt:32, cache_read_size:8050429253, hit_ratio:"9.999994892286543369e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:51.459730] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657049, miss_cnt:17, cache_read_size:8050176634, hit_ratio:"9.999997286818377296e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:51.460618] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e2917e030, session_id:3221719884, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:224, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.460639] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=16] kill and revert session(conn.sessid_=3221719884, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:57:51.460645] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] sock ref clean, do destroy(*s={this:0x2b0e2917e030, session_id:3221719884, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:1729061871403940, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:224, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:51.460656] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=7] unlock session of tenant(conn.sessid_=3221719884, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:57:51.460661] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221719884, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA26D4EA1-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:119, rec_:119, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:2, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:10}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:11}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:51.460721] INFO [SERVER] run (obmp_disconnect.cpp:82) [21654][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221719884) [2024-10-16 14:57:51.460726] INFO [SERVER] run (obmp_disconnect.cpp:89) [21654][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221719884) [2024-10-16 14:57:51.468699] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=24, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:51.468715] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=4217, avg_group_log_size=351) [2024-10-16 14:57:51.468724] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:51.468929] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=246, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:51.468947] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=298, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:51.468993] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:51.469238] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1086-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=29, avg_log_gen_to_submit_time=58, avg_log_submit_to_flush_time=567, avg_log_submit_to_slide_time=1237) [2024-10-16 14:57:51.469627] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=188, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:51.469641] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=459, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:51.469916] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=61] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:51.469922] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.469948] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:51.469955] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:51.479043] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:57:51.479067] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=20] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:51.479077] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=8] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:51.479084] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:51.480216] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A89-0-0] [lt=10] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:51.480234] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A89-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:51.481312] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=5] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:57:51.481455] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=140] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:51.481615] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.482047] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91E8-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=318, avg_log_submit_to_slide_time=9717) [2024-10-16 14:57:51.482064] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:51.490447] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:51.490463] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:51.490478] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70781092, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:51.491261] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:51.491271] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:51.491280] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:51.491278] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A27-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:51.491285] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:51.491289] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:51.491288] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A27-0-0] [lt=9] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:51.491292] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:51.491295] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:51.491297] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:51.491300] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:51.491302] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:51.491305] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:51.495135] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C440B-0-0] [lt=25] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:51.495146] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C440B-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9532, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:51.495360] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=43] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:51.495655] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:51.496000] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:51.496058] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061868730909000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:51.515090] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061861514055, end_time=1729061871515088, dag_status={}) [2024-10-16 14:57:51.515111] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=19] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:51.515119] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:51.515123] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:51.515126] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:51.515130] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:51.515133] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:51.515140] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:51.515143] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:51.515150] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:51.515162] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515171] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515175] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515179] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515184] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515192] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515201] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515211] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515218] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515223] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515232] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515240] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515244] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515248] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515255] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515258] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515266] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515269] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515276] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515280] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515284] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515288] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515292] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515299] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515303] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515310] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515314] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515320] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515324] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515331] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515335] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515342] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515346] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515356] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515363] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515367] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515372] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515378] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515385] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515389] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515407] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515410] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515417] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515421] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515428] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515432] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515435] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515439] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515443] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515448] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515452] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515456] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515460] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515463] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515467] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515471] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:51.515476] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:51.515480] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:51.515483] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:51.515486] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:51.515488] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:51.515491] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:51.515493] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:51.515495] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:51.515498] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:51.515503] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:51.515507] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:51.517477] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061871516265000, start_id=1729061871516265000, end_id=1729061871516265001) [2024-10-16 14:57:51.517978] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061871516265000, start_id=1729061871516265000, end_id=1729061871516265001) [2024-10-16 14:57:51.518414] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061871298094000, v:0}, new_max_version={val:1729061871298094000, v:0}, last_min_version={val:1729061871098151000, v:0}, last_max_version={val:1729061871098151000, v:0}, total_time_us=1784, delta_us=220319) [2024-10-16 14:57:51.518434] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061871298094000, v:0}, min_version={val:1729061871298094000, v:0}, max_version={val:1729061871298094000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:51.526744] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:51.526760] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:51.541226] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:51.541578] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:51.541595] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:51.541934] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:51.542183] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F422A-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:51.542228] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F422A-0-0] [lt=44] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2012, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:51.543035] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=909004800, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=164737024, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:51.543092] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=22] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:51.543101] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:51.543133] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=969699328, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=104042496, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:51.543170] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:51.543174] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:51.543203] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=927879168, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=145862656, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:51.543237] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:51.543242] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011868, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:51.543278] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=958939136, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=114802688, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:51.543311] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:51.543315] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:51.548992] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=798, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:51.549012] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=111, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:51.549028] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10329) [2024-10-16 14:57:51.549087] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=53] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:51.554121] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E96-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:51.554138] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E96-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10531, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:51.554145] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E96-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=533, avg_log_submit_to_slide_time=10525) [2024-10-16 14:57:51.554157] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E96-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173248, log_task={header:{begin_lsn:{lsn:14352386413}, end_lsn:{lsn:14352386535}, log_id:115173248, min_scn:{val:1729061871150609003, v:0}, max_scn:{val:1729061871150609003, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352386291}, prev_proposal_id:17, committed_end_lsn:{lsn:14352386413}, data_checksum:940009244, accum_checksum:3950658550, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061871453959, freeze_ts:1729061871453959, submit_ts:1729061871453963, flushed_ts:1729061871455832, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1869}, fs_cb_begin_ts=1729061871554106, log_life_time=100147) [2024-10-16 14:57:51.554560] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061871553119000, start_id=1729061871553119000, end_id=1729061871553119001) [2024-10-16 14:57:51.554680] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:51.556597] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061871555911000, start_id=1729061871555911000, end_id=1729061871555911001) [2024-10-16 14:57:51.556631] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=22] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061871556626], gts:1729061871555911000, latest_srr:[mts=1729061871556626]}) [2024-10-16 14:57:51.558492] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061871558203000, start_id=1729061871558203000, end_id=1729061871558203001) [2024-10-16 14:57:51.559033] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797CF6-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:51.559044] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797CF6-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=1512, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:51.559049] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797CF6-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=919, avg_log_submit_to_slide_time=1483) [2024-10-16 14:57:51.568710] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.569266] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.569267] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.569511] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.569710] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:51.569981] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.581675] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.594779] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E99801B4F14-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061871594211000, start_id=1729061871594211000, end_id=1729061871594211001) [2024-10-16 14:57:51.596750] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:51.596767] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:51.596903] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1001}, {id:1002}]) [2024-10-16 14:57:51.600750] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4810-0-0] [lt=5] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061871568514, keepalive_ts:-1}) [2024-10-16 14:57:51.600774] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4810-0-0] [lt=22] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061871568514, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:51.606407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E99801B4F14-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061871605559000, start_id=1729061871605559000, end_id=1729061871605559001) [2024-10-16 14:57:51.610192] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5536, mem_used=10398720) [2024-10-16 14:57:51.618148] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E99801B4F14-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061871617147000, start_id=1729061871617147000, end_id=1729061871617147001) [2024-10-16 14:57:51.627242] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E99801B4F14-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061871627073000, start_id=1729061871627073000, end_id=1729061871627073001) [2024-10-16 14:57:51.631166] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:51.638638] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E99801B4F14-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061871637985000, start_id=1729061871637985000, end_id=1729061871637985001) [2024-10-16 14:57:51.642312] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4410-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=1703, avg_log_submit_to_slide_time=1920) [2024-10-16 14:57:51.645452] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061871645415000, start_id=1729061871645415000, end_id=1729061871645415001) [2024-10-16 14:57:51.645518] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10329, avg_group_log_size=573) [2024-10-16 14:57:51.645529] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10329) [2024-10-16 14:57:51.645728] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10329) [2024-10-16 14:57:51.653391] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CD-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:51.653420] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CD-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061871150609003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061871150609003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:51.654167] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E98-0-0] [lt=22][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173249, log_task={header:{begin_lsn:{lsn:14352386535}, end_lsn:{lsn:14352386657}, log_id:115173249, min_scn:{val:1729061871150609004, v:0}, max_scn:{val:1729061871150609004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352386413}, prev_proposal_id:17, committed_end_lsn:{lsn:14352386535}, data_checksum:940009244, accum_checksum:2200229835, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061871554091, freeze_ts:1729061871554091, submit_ts:1729061871554097, flushed_ts:1729061871554982, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:885}, fs_cb_begin_ts=1729061871654161, log_life_time=100070) [2024-10-16 14:57:51.657118] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CD-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061871656978000, start_id=1729061871656978000, end_id=1729061871656978001) [2024-10-16 14:57:51.657696] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:51.663612] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26147][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB18CC6ED-0-0] [lt=28] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:51.669104] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [21605][T1_CompactionRe][T1][YB4A0A000407-00061E9973ED0D6F-0-0] [lt=11] InfoPoolResizeTask(cost_ts=8) [2024-10-16 14:57:51.670041] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.672019] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:51.681734] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:51.681754] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:51.681760] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.699557] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:51.706647] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5A-0-0] [lt=10] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:51.706675] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5A-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061871651739001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061871651739001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:51.708160] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C5A-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061871707715000, start_id=1729061871707715000, end_id=1729061871707715001) [2024-10-16 14:57:51.711025] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C5A-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061871710466000, start_id=1729061871710466000, end_id=1729061871710466001) [2024-10-16 14:57:51.711535] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A10-0-0] [lt=15] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061871702726, keepalive_ts:-1}) [2024-10-16 14:57:51.711550] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A10-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061871702726, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:51.717552] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061871716859000, start_id=1729061871716859000, end_id=1729061871716859001) [2024-10-16 14:57:51.717958] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061871716859000, start_id=1729061871716859000, end_id=1729061871716859001) [2024-10-16 14:57:51.718333] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C52-0-0] [lt=19] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3544) [2024-10-16 14:57:51.718589] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3543) [2024-10-16 14:57:51.719225] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:51.719238] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1313, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:51.726984] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:51.727544] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD33C-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:51.727556] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD33C-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1410, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:51.733176] INFO [STORAGE] try_shrink (ob_compaction_memory_pool.cpp:522) [21478][T1_MergeMemPool][T1][Y0-0000000000000000-0-0] [lt=14] Compaction Mem Pool current stat: (ret=0, max_block_num=16, total_block_num=8, Chunk list size: =1) [2024-10-16 14:57:51.741309] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:51.741641] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:57:51.741648] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] try gc retain ctx [2024-10-16 14:57:51.741652] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:51.742255] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061869306266000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880098662000, latest_log_ts:{val:1729061863549302001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:51.742955] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=46] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061869306266000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:51.743590] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=36] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061869306266000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:51.744234] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=34] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061869306266000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:51.753849] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1001, min_version={val:1729061871374259000, v:0}, base_version={val:1729061871352718000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061871553119000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061871753834, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061871374259000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061871544081, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061871553119000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061871698758, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:51.754162] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E9A-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:51.754179] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E9A-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:51.754761] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061871753461000, start_id=1729061871753461000, end_id=1729061871753461001) [2024-10-16 14:57:51.755036] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:51.758358] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061871758191000, start_id=1729061871758191000, end_id=1729061871758191001) [2024-10-16 14:57:51.758624] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:51.758642] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:51.758648] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=5] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:51.770165] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.772255] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:51.776909] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=25] BlockGCTimerTask success(ret=0, cost_time_us=8, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:51.781796] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.782235] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:51.787753] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.788541] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.788577] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.788595] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.788936] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.788943] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.789399] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.789574] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.789587] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.789597] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.789625] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:51.827567] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD33F-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=978, avg_log_submit_to_slide_time=1392) [2024-10-16 14:57:51.828969] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=11] ====== check clog disk timer task ====== [2024-10-16 14:57:51.828983] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:51.828996] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52523576, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:51.831108] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1003}, {id:1}, {id:1001}]) [2024-10-16 14:57:51.831483] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:51.842027] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1227, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:51.842055] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=588, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:51.842071] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:51.845546] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061871844647000, start_id=1729061871844647000, end_id=1729061871844647001) [2024-10-16 14:57:51.854217] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E9C-0-0] [lt=16][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173251, log_task={header:{begin_lsn:{lsn:14352386779}, end_lsn:{lsn:14352386901}, log_id:115173251, min_scn:{val:1729061871651739001, v:0}, max_scn:{val:1729061871651739001, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352386657}, prev_proposal_id:17, committed_end_lsn:{lsn:14352386779}, data_checksum:940009244, accum_checksum:1356080740, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061871754157, freeze_ts:1729061871754157, submit_ts:1729061871754174, flushed_ts:1729061871755062, gen_to_freeze cost time:0, gen_to_submit cost time:17, submit_to_flush cost time:888}, fs_cb_begin_ts=1729061871854210, log_life_time=100053) [2024-10-16 14:57:51.860093] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5490, mem_used=10398720) [2024-10-16 14:57:51.864611] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=23] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}) [2024-10-16 14:57:51.868925] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3543) [2024-10-16 14:57:51.870225] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=37][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.875349] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26143][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB26CC6BA-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:57:51.882179] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.882452] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D1B-0-0] [lt=25] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:51.882474] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D1B-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178558, log_task={header:{begin_lsn:{lsn:14538500534}, end_lsn:{lsn:14538500656}, log_id:115178558, min_scn:{val:1729061871651739002, v:0}, max_scn:{val:1729061871651739002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538500412}, prev_proposal_id:23, committed_end_lsn:{lsn:14538500534}, data_checksum:1715074492, accum_checksum:215761947, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061871782372, freeze_ts:1729061871782373, submit_ts:1729061871782376, flushed_ts:1729061871782581, gen_to_freeze cost time:1, gen_to_submit cost time:4, submit_to_flush cost time:205}, fs_cb_begin_ts=1729061871882467, log_life_time=100095) [2024-10-16 14:57:51.882612] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:51.882624] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:51.889444] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:51.902064] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:51.908678] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D77-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:51.908702] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D77-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061871651739002, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061871651739002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:51.909876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D77-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061871909868000, start_id=1729061871909868000, end_id=1729061871909868001) [2024-10-16 14:57:51.912731] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D77-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061871912649000, start_id=1729061871912649000, end_id=1729061871912649001) [2024-10-16 14:57:51.912808] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D77-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:51.912824] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D77-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:51.917645] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061871915820000, start_id=1729061871915820000, end_id=1729061871915820001) [2024-10-16 14:57:51.918146] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061871917921000, start_id=1729061871917921000, end_id=1729061871917921001) [2024-10-16 14:57:51.920542] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:51.920557] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=10] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=215) [2024-10-16 14:57:51.921331] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DDB-0-0] [lt=7] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:57:51.924007] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:51.926954] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:51.926971] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:51.927154] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:51.935197] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15E2-0-0] [lt=6] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:51.941348] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:51.942226] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4419-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:51.942236] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4419-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2229, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:51.943886] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3572, mem_used=8318976) [2024-10-16 14:57:51.954304] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53E9E-0-0] [lt=20] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:51.954732] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061871954615000, start_id=1729061871954615000, end_id=1729061871954615001) [2024-10-16 14:57:51.958473] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061871956977000, start_id=1729061871956977000, end_id=1729061871956977001) [2024-10-16 14:57:51.968919] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=5] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061867964260, last_first_register_time_us=1722494150998907, register_time_us=1729061867964260, curr_time_us=1729061871968917) [2024-10-16 14:57:51.968959] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061871968938, last_submit_register_req_time_us=1729061871968938, register_parent_reason=3) [2024-10-16 14:57:51.968969] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:51.969740] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE783B-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:51.969971] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:51.970232] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.972739] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:51.982254] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:51.982594] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:51.995491] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D3A-0-0] [lt=19] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:51.995509] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D3A-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:52.015688] INFO [CLOG] do_thread_task_ (ob_remote_fetch_log_worker.cpp:249) [21544][T1_RFLWorker][T1][YB4A0A000407-00061E996BC960C1-0-0] [lt=8] ObRemoteFetchWorker is running(thread_index=0) [2024-10-16 14:57:52.021260] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=23] PNIO [ratelimit] time: 1729061872021258, bytes: 4446818301170, bw: 0.498423 MB/s, add_ts: 1002537, add_bytes: 523960 [2024-10-16 14:57:52.027279] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=840, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:52.027294] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=753, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:52.027316] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:52.027589] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD345-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:52.027614] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD345-0-0] [lt=25] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=2097, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:52.031120] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.96, rt: 174.50; ) [2024-10-16 14:57:52.031148] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:52.031190] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.96, rt: 2737.75; ) [2024-10-16 14:57:52.031205] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:52.031216] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:52.031770] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:52.041798] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=33] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:52.041813] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:52.041837] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:52.041846] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:52.042077] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:52.042441] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C441C-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=1886, avg_log_submit_to_slide_time=2107) [2024-10-16 14:57:52.045786] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061872044628000, start_id=1729061872044628000, end_id=1729061872044628001) [2024-10-16 14:57:52.056951] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061872056606000, start_id=1729061872056606000, end_id=1729061872056606001) [2024-10-16 14:57:52.056975] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:52.056993] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061872056988], gts:1729061872056606000, latest_srr:[mts=1729061872056988]}) [2024-10-16 14:57:52.070296] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.082350] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.094881] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85071-0-0] [lt=19][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497520, log_task={header:{begin_lsn:{lsn:207589560107}, end_lsn:{lsn:207589560229}, log_id:218497520, min_scn:{val:1729061871651739003, v:0}, max_scn:{val:1729061871651739003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589559985}, prev_proposal_id:81, committed_end_lsn:{lsn:207589560107}, data_checksum:940009244, accum_checksum:1576318924, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061871994785, freeze_ts:1729061871994786, submit_ts:1729061871994790, flushed_ts:1729061871995582, gen_to_freeze cost time:1, gen_to_submit cost time:5, submit_to_flush cost time:792}, fs_cb_begin_ts=1729061872094875, log_life_time=100090) [2024-10-16 14:57:52.096312] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:52.108444] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=18] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:52.117644] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3AD0BF-0-0] [lt=11] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.733866291046142578e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=866, Throttle Trigger(MB)=520, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.150176627216012903e+01) [2024-10-16 14:57:52.117807] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061872116747000, start_id=1729061872116747000, end_id=1729061872116747001) [2024-10-16 14:57:52.118815] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061872117807000, start_id=1729061872117807000, end_id=1729061872117807001) [2024-10-16 14:57:52.127047] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:52.127065] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:52.127271] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=896, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:52.127285] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=36, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:52.127742] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=2][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:52.127761] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:52.127899] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FFA5-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=1758, avg_log_submit_to_slide_time=2236) [2024-10-16 14:57:52.137075] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3568, mem_used=8318976) [2024-10-16 14:57:52.142623] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:52.142789] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061872142788, bytes: 0, bw: 0.000000 MB/s, add_ts: 1003711, add_bytes: 0 [2024-10-16 14:57:52.145994] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:52.151945] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA33-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061872151686000, start_id=1729061872151686000, end_id=1729061872151686001) [2024-10-16 14:57:52.151974] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA34-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061872151686000, start_id=1729061872151686000, end_id=1729061872151686001) [2024-10-16 14:57:52.152224] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA37-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061872151686000, start_id=1729061872151686000, end_id=1729061872151686001) [2024-10-16 14:57:52.152364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA36-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061872151686000, start_id=1729061872151686000, end_id=1729061872151686001) [2024-10-16 14:57:52.152385] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA35-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061872151686000, start_id=1729061872151686000, end_id=1729061872151686001) [2024-10-16 14:57:52.154613] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=12] current monitor number(seq_=-1) [2024-10-16 14:57:52.155003] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061872154860000, start_id=1729061872154860000, end_id=1729061872154860001) [2024-10-16 14:57:52.155063] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:52.155077] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:52.155492] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D58EA-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10791) [2024-10-16 14:57:52.158618] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061872157000000, start_id=1729061872157000000, end_id=1729061872157000001) [2024-10-16 14:57:52.170147] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:52.171704] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.182407] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.194804] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5036-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:52.194819] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5036-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:52.194827] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D8-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061872186055, keepalive_ts:-1}) [2024-10-16 14:57:52.194857] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D8-0-0] [lt=29] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061872186055, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:52.194935] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85073-0-0] [lt=24] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:52.194944] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85073-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:52.194956] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85073-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497521, log_task={header:{begin_lsn:{lsn:207589560229}, end_lsn:{lsn:207589560351}, log_id:218497521, min_scn:{val:1729061871651739004, v:0}, max_scn:{val:1729061871651739004, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589560107}, prev_proposal_id:81, committed_end_lsn:{lsn:207589560229}, data_checksum:940009244, accum_checksum:772843643, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061872094864, freeze_ts:1729061872094864, submit_ts:1729061872094868, flushed_ts:1729061872096344, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1476}, fs_cb_begin_ts=1729061872194950, log_life_time=100086) [2024-10-16 14:57:52.197946] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:52.198066] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D87-0-0] [lt=12] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061887008855000, latest_log_ts:{val:1729061870498146001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:52.198121] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D87-0-0] [lt=52] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:52.210734] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587D-0-0] [lt=15] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:52.210827] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587D-0-0] [lt=16][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:52.213945] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=17] Garbage Collector is running(seq_=656700, gc_interval=10000000) [2024-10-16 14:57:52.213974] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=12] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:52.213987] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=8] gc_check_member_list_ cost time(ret=0, time_us=28) [2024-10-16 14:57:52.213995] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:52.214688] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=6] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:52.214704] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=12] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:52.214710] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=7) [2024-10-16 14:57:52.214714] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:52.227432] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1914) [2024-10-16 14:57:52.231665] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88A-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:52.231690] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88A-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061872151686000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061872151686000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:52.232066] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:52.235074] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88A-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061872234853000, start_id=1729061872234853000, end_id=1729061872234853001) [2024-10-16 14:57:52.235150] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88A-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10791, avg_group_log_size=567) [2024-10-16 14:57:52.235161] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88A-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10791) [2024-10-16 14:57:52.235412] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=228, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:52.235422] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10791) [2024-10-16 14:57:52.235430] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=104, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:52.235445] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:52.235647] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F958F-0-0] [lt=24] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:52.235660] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F958F-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=745, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:52.235665] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F958F-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=357, avg_log_submit_to_slide_time=713) [2024-10-16 14:57:52.235749] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:52.245768] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061872244336000, start_id=1729061872244336000, end_id=1729061872244336001) [2024-10-16 14:57:52.245969] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD0E-0-0] [lt=8][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:52.248679] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21681][T1_L0_G0][T1][YB4A0A000407-00061E9AA26D4EA2-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510340117072, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410616, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:52.269893] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7841-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:52.271748] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.278822] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB9C-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12287, add_timestamp:1729061872278811}) [2024-10-16 14:57:52.278836] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E58-0-0] [lt=11] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12287, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:57:52.281936] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D23-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:52.282125] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=188, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:52.282138] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:52.282163] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:52.282469] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.283588] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB9C-0-0] [lt=14][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:52.283744] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E58-0-0] [lt=17] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12287, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061872283731, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061872283731, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061872283731, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:52.283765] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E58-0-0] [lt=21] success to process renew task(task={tenant_id:1, table_id:12287, add_timestamp:1729061872278811}, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061872283731, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061872283731, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061872283731, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:57:52.284511] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=19] ls blacklist refresh finish(cost_time=6280) [2024-10-16 14:57:52.287911] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C10E4-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061872287033000, start_id=1729061872287033000, end_id=1729061872287033001) [2024-10-16 14:57:52.294975] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85075-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:52.295177] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107484][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D3D-0-0] [lt=13] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:52.295195] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107484][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D3D-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:52.312370] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:52.312390] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:52.312407] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373666684, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:52.317828] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061872317741000, start_id=1729061872317741000, end_id=1729061872317741001) [2024-10-16 14:57:52.318991] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061872318800000, start_id=1729061872318800000, end_id=1729061872318800001) [2024-10-16 14:57:52.327400] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:52.328200] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=6] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=71, mem_used=2079744) [2024-10-16 14:57:52.332436] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=12] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:52.333052] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:52.333062] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28020300, tenant_id=1002) [2024-10-16 14:57:52.333078] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:52.342697] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:52.343008] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=198, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:52.343025] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=215, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:52.343042] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1333) [2024-10-16 14:57:52.343059] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:52.352823] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E9-0-0] [lt=28] [LS_LOCATION]dump tenant ls location caches(tenant_id=1, tenant_ls_locations=[{cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061869994524, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:52.352847] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:52.352857] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E9-0-0] [lt=33] [LS_LOCATION]dump tenant ls location caches(tenant_id=1001, tenant_ls_locations=[{cache_key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061869995299, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:52.352863] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:52.352871] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:52.352875] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:52.352878] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:52.352881] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17117) [2024-10-16 14:57:52.352868] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E9-0-0] [lt=10] [LS_LOCATION]dump tenant ls location caches(tenant_id=1002, tenant_ls_locations=[{cache_key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061869996001, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061869996000, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061869996002, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}]}, {cache_key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061869995997, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:52.352895] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E9-0-0] [lt=26] [LS_LOCATION]dump tenant ls location caches(tenant_id=1003, tenant_ls_locations=[{cache_key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061869996677, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:52.352903] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6E9-0-0] [lt=9] [LS_LOCATION]dump tenant ls location caches(tenant_id=1004, tenant_ls_locations=[{cache_key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061869997396, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061869997398, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061869997399, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}]}, {cache_key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061869997397, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:57:52.355050] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061872354716000, start_id=1729061872354716000, end_id=1729061872354716001) [2024-10-16 14:57:52.355860] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=42, this=0x2b0e07718280, extra_info="") [2024-10-16 14:57:52.355886] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=25] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=1248, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:57:52.355890] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:57:52.355893] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=-535, this=0x2b0e07718910, extra_info="") [2024-10-16 14:57:52.355896] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=755, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:57:52.357351] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:52.357363] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54641860, tenant_id=1003) [2024-10-16 14:57:52.357375] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:52.358740] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061872357960000, start_id=1729061872357960000, end_id=1729061872357960001) [2024-10-16 14:57:52.370411] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B29FF6757-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061872368695000, start_id=1729061872368695000, end_id=1729061872368695001) [2024-10-16 14:57:52.370679] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7843-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:52.372232] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.374702] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5480-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:52.374713] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5480-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=810, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:52.375003] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561384, miss_cnt:1, cache_read_size:152590692244, hit_ratio:"9.999999928346942069e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:52.375051] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:52.377850] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601446, offset=9637848}, end_cursor:ObLogCursor{file_id=192, log_id=47601447, offset=9638132}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e48050, len:4136, log_data_len:284, seq:47601446, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:52.381191] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4AE-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061872378657000, start_id=1729061872378657000, end_id=1729061872378657001) [2024-10-16 14:57:52.381379] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=66] get number(ret=0, service_type_=0, range=1, base_id=1729061872378657000, start_id=1729061872378657001, end_id=1729061872378657002) [2024-10-16 14:57:52.381773] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0e7902c7d0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:193016628} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059900785870 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:81} seq_no={branch:0, seq:4139}, prev=this=0x2b0e7902a150 trans_version={val:1729060072220252000, v:0} scn={val:1729060072220252000, v:0} tx_id={txid:193013940} prev=0x2b0e79029fc8 next=(nil) modify_count=1 acc_checksum=3499626446 version=1729059900785870 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:4257}, snapshot_version={val:1729061872378657001, v:0}, *this={this=0x2b0e79029e78 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e7902a150 latest_compact_node=(nil) max_trans_version={val:1729060072220252000, v:0} max_trans_id=193013940 max_elr_trans_version={val:1729060072220252000, v:0} max_elr_trans_id=193013940 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729060072220252000, v:0} min_modify_scn={val:1729060072220252000, v:0}}) [2024-10-16 14:57:52.381801] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0e7902c7d0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:193016628} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059900785870 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:81} seq_no={branch:0, seq:4139}, ctx={this:0x2b0e0b6d2080, ls_id:{id:1}, ls:0x2b0e076c4150, branch:0, timeout:1729061902279781, tablet_id:{id:344}, table_iter:0x2b0e0b6d1210, table_version:1717560274536320, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061902279781, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061872378657001, v:0}, tx_id:{txid:193016628}, scn:{branch:0, seq:1320}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e076d2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:193016628}, tx_desc:{this:0x2b0e0fd1c090, tx_id:{txid:193016628}, state:2, addr:"10.0.4.8:2890", tenant_id:1001, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:-1, active_ts:1729061872369653, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1729061902369646, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061872377346, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0df705ba10, mem_ctx:0x2b0df705cf90, tx_scn:{branch:0, seq:4139}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:52.381864] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=63][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560274536320 max_table_version=1717560274536320 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:193016628} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:193016628}, data:0x2b0e0b6ce438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729059900785870, seq_no:{branch:0, seq:4139}, scn:{val:4611686018427387903, v:0}, column_cnt:12}) [2024-10-16 14:57:52.381878] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":1001},{"BIGINT":226},{"BIGINT":1},{"VARCHAR":"10.0.4.8", collation:"utf8mb4_general_ci"},{"BIGINT":2890}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560274536320 max_table_version=1717560274536320 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:193016628} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:57:52.381892] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:52.381898] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=344, col_descs=[cnt:12, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=12,cells=[{"BIGINT":1001},{"BIGINT":226},{"BIGINT":1},{"VARCHAR":"10.0.4.8", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT":2890},{"TIMESTAMP":"2024-10-16 06:57:52.369653"},{"TIMESTAMP":"2024-10-16 06:57:52.369653"},{"BIGINT UNSIGNED":1729061698605210000},{"BIGINT":688},{"BIGINT":8192},{"BIGINT UNSIGNED":0},{"BIGINT":0},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:52.381949] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=32][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:52.382467] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=49] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601447, offset=9638132}, end_cursor:ObLogCursor{file_id=192, log_id=47601448, offset=9638406}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72a2050, len:3852, log_data_len:274, seq:47601447, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:52.382535] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.382789] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D25-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:52.384098] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=10] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6283}, to_seq_no={branch:0, seq:1321}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560274536320 max_table_version=1717560274536320 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:193016628} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:57:52.384162] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:145) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=47] transaction rpc statistics(total_rt=146, total_process=2, avg_rt=48) [2024-10-16 14:57:52.384185] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FFAD-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:52.384196] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FFAD-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20621, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:52.384847] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601448, offset=9638406}, end_cursor:ObLogCursor{file_id=192, log_id=47601449, offset=9638680}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2c90050, len:3578, log_data_len:274, seq:47601448, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:52.385091] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601449, offset=9638680}, end_cursor:ObLogCursor{file_id=192, log_id=47601450, offset=9638954}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72be050, len:3304, log_data_len:274, seq:47601449, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:52.385146] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598970, miss_cnt:9, cache_read_size:149777925036, hit_ratio:"9.999999345925378869e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:52.387439] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [21933][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=11][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f8f2cbef8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:193016628} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059900788494 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:377} seq_no={branch:0, seq:9108}, prev=this=0x2b0f8f2c83c8 trans_version={val:1729060072220252000, v:0} scn={val:1729060072220252000, v:0} tx_id={txid:193013940} prev=0x2b0f8f2c8240 next=(nil) modify_count=1 acc_checksum=1282202574 version=1729059900788494 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:377} seq_no={branch:0, seq:7408}, snapshot_version={val:1729061872386450000, v:0}, *this={this=0x2b0f8f2c7fc8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f8f2c83c8 latest_compact_node=(nil) max_trans_version={val:1729060072220252000, v:0} max_trans_id=193013940 max_elr_trans_version={val:1729060072220252000, v:0} max_elr_trans_id=193013940 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729060072220252000, v:0} min_modify_scn={val:1729060072220252000, v:0}}) [2024-10-16 14:57:52.387478] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [21933][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f8f2cbef8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:193016628} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729059900788494 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:377} seq_no={branch:0, seq:9108}, ctx={this:0x2b0e0b9ce080, ls_id:{id:1}, ls:0x2b0e076c4150, branch:0, timeout:1729061902286958, tablet_id:{id:372}, table_iter:0x2b0e0b9cd210, table_version:1717560274799912, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729061902286958, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061872386450000, v:0}, tx_id:{txid:193016628}, scn:{branch:0, seq:9105}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e076d2a50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:193016628}, tx_desc:{this:0x2b0ed958e2d0, tx_id:{txid:193016628}, state:2, addr:"10.0.4.8:2890", tenant_id:1001, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:-1, active_ts:1729061872369653, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1729061902369646, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"10.0.4.7:2890", epoch:263984618602773115, first_scn:{branch:0, seq:4139}, last_scn:{branch:0, seq:7377}, last_touch_ts:0}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061872377346, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0df705ba10, mem_ctx:0x2b0df705cf90, tx_scn:{branch:0, seq:9108}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:57:52.387544] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [21933][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=65][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560274536320 max_table_version=1717560274799912 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:193016628} ls_id=1 row_callback[alloc:2, free:0, unsubmit:2] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=4 pending_log_size=171 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(2,4,0,0,5,1,1,0)|]}}, arg={tx_id:{txid:193016628}, data:0x2b0e0b9ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729059900788494, seq_no:{branch:0, seq:9108}, scn:{val:4611686018427387903, v:0}, column_cnt:12}) [2024-10-16 14:57:52.387561] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [21933][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":1001},{"BIGINT":226},{"VARCHAR":"10.0.4.8", collation:"utf8mb4_general_ci"},{"BIGINT":2890},{"BIGINT":1}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560274536320 max_table_version=1717560274799912 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:193016628} ls_id=1 row_callback[alloc:2, free:0, unsubmit:2] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=4 pending_log_size=171 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(2,4,0,0,5,1,1,0)|]}}) [2024-10-16 14:57:52.387578] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [21933][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=15][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:57:52.387583] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [21933][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=372, col_descs=[cnt:12, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=12,cells=[{"BIGINT":1001},{"BIGINT":226},{"VARCHAR":"10.0.4.8", collation:"utf8mb4_general_ci", coercibility:"INVALID"},{"BIGINT":2890},{"BIGINT":1},{"TIMESTAMP":"2024-10-16 06:57:52.377481"},{"TIMESTAMP":"2024-10-16 06:57:52.377481"},{"BIGINT UNSIGNED":1729061698605210000},{"BIGINT":2},{"BIGINT":3187600262},{"LONGTEXT":"„", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"},{"VARCHAR":"636865636B636F6C0000160A8D8F9D9C0EE2DD96E61100A59BA5F40BE2DD96E611EEC5E84F99A1829310EAAED9FD11DF89E1BA0ADF89E1BA0A", collation:"binary", coercibility:"INVALID"},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:57:52.387643] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [21933][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=42][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:57:52.387928] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10117}, to_seq_no={branch:0, seq:9106}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560274536320 max_table_version=1717560274799912 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:193016628} ls_id=1 row_callback[alloc:2, free:0, unsubmit:2] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=3 pending_log_size=171 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=2, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(2,3,0,0,5,2,2,0)|]}}) [2024-10-16 14:57:52.389412] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E99871BEC34-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061872388528000, start_id=1729061872388528000, end_id=1729061872388528001) [2024-10-16 14:57:52.395086] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:52.395181] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=203, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:52.395199] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=127, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:52.395234] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:52.403547] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183786218, max_lsn:114618631604, max_scn:{val:1729061872388528000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45685691230}, reuse_lsn:{lsn:114618631604}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114576688564}}, last_submit_lsn:{lsn:114618629760}, last_submit_end_lsn:{lsn:114618631604}, last_submit_log_id:183786218, last_submit_log_pid:33705, max_flushed_lsn:{lsn:114618629760}, max_flushed_end_lsn:{lsn:114618631604}, max_flushed_log_pid:33705, committed_end_lsn:{lsn:114618631604}, last_slide_log_id:183786218, last_slide_scn:{val:1729061872388528000, v:0}, last_slide_lsn:{lsn:114618629760}, last_slide_end_lsn:{lsn:114618631604}, last_slide_log_pid:33705, last_slide_log_accum_checksum:895056621, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0df99e2530}, [StateMgr]={this:0x2b0df99e6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, leader:"10.0.4.7:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183786219, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261592457, palf_role_change_cb:0x2b0df99ed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, reconfig_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, persistent_config_version:{proposal_id:33705, config_seq:14}, ms_ack_list:0, resend_config_version:{proposal_id:33705, config_seq:14}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258289572, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258289572, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0df99e3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261792660, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729029056668344001, v:0}, min_block_id:1702, min_block_min_scn:{val:1729022447183332001, v:0}, base_lsn_for_block_gc:{lsn:114547838976}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, log_config_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, log_snapshot_meta:{version:1, base_lsn:{lsn:114547838976}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:233218048}, readable_log_tail:{lsn:233218048}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114682048512}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/meta", dir_fd:146, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:35201024, block_header_serialize_buf_:0x2b0df99e89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114618631604}, readable_log_tail:{lsn:114618631604}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114614943744}, min_scn:{val:1729061506448191001, v:0}, curr_block_id:1708, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/log", dir_fd:148, min_block_id:1702, max_block_id:1709, curr_writable_block_id:1708}, logical_block_size_:67104768, curr_block_writable_size_:63416908, block_header_serialize_buf_:0x2b0df99ea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261592575, this:0x2b0df99e6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33705, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33704, majority_max_lsn:{lsn:45686897415}, saved_end_lsn:{lsn:45686897415}, last_submit_prepare_req_time_us:1722494261592686, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0df99e69b0}) [2024-10-16 14:57:52.403699] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=149] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061872389698, last_flushed_end_lsn:{lsn:114618631604}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061872389855, last_flushed_end_lsn:{lsn:114618631604}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061872390038, last_flushed_end_lsn:{lsn:114618631604}}]) [2024-10-16 14:57:52.403710] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183786219, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061872389698, last_flushed_end_lsn:{lsn:114618631604}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061872389855, last_flushed_end_lsn:{lsn:114618631604}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061872390038, last_flushed_end_lsn:{lsn:114618631604}}]) [2024-10-16 14:57:52.418675] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=10][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:52.432356] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:52.442044] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=6] worker thread began to shrink(tenant_->id()=1003, group_id_=1, token=0) [2024-10-16 14:57:52.442844] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:52.442857] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:52.443093] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=25] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:52.445996] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] submit log success(service_type_=0, last_id=1729061889029444000, limited_id=1729061889029444000) [2024-10-16 14:57:52.446018] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] submit log with lock success(service_type_=0, last_id=1729061889029444000, limited_id=1729061889029444000) [2024-10-16 14:57:52.446023] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061872445301000, start_id=1729061872445301000, end_id=1729061872445301001) [2024-10-16 14:57:52.446086] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107515][T1003_L0_G1][T1003][Y0-0000000000000000-0-0] [lt=10] worker thread exit(tenant_->id()=1003, workers_.get_size()=2) [2024-10-16 14:57:52.446099] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107515][T1003_L0_G1][T0][Y0-0000000000000000-0-0] [lt=10] tenant thread end_run(id_=1003, ret=0, thread_count_=172) [2024-10-16 14:57:52.446548] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] update id meta(ret=0, service_type=0, limited_id=1729061889029444000, latest_scn={val:1729061872445301001, v:0}, *this={tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061889029444000, latest_log_ts:{val:1729061872445301001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:52.446613] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=64] handle submit callback(service_type_=0, success=true, used_ts=618, log_ts={val:1729061872445301001, v:0}, limited_id=1729061889029444000) [2024-10-16 14:57:52.446620] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:52.446633] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:52.446863] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061872446329000, start_id=1729061872446329000, end_id=1729061872446329001) [2024-10-16 14:57:52.446990] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=5] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:52.456757] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257794, offset=39042063}, end_cursor:ObLogCursor{file_id=197, log_id=49257795, offset=39042346}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db730a050, len:1009, log_data_len:283, seq:49257794, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:52.458830] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB379-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061872457948000, start_id=1729061872457948000, end_id=1729061872457948001) [2024-10-16 14:57:52.460453] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257795, offset=39042346}, end_cursor:ObLogCursor{file_id=197, log_id=49257796, offset=39042628}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73b2050, len:726, log_data_len:282, seq:49257795, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:52.463684] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257796, offset=39042628}, end_cursor:ObLogCursor{file_id=197, log_id=49257797, offset=39042910}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7346050, len:444, log_data_len:282, seq:49257796, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:52.463930] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257797, offset=39042910}, end_cursor:ObLogCursor{file_id=197, log_id=49257798, offset=39043192}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7286050, len:4258, log_data_len:282, seq:49257797, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:52.463977] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657060, miss_cnt:17, cache_read_size:8050177976, hit_ratio:"9.999997286818853581e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:52.464003] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650321, miss_cnt:32, cache_read_size:8050430473, hit_ratio:"9.999994892287358272e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:52.470129] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=172, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:52.470148] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:52.470385] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:52.470449] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:52.470451] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:52.470455] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:52.470463] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3544, avg_group_log_size=354) [2024-10-16 14:57:52.470472] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:52.470599] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=252, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:52.470619] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=69, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:52.470635] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:52.470869] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1090-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=506, avg_log_submit_to_slide_time=758) [2024-10-16 14:57:52.472291] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.481205] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8A-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:52.481228] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8A-0-0] [lt=21] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:52.482538] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B91FC-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=268, avg_log_submit_to_slide_time=20611) [2024-10-16 14:57:52.482552] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:52.482595] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.495478] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5483-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:52.495492] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5483-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20589, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:52.518003] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061872518001000, start_id=1729061872518001000, end_id=1729061872518001001) [2024-10-16 14:57:52.519044] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061872518001000, start_id=1729061872518001000, end_id=1729061872518001001) [2024-10-16 14:57:52.519215] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061872317741000, v:0}, new_max_version={val:1729061872317741000, v:0}, last_min_version={val:1729061872056606000, v:0}, last_max_version={val:1729061872056606000, v:0}, total_time_us=2095, delta_us=201472) [2024-10-16 14:57:52.519261] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=44] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061872317741000, v:0}, min_version={val:1729061872317741000, v:0}, max_version={val:1729061872317741000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:52.527032] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=25] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a5feb0, tx_id:{txid:199664361}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061872526461, active_ts:1729061872526461, commit_ts:-1, finish_ts:1729061872526461, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061902526456, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061872519059, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:52.527301] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:52.527313] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1342) [2024-10-16 14:57:52.527536] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2036) [2024-10-16 14:57:52.527584] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:52.528114] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=40] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a540f0, tx_id:{txid:199664362}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061872527608, active_ts:1729061872527608, commit_ts:-1, finish_ts:1729061872527608, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1729061902527602, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061872527607, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:52.533801] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:52.533819] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:52.533835] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444582284, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:52.542778] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:52.542826] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:52.542845] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:52.543089] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:52.543422] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4248-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:52.543437] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4248-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=756, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:52.555177] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061872555082000, start_id=1729061872555082000, end_id=1729061872555082001) [2024-10-16 14:57:52.555451] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=194, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:52.555466] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=257, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:52.555494] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:52.557305] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061872557205000, start_id=1729061872557205000, end_id=1729061872557205001) [2024-10-16 14:57:52.557490] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061872557480], gts:1729061872557205000, latest_srr:[mts=1729061872557480]}) [2024-10-16 14:57:52.558788] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061872558262000, start_id=1729061872558262000, end_id=1729061872558262001) [2024-10-16 14:57:52.559124] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=335, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:52.559142] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=53, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:52.559162] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10224) [2024-10-16 14:57:52.559175] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:52.559273] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1F3-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:52.559279] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1F3-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=661, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:52.559289] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD1F3-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=11, avg_log_gen_to_submit_time=39, avg_log_submit_to_flush_time=426, avg_log_submit_to_slide_time=621) [2024-10-16 14:57:52.568605] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.569095] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.569220] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.569394] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.572361] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.572982] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=16) [2024-10-16 14:57:52.582655] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.585014] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCB1E-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061872584571000, start_id=1729061872584571000, end_id=1729061872584571001) [2024-10-16 14:57:52.596663] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:52.596687] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:52.596999] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=23] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1002}, {id:1003}]) [2024-10-16 14:57:52.610270] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5559, mem_used=10398720) [2024-10-16 14:57:52.627353] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2036, avg_group_log_size=185) [2024-10-16 14:57:52.627372] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2036) [2024-10-16 14:57:52.632667] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:52.643532] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F424C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=357, avg_log_submit_to_slide_time=774) [2024-10-16 14:57:52.646146] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061872645525000, start_id=1729061872645525000, end_id=1729061872645525001) [2024-10-16 14:57:52.646222] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10101, avg_group_log_size=594) [2024-10-16 14:57:52.646233] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10102) [2024-10-16 14:57:52.646521] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10101) [2024-10-16 14:57:52.652462] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA38-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061872650827000, start_id=1729061872650827000, end_id=1729061872650827001) [2024-10-16 14:57:52.652583] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=12] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:57:52.652691] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA3C-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061872652685000, start_id=1729061872652685000, end_id=1729061872652685001) [2024-10-16 14:57:52.652725] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA39-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061872652685000, start_id=1729061872652685000, end_id=1729061872652685001) [2024-10-16 14:57:52.652761] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA3B-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061872652685000, start_id=1729061872652685000, end_id=1729061872652685001) [2024-10-16 14:57:52.656082] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EAC-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:52.656094] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EAC-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27801, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:52.656098] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EAC-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=472, avg_log_submit_to_slide_time=27795) [2024-10-16 14:57:52.656106] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EAC-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173259, log_task={header:{begin_lsn:{lsn:14352387755}, end_lsn:{lsn:14352387877}, log_id:115173259, min_scn:{val:1729061872151686004, v:0}, max_scn:{val:1729061872151686004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352387633}, prev_proposal_id:17, committed_end_lsn:{lsn:14352387755}, data_checksum:940009244, accum_checksum:204966420, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061872555294, freeze_ts:1729061872555294, submit_ts:1729061872555297, flushed_ts:1729061872555475, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:178}, fs_cb_begin_ts=1729061872656074, log_life_time=100780) [2024-10-16 14:57:52.656156] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:52.659335] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CE-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:52.659365] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CE-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061872151686004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061872151686004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:52.664803] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CE-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061872664288000, start_id=1729061872664288000, end_id=1729061872664288001) [2024-10-16 14:57:52.665406] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:52.668052] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:57:52.668067] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:57:52.668073] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:57:52.668078] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:52.670203] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=115, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:787547, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787548, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787549, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:787550, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:57:52.670363] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=158] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:52.672086] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:52.672424] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.677424] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=10] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:57:52.678236] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061872677033000, start_id=1729061872677033000, end_id=1729061872677033001) [2024-10-16 14:57:52.682716] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.695122] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8507D-0-0] [lt=15][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497526, log_task={header:{begin_lsn:{lsn:207589560839}, end_lsn:{lsn:207589560961}, log_id:218497526, min_scn:{val:1729061872151686004, v:0}, max_scn:{val:1729061872151686004, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589560717}, prev_proposal_id:81, committed_end_lsn:{lsn:207589560839}, data_checksum:940009244, accum_checksum:4145538844, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061872595096, freeze_ts:1729061872595097, submit_ts:1729061872595100, flushed_ts:1729061872596925, gen_to_freeze cost time:1, gen_to_submit cost time:4, submit_to_flush cost time:1825}, fs_cb_begin_ts=1729061872695117, log_life_time=100021) [2024-10-16 14:57:52.696608] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:52.699617] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=17] gc stale ls task succ [2024-10-16 14:57:52.714067] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5B-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:52.714104] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5B-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061872584571000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061872584571000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:52.715955] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15E6-0-0] [lt=12] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:52.717399] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] no need to update min/max version(new_version={val:1729061872317741000, v:0}, min_version={val:1729061872317741000, v:0}, max_version={val:1729061872317741000, v:0}) [2024-10-16 14:57:52.717587] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C5B-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061872717306000, start_id=1729061872717306000, end_id=1729061872717306001) [2024-10-16 14:57:52.719159] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061872718333000, start_id=1729061872718333000, end_id=1729061872718333001) [2024-10-16 14:57:52.719766] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:52.727112] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=25] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:52.727889] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD359-0-0] [lt=22] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:52.727901] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD359-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=630, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:52.742844] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:52.754717] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EAE-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:52.754739] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EAE-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:52.755411] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061872753530000, start_id=1729061872753530000, end_id=1729061872753530001) [2024-10-16 14:57:52.758804] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:52.758824] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:52.758830] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=5] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:52.758898] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061872757714000, start_id=1729061872757714000, end_id=1729061872757714001) [2024-10-16 14:57:52.770920] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3080) [2024-10-16 14:57:52.770948] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3544) [2024-10-16 14:57:52.771151] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1093-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:52.771168] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1093-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=654, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:52.772315] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:52.772468] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.776164] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:52.780099] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:52.780117] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:52.780126] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=7] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=53) [2024-10-16 14:57:52.782230] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:52.783321] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.788518] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.788521] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.788620] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.788657] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.789195] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.789241] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.789277] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.789480] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.789497] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.789622] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.789638] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:52.810711] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [21910][T1001_Compactio][T1001][YB4A0A000407-00061E9973FD0D6F-0-0] [lt=11] InfoPoolResizeTask(cost_ts=6) [2024-10-16 14:57:52.817433] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1, min_version={val:1729061872317741001, v:0}, base_version={val:1729061872317741000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061872518001000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061872817420, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061872317741001, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061872731773, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061872557205000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061872779791, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:52.818290] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061872817407000, start_id=1729061872817407000, end_id=1729061872817407001) [2024-10-16 14:57:52.818504] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] get_min_unreplayed_log_info(lsn={lsn:207589561205}, scn={val:1729061872650827002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061872795650, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589561205}, next_to_submit_scn_:{val:1729061872650827002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589561083}, end_lsn:{lsn:207589561205}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061872650827001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061872650827001, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:393300213, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061872650827001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598975, miss_cnt:9, cache_read_size:149777925646, hit_ratio:"9.999999345925403293e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:57:52.828032] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD35C-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=260, avg_log_submit_to_slide_time=609) [2024-10-16 14:57:52.831211] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1001}, {id:1}]) [2024-10-16 14:57:52.832960] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:52.843295] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=222, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:52.843317] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=102, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:52.843322] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=37] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:52.846194] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061872844730000, start_id=1729061872844730000, end_id=1729061872844730001) [2024-10-16 14:57:52.855002] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:52.860168] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=24] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5548, mem_used=10398720) [2024-10-16 14:57:52.872007] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061868866109, last_first_register_time_us=1722494157071428, register_time_us=1729061868866109, curr_time_us=1729061872872005) [2024-10-16 14:57:52.872039] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061872872026, last_submit_register_req_time_us=1729061872872026, register_parent_reason=3) [2024-10-16 14:57:52.872052] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=13] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:52.872528] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.882783] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=55, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:52.882795] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:52.883377] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.893852] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:52.902141] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=15] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:52.903378] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:52.904251] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:52.904334] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=27] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061867249743000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:52.904939] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061862902614, end_time=1729061872904938, dag_status={}) [2024-10-16 14:57:52.904954] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:52.904964] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:52.904971] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:52.904977] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:52.904982] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:52.904987] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:52.904993] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:52.904998] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:52.905003] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:52.905009] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905019] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905025] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905032] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905039] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905047] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905053] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905060] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905066] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905073] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905085] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905092] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905106] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905113] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905119] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905126] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905132] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905139] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905145] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905151] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905158] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905165] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905172] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905179] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905185] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905191] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905198] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905204] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905211] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905217] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905224] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905231] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905238] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905244] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905251] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905257] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905264] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905271] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905277] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905283] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905290] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905297] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905308] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905315] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905321] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905328] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905335] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905341] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905347] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905353] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905360] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905367] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905373] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905380] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905387] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905393] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905406] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:52.905413] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:52.905419] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:52.905424] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:52.905429] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:52.905434] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:52.905438] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:52.905442] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:52.905446] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:52.905449] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:52.905453] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:52.905463] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:52.908110] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=17] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:52.916696] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D78-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:52.916733] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D78-0-0] [lt=25] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061872652685003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061872652685003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:52.917669] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D78-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061872916520000, start_id=1729061872916520000, end_id=1729061872916520001) [2024-10-16 14:57:52.919490] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=58] get number(ret=0, service_type_=0, range=1, base_id=1729061872919391000, start_id=1729061872919391000, end_id=1729061872919391001) [2024-10-16 14:57:52.924099] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:52.925231] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D78-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061872925071000, start_id=1729061872925071000, end_id=1729061872925071001) [2024-10-16 14:57:52.925326] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D78-0-0] [lt=20] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:52.925347] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D78-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:52.928427] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:52.928447] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:52.928759] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:52.935147] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107484][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2017-0-0] [lt=15] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:52.935176] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107484][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2017-0-0] [lt=23] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:52.936644] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE6-0-0] [lt=23] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061872903793, keepalive_ts:-1}) [2024-10-16 14:57:52.936663] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE6-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061872903793, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:52.944334] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2296361, replace_num=31457) [2024-10-16 14:57:52.944351] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:52.944430] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3606, mem_used=8318976) [2024-10-16 14:57:52.945072] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4255-0-0] [lt=29] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:52.945095] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4255-0-0] [lt=22] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=795, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:52.951859] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=20] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:52.954337] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:57:52.954349] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=10] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:52.954355] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:52.954360] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:52.955573] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061872954239000, start_id=1729061872954239000, end_id=1729061872954239001) [2024-10-16 14:57:52.956596] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:57:52.956963] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=365] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:52.957290] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EB2-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:52.957823] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:52.957840] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:52.957848] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=161) [2024-10-16 14:57:52.959056] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061872958325000, start_id=1729061872958325000, end_id=1729061872958325001) [2024-10-16 14:57:52.969203] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:52.969223] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=15] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:57:52.969235] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:52.969261] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] TxData Memory Statistic(logstream info): (ls_id={id:1}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=7) [2024-10-16 14:57:52.969273] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=10] TxData Memory Statistic(logstream info): (ls_id={id:1001}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=1) [2024-10-16 14:57:52.969276] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=3] TxData Memory Statistic(logstream info): (ls_id={id:1002}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=1) [2024-10-16 14:57:52.969279] INFO [STORAGE] get_ls_tx_data_memory_info_ (ob_tenant_freezer.cpp:660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=3] TxData Memory Statistic(logstream info): (ls_id={id:1003}, Frozen TxData Memory(MB)=0, Active TxData Memory(MB)=1) [2024-10-16 14:57:52.969283] INFO [STORAGE] check_and_freeze_tx_data_ (ob_tenant_freezer.cpp:566) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=3] TxData Memory Statistic : (Tenant Total Memory(MB)=1024, Tenant Frozen TxData Memory(MB)=0, Tenant Active TxData Memory(MB)=10, Freeze TxData Trigger Memory(MB)=20, Total TxDataTable Hold Memory(MB)=24, Total TxDataTable Memory Limit(MB)=204) [2024-10-16 14:57:52.969344] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=10] ====== tenant freeze timer task ====== [2024-10-16 14:57:52.973265] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061868965202, last_first_register_time_us=1722494150998907, register_time_us=1729061868965202, curr_time_us=1729061872973263) [2024-10-16 14:57:52.973320] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=30] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061872973297, last_submit_register_req_time_us=1729061872973297, register_parent_reason=3) [2024-10-16 14:57:52.973331] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:52.973537] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=27] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:52.973656] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.973654] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3667) [2024-10-16 14:57:52.974041] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE784D-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1817) [2024-10-16 14:57:52.974232] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:52.979057] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=14] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:52.981361] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=15] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:52.983063] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D31-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:52.983306] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:52.983437] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:52.987141] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:52.987895] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=10] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:52.987923] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=28] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:52.987952] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=24] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:52.988024] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=11] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061889029444000, latest_log_ts:{val:1729061872445301001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:52.988071] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:52.988091] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:57:52.988099] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=0, rec_log_ts={val:1729061872445301001, v:0}) [2024-10-16 14:57:52.988106] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:52.988109] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:52.988119] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:52.988133] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:57:52.988143] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:52.988148] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:53.015133] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=10] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:53.015845] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=11] ====== tenant freeze timer task ====== [2024-10-16 14:57:53.015877] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=14] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151574520, tenant_id=1) [2024-10-16 14:57:53.015895] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=11] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:53.018441] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061873017952000, start_id=1729061873017952000, end_id=1729061873017952001) [2024-10-16 14:57:53.028478] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=16] PNIO [ratelimit] time: 1729061873028475, bytes: 4446818863441, bw: 0.532381 MB/s, add_ts: 1007217, add_bytes: 562271 [2024-10-16 14:57:53.028757] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=224, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:53.028776] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=58, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:53.028798] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:53.029185] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD362-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:53.029194] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD362-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=641, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:53.032927] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 174.50; ) [2024-10-16 14:57:53.032951] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:53.033006] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 154.50; ) [2024-10-16 14:57:53.033015] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:53.033027] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:53.034406] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:53.037289] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Cache wash clean map node details(ret=0, clean_node_count=31, clean_time=0, clean_start_pos=1384108, clean_num=62914) [2024-10-16 14:57:53.042598] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21575][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:53.042691] WDIAG read_multi_row (ob_table_access_helper.h:238) [21575][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:53.042911] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:57:53.042926] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:53.042941] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=291970964, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:53.043096] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:53.043108] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:53.043133] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:53.043142] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:53.043454] INFO delete_row (ob_table_access_helper.h:312) [21575][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061873042700) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061873042700) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:53.043681] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:53.043895] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C443A-0-0] [lt=37] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=415, avg_log_submit_to_slide_time=713) [2024-10-16 14:57:53.046568] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061873045010000, start_id=1729061873045010000, end_id=1729061873045010001) [2024-10-16 14:57:53.057680] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061873056699000, start_id=1729061873056699000, end_id=1729061873056699001) [2024-10-16 14:57:53.057706] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=17] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=684, get_gts_with_stc_cnt=29, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=29, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:53.057715] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061873057674], gts:1729061873056699000, latest_srr:[mts=1729061873057674]}) [2024-10-16 14:57:53.057731] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:53.057751] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=536, get_gts_with_stc_cnt=11, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:53.073714] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:53.073736] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:53.073757] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.073770] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] LogLoopThread round_cost_time(us)(round_cost_time=15) [2024-10-16 14:57:53.078242] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [21603][T1_MediumLoop][T1][YB4A0A000407-00061E99739F3877-0-0] [lt=3] MediumCheckTask(cost_ts=7) [2024-10-16 14:57:53.083380] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D33-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178570, log_task={header:{begin_lsn:{lsn:14538501998}, end_lsn:{lsn:14538502120}, log_id:115178570, min_scn:{val:1729061872652685004, v:0}, max_scn:{val:1729061872652685004, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538501876}, prev_proposal_id:23, committed_end_lsn:{lsn:14538501998}, data_checksum:1715074492, accum_checksum:3844462549, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061872983051, freeze_ts:1729061872983052, submit_ts:1729061872983057, flushed_ts:1729061872983328, gen_to_freeze cost time:1, gen_to_submit cost time:6, submit_to_flush cost time:271}, fs_cb_begin_ts=1729061873083373, log_life_time=100322) [2024-10-16 14:57:53.083469] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.090861] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:53.091452] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:53.091504] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=17] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061870205428000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:53.095357] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85085-0-0] [lt=26][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497530, log_task={header:{begin_lsn:{lsn:207589561327}, end_lsn:{lsn:207589561449}, log_id:218497530, min_scn:{val:1729061872650827003, v:0}, max_scn:{val:1729061872650827003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589561205}, prev_proposal_id:81, committed_end_lsn:{lsn:207589561327}, data_checksum:940009244, accum_checksum:1298071948, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061872995311, freeze_ts:1729061872995311, submit_ts:1729061872995318, flushed_ts:1729061872995641, gen_to_freeze cost time:0, gen_to_submit cost time:7, submit_to_flush cost time:323}, fs_cb_begin_ts=1729061873095350, log_life_time=100039) [2024-10-16 14:57:53.119438] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22235][T1002_TenantWea][T1001][YB4A0A000407-00061E9B3B0B77B1-0-0] [lt=20] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.132345581054687500e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=306, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.971678898181612340e+03) [2024-10-16 14:57:53.119621] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061873119090000, start_id=1729061873119090000, end_id=1729061873119090001) [2024-10-16 14:57:53.124240] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=12] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97898434560) [2024-10-16 14:57:53.127754] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:53.127773] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:53.128516] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:53.128534] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:53.128837] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=223, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:53.128859] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=37, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:53.129014] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FFC3-0-0] [lt=31] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=311, avg_log_submit_to_slide_time=572) [2024-10-16 14:57:53.137194] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3690, mem_used=8318976) [2024-10-16 14:57:53.145137] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB APPEND COST TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=23, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:57:53.145155] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=18] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=946, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:57:53.145161] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=6] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:57:53.145166] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB EXECUTE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=-524, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:57:53.145171] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB TOTAL TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=444, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:57:53.149141] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061873149139, bytes: 0, bw: 0.000000 MB/s, add_ts: 1006351, add_bytes: 0 [2024-10-16 14:57:53.149366] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:53.149401] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:53.153200] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA41-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061873152656000, start_id=1729061873152656000, end_id=1729061873152656001) [2024-10-16 14:57:53.153379] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA40-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061873152656000, start_id=1729061873152656000, end_id=1729061873152656001) [2024-10-16 14:57:53.153492] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA3D-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061873152656000, start_id=1729061873152656000, end_id=1729061873152656001) [2024-10-16 14:57:53.154360] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15E4-0-0] [lt=13] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:53.155138] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15DE-0-0] [lt=14] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:53.155690] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=55] get number(ret=0, service_type_=0, range=1, base_id=1729061873153759000, start_id=1729061873153759000, end_id=1729061873153759001) [2024-10-16 14:57:53.155757] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:53.155771] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:53.157713] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95A0-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11536) [2024-10-16 14:57:53.159261] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061873158154000, start_id=1729061873158154000, end_id=1729061873158154001) [2024-10-16 14:57:53.160472] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15E0-0-0] [lt=14] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:53.160644] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA3F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061873160610000, start_id=1729061873160610000, end_id=1729061873160610001) [2024-10-16 14:57:53.160765] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA3E-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061873160610000, start_id=1729061873160610000, end_id=1729061873160610001) [2024-10-16 14:57:53.170581] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1817) [2024-10-16 14:57:53.173771] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.180937] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:53.183533] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.195687] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5037-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:53.195712] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5037-0-0] [lt=23] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:53.195935] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:53.210716] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB0-0-0] [lt=10] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061873178389, keepalive_ts:-1}) [2024-10-16 14:57:53.210737] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB0-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061873178389, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:53.210759] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB1-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061873178444, keepalive_ts:-1}) [2024-10-16 14:57:53.210763] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB1-0-0] [lt=4] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061873178444, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:53.218610] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061873217897000, start_id=1729061873217897000, end_id=1729061873217897001) [2024-10-16 14:57:53.228891] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:53.237575] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:53.237638] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88B-0-0] [lt=22] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:53.237660] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88B-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061872650827004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061872650827004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:53.241937] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88B-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061873241640000, start_id=1729061873241640000, end_id=1729061873241640001) [2024-10-16 14:57:53.242021] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88B-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=11536, avg_group_log_size=640) [2024-10-16 14:57:53.242032] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88B-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11536) [2024-10-16 14:57:53.242318] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=225, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:53.242333] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11536) [2024-10-16 14:57:53.242347] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=131, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:53.242387] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=6, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:53.242620] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D58FD-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:53.242635] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D58FD-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=775, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:53.242640] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D58FD-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=461, avg_log_submit_to_slide_time=742) [2024-10-16 14:57:53.242710] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=5, total_batch_msg_count=0) [2024-10-16 14:57:53.246741] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061873245857000, start_id=1729061873245857000, end_id=1729061873245857001) [2024-10-16 14:57:53.252554] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DAB-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510341120947, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410617, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:53.270536] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7853-0-0] [lt=32] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1817, avg_group_log_size=181) [2024-10-16 14:57:53.273833] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.282850] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=271, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:53.282881] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=73, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:53.282931] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:53.283056] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FFC8-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:53.283124] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] succ to submit log task to replay service(to_submit_lsn={lsn:14538502420}, to_submit_scn={val:1729061873152656002, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061873283075, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538502486}, next_to_submit_scn_:{val:1729061873152656003, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538502364}, end_lsn:{lsn:14538502486}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061873152656002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061873152656002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:173190179, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061873152656002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650330, miss_cnt:32, cache_read_size:8050431571, hit_ratio:"9.999994892288092130e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:53.283597] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.294346] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C10F5-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061873294294000, start_id=1729061873294294000, end_id=1729061873294294001) [2024-10-16 14:57:53.295611] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85089-0-0] [lt=22] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:53.295639] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85089-0-0] [lt=20] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:53.295652] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85089-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:53.295669] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85089-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497532, log_task={header:{begin_lsn:{lsn:207589561571}, end_lsn:{lsn:207589561693}, log_id:218497532, min_scn:{val:1729061873160610000, v:0}, max_scn:{val:1729061873160610000, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589561449}, prev_proposal_id:81, committed_end_lsn:{lsn:207589561571}, data_checksum:940009244, accum_checksum:2308966682, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061873195383, freeze_ts:1729061873195383, submit_ts:1729061873195388, flushed_ts:1729061873195969, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:581}, fs_cb_begin_ts=1729061873295662, log_life_time=100279) [2024-10-16 14:57:53.305099] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD103C-0-0] [lt=8] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:53.319727] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061873319080000, start_id=1729061873319080000, end_id=1729061873319080001) [2024-10-16 14:57:53.328310] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=156, mem_used=2079744) [2024-10-16 14:57:53.328997] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:53.332513] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=10] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:53.343608] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=384, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:53.343626] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=87, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:53.343648] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:53.343704] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=54] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:53.349412] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:53.355626] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061873354194000, start_id=1729061873354194000, end_id=1729061873354194001) [2024-10-16 14:57:53.357908] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EBA-0-0] [lt=13][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173266, log_task={header:{begin_lsn:{lsn:14352388609}, end_lsn:{lsn:14352388731}, log_id:115173266, min_scn:{val:1729061873152656001, v:0}, max_scn:{val:1729061873152656001, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352388487}, prev_proposal_id:17, committed_end_lsn:{lsn:14352388609}, data_checksum:940009244, accum_checksum:3077760318, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061873254954, freeze_ts:1729061873254955, submit_ts:1729061873254965, flushed_ts:1729061873255272, gen_to_freeze cost time:1, gen_to_submit cost time:11, submit_to_flush cost time:307}, fs_cb_begin_ts=1729061873357901, log_life_time=102947) [2024-10-16 14:57:53.359398] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061873358784000, start_id=1729061873358784000, end_id=1729061873358784001) [2024-10-16 14:57:53.372769] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB3-0-0] [lt=7] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:53.372784] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB3-0-0] [lt=11] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=16) [2024-10-16 14:57:53.373891] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.383404] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D39-0-0] [lt=32] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:53.383632] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.383910] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=17] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:53.385980] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601450, offset=9638954}, end_cursor:ObLogCursor{file_id=192, log_id=47601451, offset=9639238}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7372050, len:3030, log_data_len:284, seq:47601450, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:53.387822] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4AF-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061873386964000, start_id=1729061873386964000, end_id=1729061873386964001) [2024-10-16 14:57:53.388898] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601451, offset=9639238}, end_cursor:ObLogCursor{file_id=192, log_id=47601452, offset=9639512}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73a6050, len:2746, log_data_len:274, seq:47601451, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:53.391118] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601452, offset=9639512}, end_cursor:ObLogCursor{file_id=192, log_id=47601453, offset=9639786}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72e6050, len:2472, log_data_len:274, seq:47601452, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:53.391307] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601453, offset=9639786}, end_cursor:ObLogCursor{file_id=192, log_id=47601454, offset=9640060}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db727a050, len:2198, log_data_len:274, seq:47601453, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:53.391360] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561394, miss_cnt:1, cache_read_size:152590694061, hit_ratio:"9.999999928346947620e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:53.391374] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598980, miss_cnt:9, cache_read_size:149777926256, hit_ratio:"9.999999345925426608e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:53.395113] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:57:53.395717] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=198, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:53.395738] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=358, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:53.395779] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:53.397637] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=6] query dest_info_ succ(dest_info=[], archive_dest_changed=false) [2024-10-16 14:57:53.397654] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=15] finish to resize log external storage handler(current_ts=1729061873397653, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:53.408940] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3826-0-0] [lt=18] MediumCheckTask(cost_ts=7) [2024-10-16 14:57:53.418501] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061873417888000, start_id=1729061873417888000, end_id=1729061873417888001) [2024-10-16 14:57:53.418947] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:53.437865] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=32] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:53.443342] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:53.443364] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:53.443803] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:53.447176] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061873446436000, start_id=1729061873446436000, end_id=1729061873446436001) [2024-10-16 14:57:53.464841] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257798, offset=39043192}, end_cursor:ObLogCursor{file_id=197, log_id=49257799, offset=39043475}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e0c050, len:3976, log_data_len:283, seq:49257798, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:53.466977] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB37A-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061873466795000, start_id=1729061873466795000, end_id=1729061873466795001) [2024-10-16 14:57:53.468087] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257799, offset=39043475}, end_cursor:ObLogCursor{file_id=197, log_id=49257800, offset=39043757}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72de050, len:3693, log_data_len:282, seq:49257799, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:53.470118] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257800, offset=39043757}, end_cursor:ObLogCursor{file_id=197, log_id=49257801, offset=39044039}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dac050, len:3411, log_data_len:282, seq:49257800, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:53.470259] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257801, offset=39044039}, end_cursor:ObLogCursor{file_id=197, log_id=49257802, offset=39044321}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd216c050, len:3129, log_data_len:282, seq:49257801, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:53.470302] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657070, miss_cnt:17, cache_read_size:8050179187, hit_ratio:"9.999997286819286568e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:53.470335] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650330, miss_cnt:32, cache_read_size:8050431571, hit_ratio:"9.999994892288092130e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:53.470535] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7857-0-0] [lt=21] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:53.470763] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=193, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:53.470778] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:53.470991] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4265-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:53.471000] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4265-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9238, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:53.471001] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1817, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:53.471030] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:53.471043] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:53.473950] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.482049] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8B-0-0] [lt=9] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:53.482065] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8B-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:53.482589] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D3B-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:53.482609] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D3B-0-0] [lt=19] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40036, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:53.483062] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9210-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=325, avg_log_submit_to_slide_time=40033) [2024-10-16 14:57:53.483071] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:53.483688] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.490525] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:53.490542] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:53.490555] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70802321, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:53.491611] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:53.491620] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:53.491630] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:53.491635] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:53.491638] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:53.491629] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A28-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:53.491641] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:53.491644] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:53.491647] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:53.491644] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A28-0-0] [lt=12] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:53.491649] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:53.491652] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:53.491655] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:53.495980] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5497-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:53.495993] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5497-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40358, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:53.506174] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] Garbage Collector is running(seq_=656496, gc_interval=10000000) [2024-10-16 14:57:53.506205] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=11] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:53.506213] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:53.506498] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:53.507413] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:53.507422] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=1236) [2024-10-16 14:57:53.507431] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:53.508228] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:53.508874] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=10] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:53.509475] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=18] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:53.510070] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:53.510081] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:53.510089] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:53.510095] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:53.510100] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:53.510105] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=25) [2024-10-16 14:57:53.510110] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:53.510161] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:53.510907] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:53.510927] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=20] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:53.510945] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:53.511013] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061885590307000, latest_log_ts:{val:1729061868948338001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:53.511057] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=41] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:53.511075] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:53.511082] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061868948338001, v:0}) [2024-10-16 14:57:53.511085] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:53.511088] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:53.511095] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:53.511104] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:53.511120] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=15] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:53.511124] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:53.519874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061873519354000, start_id=1729061873519354000, end_id=1729061873519354001) [2024-10-16 14:57:53.520765] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061873160610001, v:0}, new_max_version={val:1729061873160610001, v:0}, last_min_version={val:1729061872557205004, v:0}, last_max_version={val:1729061872557205004, v:0}, total_time_us=1784, delta_us=360153) [2024-10-16 14:57:53.520783] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061873160610001, v:0}, min_version={val:1729061873160610001, v:0}, max_version={val:1729061873160610001, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:57:53.528837] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:53.528855] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:53.529148] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:53.529168] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:53.543400] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:53.543443] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:53.543690] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:53.544154] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4448-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:53.544163] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4448-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=845, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:53.544998] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E998FCD75CB-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061873544759000, start_id=1729061873544759000, end_id=1729061873544759001) [2024-10-16 14:57:53.545363] INFO [STORAGE] try_shrink (ob_compaction_memory_pool.cpp:522) [21758][T1001_MergeMemP][T1001][Y0-0000000000000000-0-0] [lt=10] Compaction Mem Pool current stat: (ret=0, max_block_num=16, total_block_num=8, Chunk list size: =1) [2024-10-16 14:57:53.546042] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=20] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:53.546064] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:14538502730}, scn={val:1729061873152656005, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061873483056, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538502730}, next_to_submit_scn_:{val:1729061873152656005, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538502608}, end_lsn:{lsn:14538502730}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061873152656004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061873152656004, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:3795567739, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061873152656004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650332, miss_cnt:32, cache_read_size:8050431815, hit_ratio:"9.999994892288255333e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:53.546113] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=45] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:53.549490] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:53.555823] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061873554214000, start_id=1729061873554214000, end_id=1729061873554214001) [2024-10-16 14:57:53.559553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061873558226000, start_id=1729061873558226000, end_id=1729061873558226001) [2024-10-16 14:57:53.559569] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=13] ts response task statistics(alloc_count=261364910, free_count=130682454) [2024-10-16 14:57:53.559629] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=5] submit log success(service_type_=0, last_id=1729061890098662000, limited_id=1729061890098662000) [2024-10-16 14:57:53.559634] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=5] submit log with lock success(service_type_=0, last_id=1729061890098662000, limited_id=1729061890098662000) [2024-10-16 14:57:53.559636] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061873558226000, start_id=1729061873558226000, end_id=1729061873558226001) [2024-10-16 14:57:53.560901] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=271, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:53.560916] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=37, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:53.560913] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] update id meta(ret=0, service_type=0, limited_id=1729061890098662000, latest_scn={val:1729061873558226001, v:0}, *this={tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871279755000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890098662000, latest_log_ts:{val:1729061873558226001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:53.560949] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=35] handle submit callback(service_type_=0, success=true, used_ts=1320, log_ts={val:1729061873558226001, v:0}, limited_id=1729061890098662000) [2024-10-16 14:57:53.560956] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:53.560961] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:53.560963] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797D18-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10643) [2024-10-16 14:57:53.560985] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:53.560993] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=592, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:53.560998] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=331, avg_log_submit_to_slide_time=558) [2024-10-16 14:57:53.561016] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:53.561356] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=227, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:53.561370] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=55, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:53.561396] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:53.568941] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.569204] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.569264] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.569622] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.574018] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.577520] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=24, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:53.577544] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3666, avg_group_log_size=333) [2024-10-16 14:57:53.577556] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:53.577854] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=260, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:53.577872] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=104, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:53.577888] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:53.578201] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C64-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=383, avg_log_submit_to_slide_time=636) [2024-10-16 14:57:53.582024] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:53.582027] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26148][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB19CC6ED-0-0] [lt=28] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:53.583755] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.596736] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107522][][T1002][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1002, ret=0, thread_count_=191) [2024-10-16 14:57:53.596759] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107522][][T1002][Y0-0000000000000000-0-0] [lt=21] Init thread local success [2024-10-16 14:57:53.596786] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107522][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8508F-0-0] [lt=5] init point thread id with(&point=0x55e3435caf00, idx_=1026, point=[thread id=107522, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107522) [2024-10-16 14:57:53.597042] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=15] worker thread created(tenant_->id()=1002, group_id_=1, token=1) [2024-10-16 14:57:53.597103] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1001}, {id:1}]) [2024-10-16 14:57:53.597180] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:53.597211] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=30][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:53.609283] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D9-0-0] [lt=42] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061873576991, keepalive_ts:-1}) [2024-10-16 14:57:53.609306] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47D9-0-0] [lt=21] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061873576991, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:53.610351] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5577, mem_used=10398720) [2024-10-16 14:57:53.618499] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061873617607000, start_id=1729061873617607000, end_id=1729061873617607001) [2024-10-16 14:57:53.621813] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:53.628843] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:53.628858] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:53.638170] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:53.642374] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:33216819320}, base_lsn={lsn:33149755392}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:53.642447] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=72] get_replay_process success(id={id:1}, replayed_log_size=67063928, unreplayed_log_size=0) [2024-10-16 14:57:53.642524] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=61] dump tenant replay process(tenant_id=1, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=63, last_replayed_log_size(MB)=63, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:53.646979] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061873646821000, start_id=1729061873646821000, end_id=1729061873646821001) [2024-10-16 14:57:53.647050] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10644, avg_group_log_size=626) [2024-10-16 14:57:53.647061] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10643) [2024-10-16 14:57:53.647506] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10644) [2024-10-16 14:57:53.653491] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA45-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061873653297000, start_id=1729061873653297000, end_id=1729061873653297001) [2024-10-16 14:57:53.653493] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA42-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061873653297000, start_id=1729061873653297000, end_id=1729061873653297001) [2024-10-16 14:57:53.653575] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA44-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061873653297000, start_id=1729061873653297000, end_id=1729061873653297001) [2024-10-16 14:57:53.653722] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA46-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061873653297000, start_id=1729061873653297000, end_id=1729061873653297001) [2024-10-16 14:57:53.653886] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA43-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061873653297000, start_id=1729061873653297000, end_id=1729061873653297001) [2024-10-16 14:57:53.665727] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:53.665744] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=15] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:57:53.665751] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:57:53.665755] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:53.667888] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=108, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}, {obj_id:616254, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616255, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616256, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:616257, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}]) [2024-10-16 14:57:53.668054] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=163] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:53.668293] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CF-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:53.668312] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9CF-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061873653297000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061873653297000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:53.672053] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9CF-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061873671263000, start_id=1729061873671263000, end_id=1729061873671263001) [2024-10-16 14:57:53.672179] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:53.672733] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:53.674036] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.683832] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.688812] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=9] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:53.689589] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061873688223000, start_id=1729061873688223000, end_id=1729061873688223001) [2024-10-16 14:57:53.696573] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D8-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061873687810, keepalive_ts:-1}) [2024-10-16 14:57:53.696588] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D8-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061873687810, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:53.699686] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:53.718856] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=10] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:53.720156] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061873718909000, start_id=1729061873718909000, end_id=1729061873718909001) [2024-10-16 14:57:53.720849] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5C-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:53.720870] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5C-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061873653297001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061873653297001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:53.723167] INFO [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:262) [22249][T1002_ArcSender][T1002][YB4A0A000407-00061E996F9960C1-0-0] [lt=10] ObArchiveSender is running(thread_index=1) [2024-10-16 14:57:53.724738] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C5C-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061873724536000, start_id=1729061873724536000, end_id=1729061873724536001) [2024-10-16 14:57:53.725260] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22859][T1004_L0_G2][T1004][YB4A0A000409-00061E9A39C4D8AC-0-0] [lt=59] get_min_unreplayed_log_info(lsn={lsn:14352389210}, scn={val:1729061873653297001, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061873655680, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352389210}, next_to_submit_scn_:{val:1729061873653297001, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352389088}, end_lsn:{lsn:14352389210}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061873653297000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061873653297000, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:1959171417, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061873653297000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657072, miss_cnt:17, cache_read_size:8050179431, hit_ratio:"9.999997286819373166e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:53.727212] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:53.728849] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:53.728862] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:57:53.728865] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:57:53.728880] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=2] get_max_applied_scn(scn={val:1729061873558226000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493487506}, palf_committed_end_scn_:{val:1729061873688223000, v:0}, last_check_scn_:{val:1729061873688223000, v:0}, max_applied_cb_scn_:{val:1729061873558226000, v:0}}) [2024-10-16 14:57:53.728897] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:53.729590] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=39] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061880721265000, latest_log_ts:{val:1729061864144741001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:53.729716] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=57] get_max_applied_scn(scn={val:1729061873558226000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645433116}, palf_committed_end_scn_:{val:1729061873558226000, v:0}, last_check_scn_:{val:1729061873558226000, v:0}, max_applied_cb_scn_:{val:1729061873558226000, v:0}}) [2024-10-16 14:57:53.729732] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:53.730377] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=41] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:53.731093] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=40] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:53.731726] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=36] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:53.732043] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:53.732061] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=700, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:53.744184] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C444E-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=1068, avg_log_submit_to_slide_time=1351) [2024-10-16 14:57:53.745637] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=28] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:53.749573] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:53.755880] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061873755150000, start_id=1729061873755150000, end_id=1729061873755150001) [2024-10-16 14:57:53.756140] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EC2-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:53.756248] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EC2-0-0] [lt=100] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:53.756915] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9215-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:53.756931] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9215-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=41379, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:53.756930] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1455, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:53.756938] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9215-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=294, avg_log_submit_to_slide_time=41364) [2024-10-16 14:57:53.758861] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:53.758890] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=17] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:53.758906] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:53.759754] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15E8-0-0] [lt=17] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:53.759815] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061873759776000, start_id=1729061873759776000, end_id=1729061873759776001) [2024-10-16 14:57:53.772375] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:53.774093] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.777771] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3543) [2024-10-16 14:57:53.777814] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3543) [2024-10-16 14:57:53.778107] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C66-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:53.778124] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C66-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=945, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:53.782359] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:53.783888] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.788226] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=32] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.788243] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.788525] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.788535] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.788543] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.788551] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.789108] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.789361] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.789360] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.789588] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.789609] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:53.796181] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1464, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:53.818727] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061873817789000, start_id=1729061873817789000, end_id=1729061873817789001) [2024-10-16 14:57:53.829068] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:53.829087] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=16] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:53.829123] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=52544319, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:53.829585] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD37B-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=420, avg_log_submit_to_slide_time=684) [2024-10-16 14:57:53.831314] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1003}, {id:1}]) [2024-10-16 14:57:53.838483] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:53.843931] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=415, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:53.843952] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=712, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:53.843983] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:53.847041] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061873846123000, start_id=1729061873846123000, end_id=1729061873846123001) [2024-10-16 14:57:53.853399] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=42] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:53.854000] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:53.854048] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061864434291000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:53.854098] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=48] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061864434291000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1, freeze_info:{latest_snapshot_gc_scn:{val:1729061864434291000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}}) [2024-10-16 14:57:53.860646] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5604, mem_used=10398720) [2024-10-16 14:57:53.866406] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=34] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}) [2024-10-16 14:57:53.874156] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.877642] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=11] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061869867164, last_first_register_time_us=1722494157071428, register_time_us=1729061869867164, curr_time_us=1729061873877641) [2024-10-16 14:57:53.877684] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=26] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061873877670, last_submit_register_req_time_us=1729061873877670, register_parent_reason=3) [2024-10-16 14:57:53.877694] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:53.883342] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:53.883358] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:53.883951] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.891783] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1323) [2024-10-16 14:57:53.892137] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:14352389510}, scn={val:1729061873653297003, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:53.892166] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:13, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061873892093, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352389566}, next_to_submit_scn_:{val:1729061873653297004, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352389454}, end_lsn:{lsn:14352389566}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061873653297003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061873653297003, v:0}, data_checksum:1407533925, flag:1}}, init_mode_version:0, accumulate_checksum:2550335518, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061873653297003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657075, miss_cnt:17, cache_read_size:8050179787, hit_ratio:"9.999997286819503062e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14352389510}, scn_:{val:1729061873653297003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061873859223, first_handle_ts_:1729061873892165, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f77c1c6a0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:53.892291] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=47] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060071843583000) [2024-10-16 14:57:53.892300] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14352389510}, scn_:{val:1729061873653297003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061873859223, first_handle_ts_:1729061873892276, replay_cost_:23, retry_cost_:-1, read_log_buf_:0x2b0f77c1c6b0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061873892093, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352389566}, next_to_submit_scn_:{val:1729061873653297004, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352389454}, end_lsn:{lsn:14352389566}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061873653297003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061873653297003, v:0}, data_checksum:1407533925, flag:1}}, init_mode_version:0, accumulate_checksum:2550335518, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061873653297003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657075, miss_cnt:17, cache_read_size:8050179787, hit_ratio:"9.999997286819503062e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}, replay_queue_idx=23, ret=0) [2024-10-16 14:57:53.896954] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=18] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:53.901430] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=39] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:53.902048] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:53.902104] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061871843583000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:53.902145] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=40] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061871843583000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1004, freeze_info:{latest_snapshot_gc_scn:{val:1729061871843583000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}}) [2024-10-16 14:57:53.902286] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:53.920217] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061873919683000, start_id=1729061873919683000, end_id=1729061873919683001) [2024-10-16 14:57:53.924197] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:53.928740] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D79-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:53.928766] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D79-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061873653297003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061873653297003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:53.928992] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:53.929004] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:53.929147] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:53.930596] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D79-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061873930561000, start_id=1729061873930561000, end_id=1729061873930561001) [2024-10-16 14:57:53.932741] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D79-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061873932648000, start_id=1729061873932648000, end_id=1729061873932648001) [2024-10-16 14:57:53.932800] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D79-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:53.932813] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D79-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:53.944531] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=9] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3642, mem_used=8318976) [2024-10-16 14:57:53.949652] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:53.956738] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061873955913000, start_id=1729061873955913000, end_id=1729061873955913001) [2024-10-16 14:57:53.959827] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061873959086000, start_id=1729061873959086000, end_id=1729061873959086001) [2024-10-16 14:57:53.974190] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.977721] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3543) [2024-10-16 14:57:53.983202] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:57:53.984005] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:53.993066] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:53.993092] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=214) [2024-10-16 14:57:54.018034] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] no need to update min/max version(new_version={val:1729061873617607000, v:0}, min_version={val:1729061873617607000, v:0}, max_version={val:1729061873617607000, v:0}) [2024-10-16 14:57:54.029122] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=20] PNIO [ratelimit] time: 1729061874029120, bytes: 4446819240316, bw: 0.359184 MB/s, add_ts: 1000645, add_bytes: 376875 [2024-10-16 14:57:54.029441] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=234, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:54.029467] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=257, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:54.029490] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:54.029743] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FFDF-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:54.029754] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE3FFDF-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=800, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:54.032531] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:54.033273] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.033303] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=29] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.033326] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:54.033467] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871279755000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890098662000, latest_log_ts:{val:1729061873558226001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:54.033518] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=48] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871279755000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890098662000, latest_log_ts:{val:1729061873558226001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:54.033552] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:54.033562] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061873558226001, v:0}) [2024-10-16 14:57:54.033566] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:54.033574] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:54.033587] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:54.033597] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:54.034346] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.034364] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=17] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.034375] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:54.034420] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871279755000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:54.034452] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871279755000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:54.034479] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:54.034489] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:54.034495] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:54.034502] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:54.035211] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.035233] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=23] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.035244] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:54.035284] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871279755000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:54.035316] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871279755000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:54.035346] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:54.035352] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:54.035358] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:54.035363] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:54.036069] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.036083] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.036092] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:54.036128] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871279755000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:54.036157] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871279755000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:54.036180] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:54.036189] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:54.036194] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:54.036199] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:54.036203] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:54.038766] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:54.043733] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:54.043747] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:54.044022] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1226) [2024-10-16 14:57:54.044263] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107522][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4457-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=427, avg_log_submit_to_slide_time=739) [2024-10-16 14:57:54.044298] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4458-0-0] [lt=28] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:54.044314] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4458-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1259, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:54.047240] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061874047006000, start_id=1729061874047006000, end_id=1729061874047006001) [2024-10-16 14:57:54.047514] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=11] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:54.055417] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EC8-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=12, avg_log_batch_cnt=0, total_group_log_size=1445, avg_group_log_size=120) [2024-10-16 14:57:54.055441] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EC8-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173274, log_task={header:{begin_lsn:{lsn:14352389566}, end_lsn:{lsn:14352389688}, log_id:115173274, min_scn:{val:1729061873653297004, v:0}, max_scn:{val:1729061873653297004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352389454}, prev_proposal_id:17, committed_end_lsn:{lsn:14352389566}, data_checksum:940009244, accum_checksum:1101227055, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061873955341, freeze_ts:1729061873955341, submit_ts:1729061873955345, flushed_ts:1729061873957187, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1842}, fs_cb_begin_ts=1729061874055435, log_life_time=100094) [2024-10-16 14:57:54.059993] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061874059797000, start_id=1729061874059797000, end_id=1729061874059797001) [2024-10-16 14:57:54.060020] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=18] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:54.060038] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061874060034], gts:1729061874059797000, latest_srr:[mts=1729061874060034]}) [2024-10-16 14:57:54.060224] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21292][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=14] submit log success(service_type_=0, last_id=1729061890721265000, limited_id=1729061890721265000) [2024-10-16 14:57:54.060230] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21292][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=6] submit log with lock success(service_type_=0, last_id=1729061890721265000, limited_id=1729061890721265000) [2024-10-16 14:57:54.060757] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=8] update id meta(ret=0, service_type=0, limited_id=1729061890721265000, latest_scn={val:1729061874059797001, v:0}, *this={tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890721265000, latest_log_ts:{val:1729061874059797001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:54.060790] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=32] handle submit callback(service_type_=0, success=true, used_ts=567, log_ts={val:1729061874059797001, v:0}, limited_id=1729061890721265000) [2024-10-16 14:57:54.060796] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:54.060801] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:54.070984] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7863-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:54.071220] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:54.074254] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.076115] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] Garbage Collector is running(seq_=656493, gc_interval=10000000) [2024-10-16 14:57:54.076143] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=11] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:54.076153] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:54.076539] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:57:54.076799] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=12] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:57:54.076809] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=7] gc_check_member_list_ cost time(ret=0, time_us=684) [2024-10-16 14:57:54.076817] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=4] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:54.077686] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:54.078326] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=11] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:54.078932] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:54.079488] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:54.079499] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:54.079507] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:54.079513] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:54.079518] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:54.079523] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=25) [2024-10-16 14:57:54.079527] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:54.079531] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:133) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-10-16 14:57:54.079558] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:140) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-10-16 14:57:54.082859] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D47-0-0] [lt=28] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:54.083063] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:54.084070] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.118924] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061874117940000, start_id=1729061874117940000, end_id=1729061874117940001) [2024-10-16 14:57:54.120138] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22235][T1002_TenantWea][T1001][YB4A0A000407-00061E9B3B0B77C0-0-0] [lt=18] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.132496929168701172e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=306, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.971485825093569474e+03) [2024-10-16 14:57:54.120267] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061874120073000, start_id=1729061874120073000, end_id=1729061874120073001) [2024-10-16 14:57:54.127671] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:54.127688] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:54.129301] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=227, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:54.129322] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=150, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:54.135193] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.63, rt: 151.50; ) [2024-10-16 14:57:54.135214] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:54.135256] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.63, rt: 138.50; ) [2024-10-16 14:57:54.135264] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:54.135275] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:54.137293] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3741, mem_used=8318976) [2024-10-16 14:57:54.143719] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1348, avg_group_log_size=122) [2024-10-16 14:57:54.143732] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1348) [2024-10-16 14:57:54.143971] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=31] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=24624930, this={dio_aligned_buf:{buf_write_offset:3996, buf_padding_size:100, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:93867026, truncate_used_ts:39944945}, log_block_size:67108864, dir_fd:173, io_fd:196}, aligned_buf_len=4096, aligned_block_offset=24621056, buf_len=122, total_write_size_=12050, total_write_size_after_dio_=413696, ob_pwrite_used_ts=47033, count=99, each_pwrite_cost=475) [2024-10-16 14:57:54.149460] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:54.151634] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:54.153870] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA47-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061874153492000, start_id=1729061874153492000, end_id=1729061874153492001) [2024-10-16 14:57:54.153970] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA4A-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061874153492000, start_id=1729061874153492000, end_id=1729061874153492001) [2024-10-16 14:57:54.154031] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA49-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061874153492000, start_id=1729061874153492000, end_id=1729061874153492001) [2024-10-16 14:57:54.154838] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA48-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061874154550000, start_id=1729061874154550000, end_id=1729061874154550001) [2024-10-16 14:57:54.154932] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA4B-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061874154550000, start_id=1729061874154550000, end_id=1729061874154550001) [2024-10-16 14:57:54.155915] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=10] current monitor number(seq_=-1) [2024-10-16 14:57:54.156753] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=16] PNIO [ratelimit] time: 1729061874156751, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007612, add_bytes: 0 [2024-10-16 14:57:54.157040] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061874156666000, start_id=1729061874156666000, end_id=1729061874156666001) [2024-10-16 14:57:54.157105] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:54.157119] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:54.160030] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061874159868000, start_id=1729061874159868000, end_id=1729061874159868001) [2024-10-16 14:57:54.171182] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:54.174316] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.180998] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [22325][T1003_COMMONLSS][T1003][YB4A0A000407-00061E9B3A19CB20-0-0] [lt=17][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:54.184095] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.196510] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5038-0-0] [lt=8] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:54.196530] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5038-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:54.196767] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:54.209338] INFO [SHARE.LOCATION] renew_all_ls_locations_by_rpc (ob_ls_location_service.cpp:801) [21248][AutoLSLocRpc][T0][YB4A0A000407-00061E9B390A2264-0-0] [lt=45] [LS_LOCATION] Get ls leaders by RPC(ret=0, ret="OB_SUCCESS", dests=["10.0.4.7:2890", "10.0.4.8:2890", "10.0.4.9:2890"], leaders=[{key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}}, {key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}}, {key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}}, {key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}}, {key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}}, {key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}}, {key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}}, {key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}}, {key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}]) [2024-10-16 14:57:54.229198] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=30, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:54.229245] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=36] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:54.229610] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1764) [2024-10-16 14:57:54.229857] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD389-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=42, avg_log_submit_to_flush_time=491, avg_log_submit_to_slide_time=750) [2024-10-16 14:57:54.239117] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:54.247710] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061874247412000, start_id=1729061874247412000, end_id=1729061874247412001) [2024-10-16 14:57:54.248541] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:54.249256] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.249283] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=26] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.249306] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=18] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:54.249437] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890721265000, latest_log_ts:{val:1729061874059797001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:54.249491] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=52] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:54.249512] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:54.249522] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061874059797001, v:0}) [2024-10-16 14:57:54.249528] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.249536] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:54.249547] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:54.249555] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:54.249751] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=9][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:54.250271] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.250287] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.250301] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:54.250367] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:54.250402] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=34] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:54.250428] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:54.250435] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:54.250445] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:54.250453] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:54.251165] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.251182] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.251192] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:54.251263] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:54.251296] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=32] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:54.251322] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:54.251328] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:54.251336] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:54.251344] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:54.252110] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.252143] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=32] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:54.252166] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=20] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:54.252260] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:54.252328] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=65] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061871055408003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:54.252381] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=49] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:54.252395] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:54.252417] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:54.252426] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:54.252433] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:54.253566] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21657][T1_L0_G0][T1][YB4A0A000408-00061E9B451E1E0E-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061874252678000, start_id=1729061874252678000, end_id=1729061874252678001) [2024-10-16 14:57:54.255995] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DAD-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510342124389, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410618, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:54.256521] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88C-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:54.256541] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88C-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061873653297004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061873653297004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:54.258317] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA88C-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061874258093000, start_id=1729061874258093000, end_id=1729061874258093001) [2024-10-16 14:57:54.260742] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88C-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061874260176000, start_id=1729061874260176000, end_id=1729061874260176001) [2024-10-16 14:57:54.260813] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88C-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9694, avg_group_log_size=570) [2024-10-16 14:57:54.260824] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88C-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9694) [2024-10-16 14:57:54.261152] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=223, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:54.261166] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9694) [2024-10-16 14:57:54.261180] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=47, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:54.261193] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10286) [2024-10-16 14:57:54.261205] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:54.261312] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D590E-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:54.261319] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D590E-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=657, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:54.261323] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D590E-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=302, avg_log_submit_to_slide_time=624) [2024-10-16 14:57:54.261372] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:54.271021] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7867-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:54.274376] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.283373] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=193, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:54.283392] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:54.283423] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:54.284270] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.287404] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1E03-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061874278088000, start_id=1729061874278088000, end_id=1729061874278088001) [2024-10-16 14:57:54.296056] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8509D-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:54.296078] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8509D-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:54.296086] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8509D-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:54.296099] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8509D-0-0] [lt=5][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497542, log_task={header:{begin_lsn:{lsn:207589562791}, end_lsn:{lsn:207589562913}, log_id:218497542, min_scn:{val:1729061874153492000, v:0}, max_scn:{val:1729061874153492000, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589562669}, prev_proposal_id:81, committed_end_lsn:{lsn:207589562791}, data_checksum:940009244, accum_checksum:376396481, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061874195976, freeze_ts:1729061874195976, submit_ts:1729061874195983, flushed_ts:1729061874196793, gen_to_freeze cost time:0, gen_to_submit cost time:7, submit_to_flush cost time:810}, fs_cb_begin_ts=1729061874296093, log_life_time=100117) [2024-10-16 14:57:54.302277] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1107-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061874302261000, start_id=1729061874302261000, end_id=1729061874302261001) [2024-10-16 14:57:54.308807] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=8] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:54.309491] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985DB-0-0] [lt=15] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=187) [2024-10-16 14:57:54.309596] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061874308653000, start_id=1729061874308653000, end_id=1729061874308653001) [2024-10-16 14:57:54.312455] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:54.312480] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=17] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:54.312501] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373676960, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:54.316982] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21658][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE897-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510342185374, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410619, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:54.319029] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061874318374000, start_id=1729061874318374000, end_id=1729061874318374001) [2024-10-16 14:57:54.320516] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061874320473000, start_id=1729061874320473000, end_id=1729061874320473001) [2024-10-16 14:57:54.322436] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=30] BlockGCTimerTask success(ret=0, cost_time_us=7, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):114, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:54.328437] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=214, mem_used=2079744) [2024-10-16 14:57:54.329340] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B25-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061874328200000, start_id=1729061874328200000, end_id=1729061874328200001) [2024-10-16 14:57:54.329869] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:54.332921] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=12] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:54.334845] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:54.334860] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28020300, tenant_id=1002) [2024-10-16 14:57:54.334876] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:54.344070] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=226, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:54.344092] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=38, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:54.344108] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:54.344125] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:54.351675] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:54.357261] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061874356552000, start_id=1729061874356552000, end_id=1729061874356552001) [2024-10-16 14:57:54.360153] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061874358642000, start_id=1729061874358642000, end_id=1729061874358642001) [2024-10-16 14:57:54.360336] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:54.360349] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54432140, tenant_id=1003) [2024-10-16 14:57:54.360361] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:54.374272] INFO [SHARE] adaptive_update_limit (ob_share_throttle_define.cpp:112) [62903][T1004_L0_G0][T1004][YB4A0A000409-00061E9A11A789DE-0-0] [lt=21] adaptive update(Tenant ID=1004, Config Specify Resource Limit(MB)=511, TxShare Current Memory Limit(MB)=154, Holding Memory(MB)=46, Tenant Remain Memory(MB)=179, Usable Remain Memory(MB)=107, Last Update Limit Timestamp=1729061874372955, Is Updated=true) [2024-10-16 14:57:54.374355] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [62903][T1004_L0_G0][T1004][YB4A0A000409-00061E9A11A789DE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061874372955000, start_id=1729061874372955000, end_id=1729061874372955001) [2024-10-16 14:57:54.374949] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=46252773, this={dio_aligned_buf:{buf_write_offset:1288, buf_padding_size:2808, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33201991, truncate_used_ts:11502150}, log_block_size:67108864, dir_fd:195, io_fd:131}, aligned_buf_len=4096, aligned_block_offset=46252032, buf_len=547, total_write_size_=15666, total_write_size_after_dio_=434176, ob_pwrite_used_ts=32268, count=102, each_pwrite_cost=316) [2024-10-16 14:57:54.375710] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.383114] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D4D-0-0] [lt=36] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:54.384518] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.391694] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183801738, max_lsn:114264863890, max_scn:{val:1729061874358642000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45336899169}, reuse_lsn:{lsn:114264863890}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114222920850}}, last_submit_lsn:{lsn:114264863297}, last_submit_end_lsn:{lsn:114264863890}, last_submit_log_id:183801738, last_submit_log_pid:33576, max_flushed_lsn:{lsn:114264863297}, max_flushed_end_lsn:{lsn:114264863890}, max_flushed_log_pid:33576, committed_end_lsn:{lsn:114264863890}, last_slide_log_id:183801738, last_slide_scn:{val:1729061874358642000, v:0}, last_slide_lsn:{lsn:114264863297}, last_slide_end_lsn:{lsn:114264863890}, last_slide_log_pid:33576, last_slide_log_accum_checksum:2805624379, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e2efe2530}, [StateMgr]={this:0x2b0e2efe6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, leader:"10.0.4.7:2890", leader_epoch:2, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183801739, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261840845, palf_role_change_cb:0x2b0e2efed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, reconfig_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, persistent_config_version:{proposal_id:33576, config_seq:9}, ms_ack_list:0, resend_config_version:{proposal_id:33576, config_seq:9}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494259837171, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494259837171, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e2efe3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262041271, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729031774558563002, v:0}, min_block_id:1697, min_block_min_scn:{val:1729025370959302001, v:0}, base_lsn_for_block_gc:{lsn:114212315136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, log_config_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, log_snapshot_meta:{version:1, base_lsn:{lsn:114212315136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:232079360}, readable_log_tail:{lsn:232079360}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114279419904}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/meta", dir_fd:187, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:36339712, block_header_serialize_buf_:0x2b0e2efe89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114264863890}, readable_log_tail:{lsn:114264863890}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114212315136}, min_scn:{val:1729057087482360001, v:0}, curr_block_id:1702, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/log", dir_fd:190, min_block_id:1697, max_block_id:1703, curr_writable_block_id:1702}, logical_block_size_:67104768, curr_block_writable_size_:14556014, block_header_serialize_buf_:0x2b0e2efea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261839896, this:0x2b0e2efe6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33576, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33575, majority_max_lsn:{lsn:45338084276}, saved_end_lsn:{lsn:45338084276}, last_submit_prepare_req_time_us:1722494261840968, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e2efe69b0}) [2024-10-16 14:57:54.391843] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=145] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061874360476, last_flushed_end_lsn:{lsn:114264863890}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061874360592, last_flushed_end_lsn:{lsn:114264863890}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061874366293, last_flushed_end_lsn:{lsn:114264863890}}]) [2024-10-16 14:57:54.391855] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183801739, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061874360476, last_flushed_end_lsn:{lsn:114264863890}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061874360592, last_flushed_end_lsn:{lsn:114264863890}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061874366293, last_flushed_end_lsn:{lsn:114264863890}}]) [2024-10-16 14:57:54.391962] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=25] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:54.395171] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:54.395622] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601454, offset=9640060}, end_cursor:ObLogCursor{file_id=192, log_id=47601455, offset=9640344}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7312050, len:1924, log_data_len:284, seq:47601454, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:54.396691] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=13] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:54.397705] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B0-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061874397421000, start_id=1729061874397421000, end_id=1729061874397421001) [2024-10-16 14:57:54.398897] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601455, offset=9640344}, end_cursor:ObLogCursor{file_id=192, log_id=47601456, offset=9640618}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e2c050, len:1640, log_data_len:274, seq:47601455, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:54.399064] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=202, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:54.399082] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=72, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:54.399105] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:54.399658] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137598991, miss_cnt:9, cache_read_size:149777927589, hit_ratio:"9.999999345925478789e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:54.401002] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=6] Garbage Collector is running(seq_=656750, gc_interval=10000000) [2024-10-16 14:57:54.401029] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=11] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:54.401044] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=8] gc_check_member_list_ cost time(ret=0, time_us=31) [2024-10-16 14:57:54.401053] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:54.401058] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:54.401067] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:54.401075] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=8) [2024-10-16 14:57:54.401079] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=2] ObSafeDestroyHandler start process [2024-10-16 14:57:54.401438] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601456, offset=9640618}, end_cursor:ObLogCursor{file_id=192, log_id=47601457, offset=9640892}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e14050, len:1366, log_data_len:274, seq:47601456, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:54.401587] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601457, offset=9640892}, end_cursor:ObLogCursor{file_id=192, log_id=47601458, offset=9641166}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e34050, len:1092, log_data_len:274, seq:47601457, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:54.401633] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561403, miss_cnt:1, cache_read_size:152590695159, hit_ratio:"9.999999928346952061e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:54.430196] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=29, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:57:54.430213] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=16] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=1337, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:57:54.430222] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=8] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:57:54.430225] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=-247, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:57:54.430228] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=1119, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:57:54.439415] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:54.443898] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:54.443916] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:54.444174] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:54.447726] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061874447173000, start_id=1729061874447173000, end_id=1729061874447173001) [2024-10-16 14:57:54.471119] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE786B-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:54.471146] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE786B-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:54.471151] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE786B-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=11532, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:54.471155] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE786B-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938397, log_task={header:{begin_lsn:{lsn:213058255177}, end_lsn:{lsn:213058255299}, log_id:220938397, min_scn:{val:1729061874153492004, v:0}, max_scn:{val:1729061874153492004, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058255055}, prev_proposal_id:88, committed_end_lsn:{lsn:213058255177}, data_checksum:1715074492, accum_checksum:131314057, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061874371077, freeze_ts:1729061874371077, submit_ts:1729061874371082, flushed_ts:1729061874372686, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:1604}, fs_cb_begin_ts=1729061874471141, log_life_time=100064) [2024-10-16 14:57:54.471177] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:54.471214] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:54.471222] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:54.471453] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=274, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:54.471471] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=81, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:54.472018] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257802, offset=39044321}, end_cursor:ObLogCursor{file_id=197, log_id=49257803, offset=39044604}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73d6050, len:2847, log_data_len:283, seq:49257802, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:54.474172] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB37B-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061874473973000, start_id=1729061874473973000, end_id=1729061874473973001) [2024-10-16 14:57:54.475065] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257803, offset=39044604}, end_cursor:ObLogCursor{file_id=197, log_id=49257804, offset=39044886}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7296050, len:2564, log_data_len:282, seq:49257803, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:54.475874] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.477224] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257804, offset=39044886}, end_cursor:ObLogCursor{file_id=197, log_id=49257805, offset=39045168}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db737e050, len:2282, log_data_len:282, seq:49257804, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:54.477359] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257805, offset=39045168}, end_cursor:ObLogCursor{file_id=197, log_id=49257806, offset=39045450}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e74050, len:2000, log_data_len:282, seq:49257805, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:54.477399] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=63] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650341, miss_cnt:32, cache_read_size:8050432913, hit_ratio:"9.999994892288989190e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:54.477412] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=40] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657081, miss_cnt:17, cache_read_size:8050180519, hit_ratio:"9.999997286819762854e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:54.482861] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8C-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:54.482878] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8C-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:54.483201] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D4F-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:54.483675] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9225-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:54.483684] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9225-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=589, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:54.484614] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.498777] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=41] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:54.499388] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:54.499439] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061868730909000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:54.499706] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4466-0-0] [lt=20] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:54.499716] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4466-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30526, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:54.505960] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C88-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061874505924000, start_id=1729061874505924000, end_id=1729061874505924001) [2024-10-16 14:57:54.519167] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061874518610000, start_id=1729061874518610000, end_id=1729061874518610001) [2024-10-16 14:57:54.520539] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061874519670000, start_id=1729061874519670000, end_id=1729061874519670001) [2024-10-16 14:57:54.520974] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=21] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061874318374000, v:0}, new_max_version={val:1729061874318374000, v:0}, last_min_version={val:1729061874117940000, v:0}, last_max_version={val:1729061874117940000, v:0}, total_time_us=2655, delta_us=202599) [2024-10-16 14:57:54.520990] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061874318374000, v:0}, min_version={val:1729061874318374000, v:0}, max_version={val:1729061874318374000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:54.529619] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:54.529631] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:54.529865] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=37] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:54.533908] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:54.533923] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:54.533938] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444593612, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:54.543418] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=909004800, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=164737024, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:54.543476] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=25] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:54.543488] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=12] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:54.543518] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=11] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=969699328, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=104042496, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:54.543561] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=14] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:54.543569] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:54.543598] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=6] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=927879168, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=145862656, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:54.543645] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=17] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:54.543652] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011869, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:54.543683] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=6] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=958939136, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=114802688, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:54.543718] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:54.543725] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:54.543929] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:54.543946] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:54.544222] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2200) [2024-10-16 14:57:54.544442] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4285-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:54.544451] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4285-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=557, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:54.551723] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:54.557316] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061874555861000, start_id=1729061874555861000, end_id=1729061874555861001) [2024-10-16 14:57:54.560281] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061874559766000, start_id=1729061874559766000, end_id=1729061874559766001) [2024-10-16 14:57:54.560492] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061874559766000, start_id=1729061874559766000, end_id=1729061874559766001) [2024-10-16 14:57:54.560513] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=505, get_gts_with_stc_cnt=75, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=75, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:54.560521] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061874560509], gts:1729061874559766000, latest_srr:[mts=1729061874560509]}) [2024-10-16 14:57:54.560706] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=511, get_gts_with_stc_cnt=8, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:54.561180] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD213-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10045) [2024-10-16 14:57:54.568868] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.568977] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.569113] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.569515] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.575923] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.583746] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9227-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=230, avg_log_submit_to_slide_time=578) [2024-10-16 14:57:54.583761] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:54.584678] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.591838] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD014-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061874583066, keepalive_ts:-1}) [2024-10-16 14:57:54.591873] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD014-0-0] [lt=34] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061874583066, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:54.591918] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD015-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061874583112, keepalive_ts:-1}) [2024-10-16 14:57:54.591927] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD015-0-0] [lt=8] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061874583112, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:54.596812] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:54.596826] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:54.597200] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=9] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1}, {id:1001}]) [2024-10-16 14:57:54.601821] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=11] worker thread began to shrink(tenant_->id()=1002, group_id_=1, token=0) [2024-10-16 14:57:54.603564] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B2F-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061874602963000, start_id=1729061874602963000, end_id=1729061874602963001) [2024-10-16 14:57:54.603635] INFO [LIB] stat (utility.h:1165) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B2F-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:54.603648] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B2F-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=5718, avg_group_log_size=476) [2024-10-16 14:57:54.603665] INFO [LIB] stat (utility.h:1165) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B2F-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:54.603882] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23562700, this={dio_aligned_buf:{buf_write_offset:3054, buf_padding_size:1042, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:47004108, truncate_used_ts:19010317}, log_block_size:67108864, dir_fd:134, io_fd:183}, aligned_buf_len=4096, aligned_block_offset=23560192, buf_len=546, total_write_size_=39350, total_write_size_after_dio_=458752, ob_pwrite_used_ts=29902, count=103, each_pwrite_cost=290) [2024-10-16 14:57:54.603901] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=268, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:54.603913] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=159, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:54.603928] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=7, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:54.604137] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10A7-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=574, avg_log_submit_to_slide_time=1197) [2024-10-16 14:57:54.604187] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DA7-0-0] [lt=8] [APPLY STAT CB APPEND COST TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=45, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:54.604193] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DA7-0-0] [lt=6] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=965, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:54.604197] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DA7-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:54.604200] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DA7-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=-418, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:54.604203] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DA7-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=593, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:54.604926] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107522][T1002_L0_G1][T1002][Y0-0000000000000000-0-0] [lt=8] worker thread exit(tenant_->id()=1002, workers_.get_size()=2) [2024-10-16 14:57:54.604941] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107522][T1002_L0_G1][T0][Y0-0000000000000000-0-0] [lt=12] tenant thread end_run(id_=1002, ret=0, thread_count_=190) [2024-10-16 14:57:54.607633] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1E02-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061874606082000, start_id=1729061874606082000, end_id=1729061874606082001) [2024-10-16 14:57:54.607890] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=52555928, this={dio_aligned_buf:{buf_write_offset:701, buf_padding_size:3395, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80311128, truncate_used_ts:29009546}, log_block_size:67108864, dir_fd:190, io_fd:179}, aligned_buf_len=4096, aligned_block_offset=52555776, buf_len=549, total_write_size_=104804, total_write_size_after_dio_=827392, ob_pwrite_used_ts=75813, count=176, each_pwrite_cost=430) [2024-10-16 14:57:54.607909] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=217, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:54.607918] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=43, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:54.607931] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:54.608148] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD214-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:54.608156] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD214-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=484, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:54.608165] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD214-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=277, avg_log_submit_to_slide_time=455) [2024-10-16 14:57:54.610442] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5600, mem_used=10398720) [2024-10-16 14:57:54.622945] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [21736][T1001_COMMONLSS][T1001][YB4A0A000407-00061E9B3A29CB07-0-0] [lt=21][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:57:54.625682] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B5A0C6797-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061874624966000, start_id=1729061874624966000, end_id=1729061874624966001) [2024-10-16 14:57:54.633724] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B5A0C6797-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061874633405000, start_id=1729061874633405000, end_id=1729061874633405001) [2024-10-16 14:57:54.638374] INFO [ARCHIVE] do_thread_task_ (ob_archive_fetcher.cpp:310) [22243][T1002_ArcFetche][T1002][YB4A0A000407-00061E996F5960C1-0-0] [lt=13] ObArchiveFetcher is running(thread_index=0) [2024-10-16 14:57:54.639710] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:54.642215] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B5A0C6797-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061874641158000, start_id=1729061874641158000, end_id=1729061874641158001) [2024-10-16 14:57:54.647852] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061874647646000, start_id=1729061874647646000, end_id=1729061874647646001) [2024-10-16 14:57:54.647912] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=10001, avg_group_log_size=625) [2024-10-16 14:57:54.647932] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10001) [2024-10-16 14:57:54.648346] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10001) [2024-10-16 14:57:54.651831] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B5A0C6797-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061874650853000, start_id=1729061874650853000, end_id=1729061874650853001) [2024-10-16 14:57:54.654593] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA50-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061874653969000, start_id=1729061874653969000, end_id=1729061874653969001) [2024-10-16 14:57:54.656234] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=200, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:54.656252] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=168, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:54.656293] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:54.665437] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:54.672287] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:54.674400] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D0-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:54.674425] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D0-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061874653969000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061874653969000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:54.676482] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.677854] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D0-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061874677278000, start_id=1729061874677278000, end_id=1729061874677278001) [2024-10-16 14:57:54.678359] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=26] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:54.684739] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:54.684758] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:54.684764] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.695233] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22398][T1003_TntShared][T1003][Y0-0000000000000000-0-0] [lt=15] clear dtl interm result cost(us)(clear_cost=15, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:54.699744] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:54.702038] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DAF-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510342570432, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410620, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:54.702825] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DB3-0-0] [lt=11][errcode=-4201] hashtable not init [2024-10-16 14:57:54.702833] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DB3-0-0] [lt=8][errcode=-4016] get cursor info failed(cursor_id=1117, get_sessid()=3221565766) [2024-10-16 14:57:54.702952] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DB4-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061874700605000, start_id=1729061874700605000, end_id=1729061874700605001) [2024-10-16 14:57:54.704300] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:54.704945] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061874704165000, start_id=1729061874704165000, end_id=1729061874704165001) [2024-10-16 14:57:54.707518] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31896][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB1FCC6DC-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:54.712683] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=24] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:54.716097] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=5] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:54.716118] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=215) [2024-10-16 14:57:54.719213] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061874717857000, start_id=1729061874717857000, end_id=1729061874717857001) [2024-10-16 14:57:54.719832] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:54.720676] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061874720383000, start_id=1729061874720383000, end_id=1729061874720383001) [2024-10-16 14:57:54.722447] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B22-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061874721443000, start_id=1729061874721443000, end_id=1729061874721443001) [2024-10-16 14:57:54.722682] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=42748403, this={dio_aligned_buf:{buf_write_offset:3095, buf_padding_size:1001, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35133309, truncate_used_ts:12254366}, log_block_size:67108864, dir_fd:169, io_fd:144}, aligned_buf_len=4096, aligned_block_offset=42745856, buf_len=548, total_write_size_=16097, total_write_size_after_dio_=434176, ob_pwrite_used_ts=41611, count=102, each_pwrite_cost=407) [2024-10-16 14:57:54.727304] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:54.727625] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5D-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:54.727646] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5D-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061874653969001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061874653969001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:54.729707] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2614, avg_group_log_size=237) [2024-10-16 14:57:54.729721] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2614) [2024-10-16 14:57:54.729992] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2736) [2024-10-16 14:57:54.730816] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=71] success to update min reserved snapshot(reserved_snapshot=1729060071843583000, duration=1800, snapshot_gc_ts=1729061871843583000) [2024-10-16 14:57:54.731548] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C5D-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061874731309000, start_id=1729061874731309000, end_id=1729061874731309001) [2024-10-16 14:57:54.731663] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23170519, this={dio_aligned_buf:{buf_write_offset:3655, buf_padding_size:441, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33385027, truncate_used_ts:14052890}, log_block_size:67108864, dir_fd:200, io_fd:174}, aligned_buf_len=4096, aligned_block_offset=23166976, buf_len=112, total_write_size_=12050, total_write_size_after_dio_=417792, ob_pwrite_used_ts=29734, count=99, each_pwrite_cost=300) [2024-10-16 14:57:54.732153] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=10] update reserved snapshot finished(cost_ts=1340, reserved_snapshot=1729060071843583000) [2024-10-16 14:57:54.751759] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:54.755867] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53ED6-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:54.757428] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061874755539000, start_id=1729061874755539000, end_id=1729061874755539001) [2024-10-16 14:57:54.758905] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:54.758929] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=13] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:54.758939] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:54.760322] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061874759665000, start_id=1729061874759665000, end_id=1729061874759665001) [2024-10-16 14:57:54.768673] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=11] LogLoopThread round_cost_time(us)(round_cost_time=11) [2024-10-16 14:57:54.772432] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:54.776530] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.782326] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=13] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:54.784748] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.788105] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.788112] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.788121] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.788125] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.788534] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.788539] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.789004] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.789205] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.789307] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.789538] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.789538] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:54.793157] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=23] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:54.796776] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:54.801278] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D4A-0-0] [lt=13] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061874792495, keepalive_ts:-1}) [2024-10-16 14:57:54.801296] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D4A-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061874792495, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:54.801317] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D4B-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061874792551, keepalive_ts:-1}) [2024-10-16 14:57:54.801323] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D4B-0-0] [lt=6] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061874792551, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:54.817837] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:54.817906] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1 and flag like "%DUPLICATE%", tenant_id=1, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:54.818469] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061874602963000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=215501, in_cluster_service=true, cluster_version={val:1729061874518610000, v:0}, min_cluster_version={val:1729061874518610000, v:0}, max_cluster_version={val:1729061874518610000, v:0}, get_cluster_version_err=0, cluster_version_delta=299854, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65249029, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=372091179, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:54.822442] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=14][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:54.822491] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=14][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1001 and flag like "%DUPLICATE%", tenant_id=1001, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:57:54.827248] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=77] success to update min reserved snapshot(reserved_snapshot=1729060067249743000, duration=1800, snapshot_gc_ts=1729061867249743000) [2024-10-16 14:57:54.828312] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=14] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060067249743000) [2024-10-16 14:57:54.828329] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=14] update reserved snapshot finished(cost_ts=1085, reserved_snapshot=1729060067249743000) [2024-10-16 14:57:54.831413] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1003}, {id:1001}, {id:1}]) [2024-10-16 14:57:54.831672] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD39A-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:54.831682] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD39A-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=726, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:54.831686] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD39A-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=23, avg_log_submit_to_flush_time=435, avg_log_submit_to_slide_time=714) [2024-10-16 14:57:54.839962] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=38] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:54.844363] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=320, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:54.844377] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=31, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:54.844392] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:54.844711] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4471-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=376, avg_log_submit_to_slide_time=1153) [2024-10-16 14:57:54.848021] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061874847552000, start_id=1729061874847552000, end_id=1729061874847552001) [2024-10-16 14:57:54.856068] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53ED8-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:54.856268] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=59079176, this={dio_aligned_buf:{buf_write_offset:2690, buf_padding_size:1406, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:38334883, truncate_used_ts:14111024}, log_block_size:67108864, dir_fd:207, io_fd:202}, aligned_buf_len=4096, aligned_block_offset=59076608, buf_len=122, total_write_size_=12294, total_write_size_after_dio_=425984, ob_pwrite_used_ts=24839, count=101, each_pwrite_cost=245) [2024-10-16 14:57:54.856392] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD39C-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:54.856407] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD39C-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=17015, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:54.856412] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD39C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=396, avg_log_submit_to_slide_time=17009) [2024-10-16 14:57:54.856449] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1454, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:54.861126] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5673, mem_used=10398720) [2024-10-16 14:57:54.866949] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4890-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061874834676, keepalive_ts:-1}) [2024-10-16 14:57:54.866967] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4890-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061874834676, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:54.876620] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.883150] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=6821) [2024-10-16 14:57:54.883185] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=6821) [2024-10-16 14:57:54.883352] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10AB-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:54.883361] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10AB-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=980, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:54.883852] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=45, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:54.883871] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:54.884814] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.897579] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=15] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:54.902360] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=9] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:54.903423] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:57:54.903490] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1004 and flag like "%DUPLICATE%", tenant_id=1004, exec_tenant_id=1003, need_member_list=false) [2024-10-16 14:57:54.919332] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061874919150000, start_id=1729061874919150000, end_id=1729061874919150001) [2024-10-16 14:57:54.920809] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=17][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:54.920865] WDIAG read_multi_row (ob_table_access_helper.h:238) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=14][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:54.921390] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061874920197000, start_id=1729061874920197000, end_id=1729061874920197001) [2024-10-16 14:57:54.921640] INFO delete_row (ob_table_access_helper.h:312) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1001, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061874920872) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061874920872) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:54.924286] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:54.925534] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234B-0-0] [lt=34] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:57:54.926166] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234B-0-0] [lt=14] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1004) [2024-10-16 14:57:54.926182] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234B-0-0] [lt=15] no transfer partition job [2024-10-16 14:57:54.926188] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234B-0-0] [lt=4] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:57:54.927455] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234B-0-0] [lt=7] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1004, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:57:54.927475] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234B-0-0] [lt=15] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:57:54.928641] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234B-0-0] [lt=12] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1002, status:0}], ls_array_=[{tenant_id:1004, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729059563578310, last_statistic_bg_stat_time=1725269669481368, last_statistic_schema_version=1725266205820024, last_statistic_max_transfer_task_id={id:5}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:57:54.929831] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:54.929844] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:54.930098] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1210) [2024-10-16 14:57:54.935229] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7A-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:54.935254] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7A-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061874665620003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061874665620003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:54.936252] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D7A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061874935284000, start_id=1729061874935284000, end_id=1729061874935284001) [2024-10-16 14:57:54.936823] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15E4-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:54.937550] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE8-0-0] [lt=10] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061874905239, keepalive_ts:-1}) [2024-10-16 14:57:54.937564] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE8-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061874905239, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:54.938523] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7A-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061874938412000, start_id=1729061874938412000, end_id=1729061874938412001) [2024-10-16 14:57:54.938586] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7A-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:54.938606] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7A-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:54.944703] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3679, mem_used=8318976) [2024-10-16 14:57:54.951283] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:54.951342] WDIAG read_multi_row (ob_table_access_helper.h:238) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:54.951838] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:54.952079] INFO delete_row (ob_table_access_helper.h:312) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=10] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1004, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061874951352) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061874951352) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:54.952146] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=16] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:54.952202] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=16] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:54.952251] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:54.960357] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061874959526000, start_id=1729061874959526000, end_id=1729061874959526001) [2024-10-16 14:57:54.960629] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061874960589000, start_id=1729061874960589000, end_id=1729061874960589001) [2024-10-16 14:57:54.960932] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:54.971221] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=11] ====== tenant freeze timer task ====== [2024-10-16 14:57:54.971301] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:57:54.971311] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=6] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27208900, tenant_id=1004) [2024-10-16 14:57:54.971322] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:54.973925] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=10] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:54.976678] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:54.981699] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2017-0-0] [lt=14] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:54.981720] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2017-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:54.984896] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.016870] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:55.016977] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAE-0-0] [lt=13] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061874960589000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061885590307000, latest_log_ts:{val:1729061868948338001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:55.017044] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAE-0-0] [lt=64] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:55.017084] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:55.017102] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151574520, tenant_id=1) [2024-10-16 14:57:55.017116] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:55.029883] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=20] PNIO [ratelimit] time: 1729061875029881, bytes: 4446819719007, bw: 0.456168 MB/s, add_ts: 1000761, add_bytes: 478691 [2024-10-16 14:57:55.030194] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=375, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:55.030209] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=66, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:55.030233] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:55.030460] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3A1-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:55.030466] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3A1-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1304, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:55.040263] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:55.042971] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:55.042985] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:55.043001] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=291980620, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:55.043256] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Cache wash clean map node details(ret=0, clean_node_count=26, clean_time=0, clean_start_pos=1447022, clean_num=62914) [2024-10-16 14:57:55.044265] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:55.044278] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:55.044850] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4477-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:55.044869] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4477-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1235, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:55.048143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061875047118000, start_id=1729061875047118000, end_id=1729061875047118001) [2024-10-16 14:57:55.055994] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EDC-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:55.056035] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EDC-0-0] [lt=16][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173284, log_task={header:{begin_lsn:{lsn:14352390786}, end_lsn:{lsn:14352390908}, log_id:115173284, min_scn:{val:1729061874653969003, v:0}, max_scn:{val:1729061874653969003, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352390664}, prev_proposal_id:17, committed_end_lsn:{lsn:14352390786}, data_checksum:940009244, accum_checksum:2190825947, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061874955909, freeze_ts:1729061874955909, submit_ts:1729061874955912, flushed_ts:1729061874961007, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:5095}, fs_cb_begin_ts=1729061875056013, log_life_time=100104) [2024-10-16 14:57:55.060892] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061875059921000, start_id=1729061875059921000, end_id=1729061875059921001) [2024-10-16 14:57:55.060916] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:55.060930] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061875060927], gts:1729061875059921000, latest_srr:[mts=1729061875060927]}) [2024-10-16 14:57:55.060944] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=548, get_gts_with_stc_cnt=80, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=77, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:55.071499] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7877-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:55.071723] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:55.076712] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.082970] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=6821) [2024-10-16 14:57:55.083496] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D5B-0-0] [lt=22] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:55.083691] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:55.084961] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.091425] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] ====== checkpoint timer task ====== [2024-10-16 14:57:55.092179] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=13] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:55.092206] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:55.092227] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:55.092350] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061887008855000, latest_log_ts:{val:1729061870498146001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:55.092409] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=55] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:55.092420] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:57:55.092430] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=0, rec_log_ts={val:1729061870498146001, v:0}) [2024-10-16 14:57:55.092437] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:55.092441] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:55.092451] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:55.092464] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:57:55.092473] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:55.092479] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:55.119497] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061875118533000, start_id=1729061875118533000, end_id=1729061875118533001) [2024-10-16 14:57:55.121408] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22235][T1002_TenantWea][T1001][YB4A0A000407-00061E9B3B0B77CF-0-0] [lt=10] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.132496929168701172e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=306, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.971485825093569474e+03) [2024-10-16 14:57:55.121532] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061875120719000, start_id=1729061875120719000, end_id=1729061875120719001) [2024-10-16 14:57:55.127753] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:55.127769] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:55.130183] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=370, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:55.130199] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=56, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:55.137380] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3827, mem_used=8318976) [2024-10-16 14:57:55.137539] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 167.50; ) [2024-10-16 14:57:55.137559] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:55.137609] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 117.25; ) [2024-10-16 14:57:55.137618] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:55.137626] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:55.144259] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2620, avg_group_log_size=238) [2024-10-16 14:57:55.144275] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2620) [2024-10-16 14:57:55.144545] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2742) [2024-10-16 14:57:55.144785] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4479-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=263, avg_log_submit_to_slide_time=487) [2024-10-16 14:57:55.144808] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=48, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:57:55.144816] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=8] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=1031, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:57:55.144819] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:57:55.144823] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=-506, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:57:55.144826] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=574, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:57:55.149518] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:55.153308] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2327818, replace_num=31457) [2024-10-16 14:57:55.153326] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:55.155200] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA52-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061875154298000, start_id=1729061875154298000, end_id=1729061875154298001) [2024-10-16 14:57:55.158708] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA53-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061875158504000, start_id=1729061875158504000, end_id=1729061875158504001) [2024-10-16 14:57:55.158770] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0CDF-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061875158504000, start_id=1729061875158504000, end_id=1729061875158504001) [2024-10-16 14:57:55.158967] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0CE0-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061875158504000, start_id=1729061875158504000, end_id=1729061875158504001) [2024-10-16 14:57:55.160898] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0CDC-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061875159518000, start_id=1729061875159518000, end_id=1729061875159518001) [2024-10-16 14:57:55.160909] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061875159518000, start_id=1729061875159518000, end_id=1729061875159518001) [2024-10-16 14:57:55.160966] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:55.160980] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:55.162087] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061875161077000, start_id=1729061875161077000, end_id=1729061875161077001) [2024-10-16 14:57:55.162371] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get_max_applied_scn(scn={val:1729061874960589000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618657274}, palf_committed_end_scn_:{val:1729061875159518000, v:0}, last_check_scn_:{val:1729061875159518000, v:0}, max_applied_cb_scn_:{val:1729061874960589000, v:0}}) [2024-10-16 14:57:55.162392] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:55.162460] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA51-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061875162175000, start_id=1729061875162175000, end_id=1729061875162175001) [2024-10-16 14:57:55.164365] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061875164364, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007613, add_bytes: 0 [2024-10-16 14:57:55.173803] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:55.176780] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.184989] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.189410] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31906][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB20CC6DC-0-0] [lt=20] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:57:55.194742] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=25] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:55.197256] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5039-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:55.197272] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5039-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:55.197385] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:55.210557] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587E-0-0] [lt=8] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:55.210662] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587E-0-0] [lt=15][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:55.230002] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:55.230026] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:55.230342] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2060) [2024-10-16 14:57:55.230505] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3A7-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=466, avg_log_submit_to_slide_time=1254) [2024-10-16 14:57:55.243521] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:55.246007] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD0F-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:55.248099] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061875247912000, start_id=1729061875247912000, end_id=1729061875247912001) [2024-10-16 14:57:55.259586] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DB5-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510343127980, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410621, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:55.261213] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061875047118000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=214090, in_cluster_service=true, cluster_version={val:1729061874847552000, v:0}, min_cluster_version={val:1729061874847552000, v:0}, max_cluster_version={val:1729061874847552000, v:0}, get_cluster_version_err=0, cluster_version_delta=413656, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244345, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=484331011, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:55.263141] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88D-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:55.263168] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88D-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061874653969004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061874653969004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:55.264062] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA88D-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061875263153000, start_id=1729061875263153000, end_id=1729061875263153001) [2024-10-16 14:57:55.266282] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88D-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061875265270000, start_id=1729061875265270000, end_id=1729061875265270001) [2024-10-16 14:57:55.266352] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88D-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10120, avg_group_log_size=595) [2024-10-16 14:57:55.266363] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88D-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10120) [2024-10-16 14:57:55.266578] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=381, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:55.266593] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10120) [2024-10-16 14:57:55.266614] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=57, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:55.266629] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10120) [2024-10-16 14:57:55.266640] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:55.267000] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95C3-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:55.267014] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95C3-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=943, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:55.267018] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95C3-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=724, avg_log_submit_to_slide_time=913) [2024-10-16 14:57:55.267042] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:55.272290] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE787B-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:55.275154] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:55.276837] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.277847] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=45] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:57:55.277858] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:57:55.277862] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:55.278450] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061885590307000, latest_log_ts:{val:1729061868948338001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:55.283425] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB9D-0-0] [lt=14][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:55.283854] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=190, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:55.283870] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:55.283975] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:55.285056] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.285645] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=13] start to print blacklist info [2024-10-16 14:57:55.285809] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10] ls blacklist refresh finish(cost_time=7508) [2024-10-16 14:57:55.296542] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850B1-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:55.296568] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850B1-0-0] [lt=20] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:55.296576] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850B1-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:55.296593] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850B1-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497552, log_task={header:{begin_lsn:{lsn:207589564002}, end_lsn:{lsn:207589564124}, log_id:218497552, min_scn:{val:1729061875158504000, v:0}, max_scn:{val:1729061875158504000, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589563880}, prev_proposal_id:81, committed_end_lsn:{lsn:207589564002}, data_checksum:940009244, accum_checksum:3682244100, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061875196461, freeze_ts:1729061875196461, submit_ts:1729061875196465, flushed_ts:1729061875197405, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:940}, fs_cb_begin_ts=1729061875296585, log_life_time=100124) [2024-10-16 14:57:55.307705] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD103D-0-0] [lt=13] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:55.309885] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1118-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061875309622000, start_id=1729061875309622000, end_id=1729061875309622001) [2024-10-16 14:57:55.319449] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061875319115000, start_id=1729061875319115000, end_id=1729061875319115001) [2024-10-16 14:57:55.321575] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061875321234000, start_id=1729061875321234000, end_id=1729061875321234001) [2024-10-16 14:57:55.328544] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=293, mem_used=2079744) [2024-10-16 14:57:55.330364] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1210) [2024-10-16 14:57:55.332974] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:55.344649] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=217, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:55.344670] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=61, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:55.344684] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:55.344697] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:55.353361] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:55.360134] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=39] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061875159518000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=200608, in_cluster_service=true, cluster_version={val:1729061874960589000, v:0}, min_cluster_version={val:1729061874960589000, v:0}, max_cluster_version={val:1729061874960589000, v:0}, get_cluster_version_err=0, cluster_version_delta=399537, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244592, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=249756297, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:55.360174] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=29] no need to update min/max version(new_version={val:1729061874960589000, v:0}, min_version={val:1729061874960589000, v:0}, max_version={val:1729061874960589000, v:0}) [2024-10-16 14:57:55.362211] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061875361270000, start_id=1729061875361270000, end_id=1729061875361270001) [2024-10-16 14:57:55.365976] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3858-0-0] [lt=12] MediumCheckTask(cost_ts=6) [2024-10-16 14:57:55.373027] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB4-0-0] [lt=9] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:55.373041] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB4-0-0] [lt=10] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=15) [2024-10-16 14:57:55.376899] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.381075] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:55.381091] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=217) [2024-10-16 14:57:55.383544] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=12] blacklist_loop exec finished(cost_time=19, is_enabled=true, send_cnt=0) [2024-10-16 14:57:55.383737] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D61-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:55.385122] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.395203] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:55.405210] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601458, offset=9641166}, end_cursor:ObLogCursor{file_id=192, log_id=47601459, offset=9641450}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73aa050, len:818, log_data_len:284, seq:47601458, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:55.407438] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B1-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061875405352000, start_id=1729061875405352000, end_id=1729061875405352001) [2024-10-16 14:57:55.408297] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601459, offset=9641450}, end_cursor:ObLogCursor{file_id=192, log_id=47601460, offset=9641724}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72ea050, len:534, log_data_len:274, seq:47601459, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:55.410337] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601460, offset=9641724}, end_cursor:ObLogCursor{file_id=192, log_id=47601461, offset=9641998}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73c2050, len:4356, log_data_len:274, seq:47601460, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:55.410474] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601461, offset=9641998}, end_cursor:ObLogCursor{file_id=192, log_id=47601462, offset=9642272}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db736a050, len:4082, log_data_len:274, seq:47601461, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:55.410517] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561414, miss_cnt:1, cache_read_size:152590696492, hit_ratio:"9.999999928346957612e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:55.410547] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599001, miss_cnt:9, cache_read_size:149777928809, hit_ratio:"9.999999345925526528e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:55.443832] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:55.448277] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061875446929000, start_id=1729061875446929000, end_id=1729061875446929001) [2024-10-16 14:57:55.456179] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=9] replay status is not follower(min_unreplayed_lsn={lsn:7668488755}, base_lsn={lsn:7582838784}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:55.456242] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=63] get_replay_process success(id={id:1}, replayed_log_size=85649971, unreplayed_log_size=0) [2024-10-16 14:57:55.456250] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:6198988121}, base_lsn={lsn:6106533888}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:55.456275] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=25] get_replay_process success(id={id:1001}, replayed_log_size=92454233, unreplayed_log_size=0) [2024-10-16 14:57:55.456280] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=8163552079, unreplayed_log_size=0) [2024-10-16 14:57:55.456289] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=6] get_replay_process success(id={id:1003}, replayed_log_size=8111647972, unreplayed_log_size=0) [2024-10-16 14:57:55.456304] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1004, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=15691, last_replayed_log_size(MB)=15691, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:55.471759] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE787F-0-0] [lt=25] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:55.471984] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=189, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:55.472001] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:55.472236] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54BE-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:55.472245] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54BE-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9466, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:55.472254] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:55.472302] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:55.472307] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=5] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:55.476957] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.481011] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257806, offset=39045450}, end_cursor:ObLogCursor{file_id=197, log_id=49257807, offset=39045733}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e20050, len:1718, log_data_len:283, seq:49257806, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:55.483033] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22628][T1004_TeRec][T1004][YB4A0A000407-00061E9B221FB37C-0-0] [lt=32] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=0, MAX_GAP=900000000000, new_sync_scn={val:1729061874372955000, v:0}, new_replayable_scn={val:1729061874372955000, v:0}, new_readable_scn={val:1729061874372955000, v:0}, old_tenant_info={tenant_id:1004, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061873056699003, v:0}, replayable_scn:{val:1729061873056699003, v:0}, standby_scn:{val:1729061873056699003, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"ARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:57:55.483235] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB37C-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061875483120000, start_id=1729061875483120000, end_id=1729061875483120001) [2024-10-16 14:57:55.483725] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D63-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:55.484022] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8D-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:55.484037] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8D-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:55.485187] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.485233] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257807, offset=39045733}, end_cursor:ObLogCursor{file_id=197, log_id=49257808, offset=39046015}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dc0050, len:1435, log_data_len:282, seq:49257807, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:55.487180] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257808, offset=39046015}, end_cursor:ObLogCursor{file_id=197, log_id=49257809, offset=39046297}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7342050, len:1153, log_data_len:282, seq:49257808, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:55.487332] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257809, offset=39046297}, end_cursor:ObLogCursor{file_id=197, log_id=49257810, offset=39046579}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73ee050, len:871, log_data_len:282, seq:49257809, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:55.487374] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650351, miss_cnt:32, cache_read_size:8050434124, hit_ratio:"9.999994892289804094e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:55.487383] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657091, miss_cnt:17, cache_read_size:8050181739, hit_ratio:"9.999997286820195841e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:55.490581] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:55.490596] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:55.490614] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=10] cannot_recycle_log_size statistics(cannot_recycle_log_size=70821653, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:55.491917] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:55.491925] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=8] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:55.491934] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:55.491930] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A29-0-0] [lt=7] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:55.491940] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:55.491944] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:55.491942] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A29-0-0] [lt=10] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:55.491946] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:55.491949] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:55.491951] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:55.491954] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:55.491956] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:55.491959] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:55.495761] INFO [SHARE] get_server_tenant_configs (ob_unit_getter.cpp:231) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=8] get_server_tenant_configs(unit_infos=[{unit:unit_id:1, resource_pool_id:1, unit_group_id:1, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, pool:{resource_pool_id:1, name:"sys_pool", unit_count:1, unit_config_id:1, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1, replica_type:0}}, {unit:unit_id:1001, resource_pool_id:1001, unit_group_id:1001, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1001, name:"ocp_pool", unit_count:1, unit_config_id:1001, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1002, replica_type:0}}, {unit:unit_id:1005, resource_pool_id:1003, unit_group_id:1002, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1003, name:"pool_bddb_myoceanbase1_oav", unit_count:1, unit_config_id:1003, zone_list:["myoceanbase1"], tenant_id:1004, replica_type:0}}]) [2024-10-16 14:57:55.495806] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=42] jx_debug: get tenant compatibility mode(tenant_id=1002, compat_mode=0) [2024-10-16 14:57:55.495812] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=5] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:55.495826] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=13] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:55.495841] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=14] jx_debug: get tenant compatibility mode(tenant_id=1004, compat_mode=0) [2024-10-16 14:57:55.495844] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=3] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:57:55.495859] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=14] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:57:55.497154] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:191) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=11] refresh resource plan success(tenant_id=1, plan_name=, directives=[]) [2024-10-16 14:57:55.497827] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=13] refresh resource user mapping rule(tenant_id=1, plan=, user_rules=[]) [2024-10-16 14:57:55.498487] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=14] refresh_resource_function_mapping_rule(tenant_id=1, plan=, rules=[]) [2024-10-16 14:57:55.498502] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=15] refresh resource mapping rule success(tenant_id=1, plan=) [2024-10-16 14:57:55.498831] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=250, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:55.498847] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=225, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:55.498874] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:55.499147] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=7] trace tenant dfc parameters(max_parallel_cnt_=30, max_blocked_buffer_size_=9830400, max_buffer_size_=19660800) [2024-10-16 14:57:55.499204] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=16] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:55.499215] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=9] tenant dfc status(ret=0, get_tenant_id()=1, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=30, get_max_blocked_buffer_size()=9830400, get_max_buffer_size()=19660800, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:55.501146] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=8] refresh resource user mapping rule(tenant_id=1001, plan=, user_rules=[]) [2024-10-16 14:57:55.501861] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=11] refresh_resource_function_mapping_rule(tenant_id=1001, plan=, rules=[]) [2024-10-16 14:57:55.501876] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=15] refresh resource mapping rule success(tenant_id=1001, plan=) [2024-10-16 14:57:55.502517] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:55.502581] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=13] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:55.502591] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=9] tenant dfc status(ret=0, get_tenant_id()=1001, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:55.504581] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=18] refresh resource user mapping rule(tenant_id=1002, plan=, user_rules=[]) [2024-10-16 14:57:55.505248] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=15] refresh_resource_function_mapping_rule(tenant_id=1002, plan=, rules=[]) [2024-10-16 14:57:55.505259] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=10] refresh resource mapping rule success(tenant_id=1002, plan=) [2024-10-16 14:57:55.505885] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:55.505941] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=8] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:55.505947] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] tenant dfc status(ret=0, get_tenant_id()=1002, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=4, get_max_size_per_channel()=655360) [2024-10-16 14:57:55.507867] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] refresh resource user mapping rule(tenant_id=1003, plan=, user_rules=[]) [2024-10-16 14:57:55.508538] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=8] refresh_resource_function_mapping_rule(tenant_id=1003, plan=, rules=[]) [2024-10-16 14:57:55.508546] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=8] refresh resource mapping rule success(tenant_id=1003, plan=) [2024-10-16 14:57:55.509167] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=9] replay status is not follower(min_unreplayed_lsn={lsn:45686897415}, base_lsn={lsn:45497032704}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:57:55.509213] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=47] get_replay_process success(id={id:1}, replayed_log_size=189864711, unreplayed_log_size=0) [2024-10-16 14:57:55.509213] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:55.509231] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1001, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=181, last_replayed_log_size(MB)=181, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:55.509270] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=8] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:55.509276] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=5] tenant dfc status(ret=0, get_tenant_id()=1003, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:55.511238] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] refresh resource user mapping rule(tenant_id=1004, plan=, user_rules=[]) [2024-10-16 14:57:55.511264] INFO [MVCC] repeat_study (ob_multi_version_garbage_collector.cpp:196) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=5] skip repeat study(ret=0, this={this:0x2b0e401fcf30, last_study_timestamp:1729061455421203, last_refresh_timestamp:1729061815496037, last_reclaim_timestamp:1729061455422915, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061399238808003, v:0}, is_inited:true}, current_timestamp=1729061875511262, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:55.511928] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=9] refresh_resource_function_mapping_rule(tenant_id=1004, plan=, rules=[]) [2024-10-16 14:57:55.511927] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=18] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1729061455417681000, snapshot_type=1, create_time=1729061455418847, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:55.511937] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=8] refresh resource mapping rule success(tenant_id=1004, plan=) [2024-10-16 14:57:55.511943] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=10] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1729061399901106000, snapshot_type=1, create_time=1729061399903561, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:55.511950] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1729061562837930000, snapshot_type=1, create_time=1729061562816991, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:55.511957] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1729061454608479003, snapshot_type=2, create_time=1729061455418847, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:55.511963] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1729061399238808003, snapshot_type=2, create_time=1729061399903561, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:55.511971] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1729061562203853004, snapshot_type=2, create_time=1729061562816991, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:55.511977] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1729061455108817000, snapshot_type=3, create_time=1729061455418847, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:55.511983] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1729061399750481000, snapshot_type=3, create_time=1729061399903561, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:55.511988] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1729061562382937000, snapshot_type=3, create_time=1729061562816991, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:55.511994] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061455418847, addr="10.0.4.7:2890", gc_status=0) [2024-10-16 14:57:55.511999] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061399903561, addr="10.0.4.8:2890", gc_status=0) [2024-10-16 14:57:55.512005] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:707) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1729061562816991, addr="10.0.4.9:2890", gc_status=0) [2024-10-16 14:57:55.512112] INFO [MVCC] decide_reserved_snapshot_version_ (ob_multi_version_garbage_collector.cpp:618) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=4] succeed to update global reserved snapshot(ret=0, this={this:0x2b0e401fcf30, last_study_timestamp:1729061455421203, last_refresh_timestamp:1729061815496037, last_reclaim_timestamp:1729061455422915, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061399238808003, v:0}, is_inited:true}, global_reserved_snapshot_={val:1729061399238808003, v:0}, reserved_snapshot={val:1729061399238808003, v:0}) [2024-10-16 14:57:55.512123] INFO [MVCC] refresh_ (ob_multi_version_garbage_collector.cpp:552) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=10] multi-version garbage collector refresh successfully(this={this:0x2b0e401fcf30, last_study_timestamp:1729061455421203, last_refresh_timestamp:1729061815496037, last_reclaim_timestamp:1729061455422915, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061399238808003, v:0}, is_inited:true}, collector={reserved_snapshot_version:{val:1729061399238808003, v:0}, reserved_snapshot_type:2, reserved_status:-1, reserved_create_time:1729061399903561, reserved_addr:"10.0.4.8:2890", is_this_server_disabled:false, status:0}) [2024-10-16 14:57:55.512132] INFO [MVCC] repeat_refresh (ob_multi_version_garbage_collector.cpp:231) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=9] repeat refresh successfully(ret=0, this={this:0x2b0e401fcf30, last_study_timestamp:1729061455421203, last_refresh_timestamp:1729061875512132, last_reclaim_timestamp:1729061455422915, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061399238808003, v:0}, is_inited:true}, current_timestamp=1729061875511283, GARBAGE_COLLECT_RETRY_INTERVAL=60000000) [2024-10-16 14:57:55.512138] INFO [MVCC] repeat_reclaim (ob_multi_version_garbage_collector.cpp:266) [22820][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6] skip repeat reclaim(ret=0, this={this:0x2b0e401fcf30, last_study_timestamp:1729061455421203, last_refresh_timestamp:1729061875512132, last_reclaim_timestamp:1729061455422915, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1729061399238808003, v:0}, is_inited:true}, current_timestamp=1729061875512138, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2024-10-16 14:57:55.512563] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:57:55.512624] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=7] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:57:55.512630] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=5] tenant dfc status(ret=0, get_tenant_id()=1004, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:57:55.512643] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:108) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] refresh tenant units(sys_unit_cnt=3, units=[{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, {tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, {tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}], ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:55.512671] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=25] finish set nway(tenant_id_=1, nway=3) [2024-10-16 14:57:55.512676] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=4] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1, nway=3, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}) [2024-10-16 14:57:55.512685] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=1.638399963378906250e+02, this=0x2b0ddc2dc170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=819, Throttle Trigger(MB)=491, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.948457122979588263e+01) [2024-10-16 14:57:55.512696] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=10] succ to set_memstore_threshold of memstore allocator(tenant_id=1, ret=0) [2024-10-16 14:57:55.512701] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=3] finish set nway(tenant_id_=1002, nway=1) [2024-10-16 14:57:55.512704] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1002, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:55.512711] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e0c298170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:55.512717] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=7] succ to set_memstore_threshold of memstore allocator(tenant_id=1002, ret=0) [2024-10-16 14:57:55.512721] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=2] finish set nway(tenant_id_=1001, nway=1) [2024-10-16 14:57:55.512724] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1001, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:57:55.512731] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0df7098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:55.512737] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] succ to set_memstore_threshold of memstore allocator(tenant_id=1001, ret=0) [2024-10-16 14:57:55.512741] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=3] finish set nway(tenant_id_=1004, nway=1) [2024-10-16 14:57:55.512743] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1004, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:55.512749] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e40098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:55.512755] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=5] succ to set_memstore_threshold of memstore allocator(tenant_id=1004, ret=0) [2024-10-16 14:57:55.512759] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=2] finish set nway(tenant_id_=1003, nway=1) [2024-10-16 14:57:55.512761] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=3] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1003, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:57:55.512768] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e2c698170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:57:55.512774] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=5] succ to set_memstore_threshold of memstore allocator(tenant_id=1003, ret=0) [2024-10-16 14:57:55.513330] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:55.513340] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=10] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1001, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:55.513345] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=4] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:55.513348] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=4] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1003, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:55.513352] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:57:55.513368] INFO [SERVER] cal_all_part_disk_default_percentage (ob_server_utils.cpp:301) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=3] cal_all_part_disk_default_percentage succ(data_dir="/mnt/myoceanbase/oceanbase/data//sstable", clog_dir="/mnt/myoceanbase/oceanbase/data//clog", shared_mode=true, data_disk_total_size=214429704192, data_disk_default_percentage=60, clog_disk_total_size=214429704192, clog_disk_default_percentage=30) [2024-10-16 14:57:55.513374] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:337) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=6] decide disk size finished(suggested_disk_size=32212254720, suggested_disk_percentage=0, default_disk_percentage=30, total_space=214429704192, disk_size=32212254720) [2024-10-16 14:57:55.513378] INFO [SERVER] get_log_disk_info_in_config (ob_server_utils.cpp:88) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=4] get_log_disk_info_in_config(suggested_data_disk_size=32212254720, suggested_clog_disk_size=32212254720, suggested_data_disk_percentage=0, suggested_clog_disk_percentage=0, log_disk_size=32212254720, log_disk_percentage=0, total_log_disk_size=214429704192) [2024-10-16 14:57:55.513384] INFO [CLOG] try_resize (ob_server_log_block_mgr.cpp:800) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=4] try_resize success(ret=0, log_disk_size=32212254720, total_log_disk_size=214429704192, this={dir::"/mnt/myoceanbase/oceanbase/data//clog/log_pool", dir_fd:15, meta_fd:16, log_pool_meta:{curr_total_size:32212254720, next_total_size:32212254720, status:0}, min_block_id:15037, max_block_id:15335, min_log_disk_size_for_all_tenants_:15032385536, is_inited:true}) [2024-10-16 14:57:55.513391] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:132) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0D-0-0] [lt=7] refresh tenant config(tenants=[1, 1001, 1002, 1003, 1004], ret=0) [2024-10-16 14:57:55.518607] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=15][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:55.519486] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061875518505000, start_id=1729061875518505000, end_id=1729061875518505001) [2024-10-16 14:57:55.520179] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get_max_applied_scn(scn={val:1729061875319115000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390179227}, palf_committed_end_scn_:{val:1729061875518505000, v:0}, last_check_scn_:{val:1729061875518505000, v:0}, max_applied_cb_scn_:{val:1729061875319115000, v:0}}) [2024-10-16 14:57:55.520199] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:57:55.521732] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=38] get number(ret=0, service_type_=0, range=1, base_id=1729061875521704000, start_id=1729061875521704000, end_id=1729061875521704001) [2024-10-16 14:57:55.522383] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061875158504001, v:0}, new_max_version={val:1729061875158504001, v:0}, last_min_version={val:1729061874653969004, v:0}, last_max_version={val:1729061874653969004, v:0}, total_time_us=1413, delta_us=363878) [2024-10-16 14:57:55.522407] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061875158504001, v:0}, min_version={val:1729061875158504001, v:0}, max_version={val:1729061875158504001, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:57:55.530217] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1210, avg_group_log_size=121) [2024-10-16 14:57:55.530234] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1210) [2024-10-16 14:57:55.530529] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:55.544484] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:55.544506] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:55.544551] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:55.544565] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1342) [2024-10-16 14:57:55.544920] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:55.544968] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1646) [2024-10-16 14:57:55.545106] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F42A4-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:55.545115] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F42A4-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=539, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:55.547493] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=57] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061875060994002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=486492, in_cluster_service=true, cluster_version={val:1729061874653969005, v:0}, min_cluster_version={val:1729061874653969005, v:0}, max_cluster_version={val:1729061874653969005, v:0}, get_cluster_version_err=0, cluster_version_delta=893517, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228372, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=420939007, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:55.550780] INFO [STORAGE] try_shrink (ob_compaction_memory_pool.cpp:522) [22090][T1002_MergeMemP][T1002][Y0-0000000000000000-0-0] [lt=14] Compaction Mem Pool current stat: (ret=0, max_block_num=16, total_block_num=8, Chunk list size: =1) [2024-10-16 14:57:55.553816] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:55.561120] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061875560687000, start_id=1729061875560687000, end_id=1729061875560687001) [2024-10-16 14:57:55.561258] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061875560687000, start_id=1729061875560687000, end_id=1729061875560687001) [2024-10-16 14:57:55.561539] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061875561530], gts:1729061875560687000, latest_srr:[mts=1729061875561530]}) [2024-10-16 14:57:55.562248] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061875561774000, start_id=1729061875561774000, end_id=1729061875561774001) [2024-10-16 14:57:55.562749] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10711) [2024-10-16 14:57:55.568860] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.568945] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.569059] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.569526] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.576990] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.582558] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC4C-0-0] [lt=9] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:55.582584] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC4C-0-0] [lt=22] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:55.582610] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC4C-0-0] [lt=24] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:55.583826] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D65-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:55.583841] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D65-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20276, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:55.583845] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D65-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178595, log_task={header:{begin_lsn:{lsn:14538505039}, end_lsn:{lsn:14538505161}, log_id:115178595, min_scn:{val:1729061875158504003, v:0}, max_scn:{val:1729061875158504003, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538504917}, prev_proposal_id:23, committed_end_lsn:{lsn:14538505039}, data_checksum:1715074492, accum_checksum:3784780360, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061875483721, freeze_ts:1729061875483721, submit_ts:1729061875483741, flushed_ts:1729061875484588, gen_to_freeze cost time:0, gen_to_submit cost time:20, submit_to_flush cost time:847}, fs_cb_begin_ts=1729061875583819, log_life_time=100098) [2024-10-16 14:57:55.585259] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.596622] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:55.596772] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850B7-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:55.596773] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:55.596781] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850B7-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20421, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:55.596785] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:55.597300] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1003}, {id:1}, {id:1002}]) [2024-10-16 14:57:55.597395] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=10] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:57:55.603117] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4811-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061875570863, keepalive_ts:-1}) [2024-10-16 14:57:55.603134] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4811-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061875570863, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:55.610547] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=5640, mem_used=10398720) [2024-10-16 14:57:55.621023] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061875158504002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=462515, in_cluster_service=true, cluster_version={val:1729061875158504001, v:0}, min_cluster_version={val:1729061875158504001, v:0}, max_cluster_version={val:1729061875158504001, v:0}, get_cluster_version_err=0, cluster_version_delta=462515, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244293, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=546950858, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:55.644147] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:55.646237] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=42] success to update min reserved snapshot(reserved_snapshot=1729060070205428000, duration=1800, snapshot_gc_ts=1729061870205428000) [2024-10-16 14:57:55.646736] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=15] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060070205428000) [2024-10-16 14:57:55.647266] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=16] submit reserved snapshot log success(ls_id={id:1001}, new_snapshot_version=1729060070205428000) [2024-10-16 14:57:55.647278] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=10] update reserved snapshot finished(cost_ts=1044, reserved_snapshot=1729060070205428000) [2024-10-16 14:57:55.648648] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061875647790000, start_id=1729061875647790000, end_id=1729061875647790001) [2024-10-16 14:57:55.648728] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10755, avg_group_log_size=597) [2024-10-16 14:57:55.648765] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10755) [2024-10-16 14:57:55.649027] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=217, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:55.649042] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10755) [2024-10-16 14:57:55.649052] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=96, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:55.649068] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:55.649292] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD227-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:55.649306] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD227-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=637, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:55.649311] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD227-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=409, avg_log_submit_to_slide_time=606) [2024-10-16 14:57:55.655255] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA56-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061875655181000, start_id=1729061875655181000, end_id=1729061875655181001) [2024-10-16 14:57:55.655401] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA5A-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061875655181000, start_id=1729061875655181000, end_id=1729061875655181001) [2024-10-16 14:57:55.655441] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0CE4-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061875655181000, start_id=1729061875655181000, end_id=1729061875655181001) [2024-10-16 14:57:55.655542] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA58-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061875655181000, start_id=1729061875655181000, end_id=1729061875655181001) [2024-10-16 14:57:55.655572] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA57-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061875655181000, start_id=1729061875655181000, end_id=1729061875655181001) [2024-10-16 14:57:55.656517] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=682, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:55.656533] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:55.656562] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:55.672392] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:55.677005] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.681168] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D1-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:55.681193] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D1-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061875655181000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061875655181000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:55.682872] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=10] query dest_info_ succ(dest_info=[[0, "file:///mnt/oceanbasedata/bddb/archive"]], archive_dest_changed=false) [2024-10-16 14:57:55.682897] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=23] finish to resize log external storage handler(current_ts=1729061875682895, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:55.683967] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D67-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=401, avg_log_submit_to_slide_time=30242) [2024-10-16 14:57:55.683998] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D67-0-0] [lt=30][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178596, log_task={header:{begin_lsn:{lsn:14538505161}, end_lsn:{lsn:14538505283}, log_id:115178596, min_scn:{val:1729061875158504004, v:0}, max_scn:{val:1729061875158504004, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538505039}, prev_proposal_id:23, committed_end_lsn:{lsn:14538505161}, data_checksum:1715074492, accum_checksum:2423781388, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061875583806, freeze_ts:1729061875583806, submit_ts:1729061875583810, flushed_ts:1729061875585027, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1217}, fs_cb_begin_ts=1729061875683959, log_life_time=100153) [2024-10-16 14:57:55.684173] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:55.684708] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=24] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:57:55.684723] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] try gc retain ctx [2024-10-16 14:57:55.684728] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:55.685437] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061887008855000, latest_log_ts:{val:1729061870498146001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:55.685545] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=63] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=22, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:55.685565] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=4638, avg_group_log_size=386) [2024-10-16 14:57:55.685574] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:55.685671] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.685861] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22494][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D1-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061875685158000, start_id=1729061875685158000, end_id=1729061875685158001) [2024-10-16 14:57:55.686013] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=215, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:55.686027] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=101, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:55.686152] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=366, avg_log_submit_to_slide_time=583) [2024-10-16 14:57:55.686171] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=16, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:55.686503] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:55.699774] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:55.717530] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15E8-0-0] [lt=9] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:55.719655] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=39] get number(ret=0, service_type_=0, range=1, base_id=1729061875718837000, start_id=1729061875718837000, end_id=1729061875718837001) [2024-10-16 14:57:55.722106] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061875720936000, start_id=1729061875720936000, end_id=1729061875720936001) [2024-10-16 14:57:55.727405] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:55.730311] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:55.730361] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=47] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:57:55.730868] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:55.734147] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5E-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:55.734176] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C5E-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061875655181001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061875655181001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:55.736641] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:55.736660] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=217) [2024-10-16 14:57:55.738165] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C5E-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061875737436000, start_id=1729061875737436000, end_id=1729061875737436001) [2024-10-16 14:57:55.753858] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:55.757053] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B923E-0-0] [lt=140] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:55.758960] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:55.758986] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=14] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:55.759003] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=17] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:55.761332] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061875760676000, start_id=1729061875760676000, end_id=1729061875760676001) [2024-10-16 14:57:55.762542] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061875760676000, start_id=1729061875760676000, end_id=1729061875760676001) [2024-10-16 14:57:55.763482] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get_max_applied_scn(scn={val:1729061875647790000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264880213}, palf_committed_end_scn_:{val:1729061875760676000, v:0}, last_check_scn_:{val:1729061875760676000, v:0}, max_applied_cb_scn_:{val:1729061875647790000, v:0}}) [2024-10-16 14:57:55.763530] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:57:55.772528] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:55.777070] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.779875] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:111799518, max_lsn:17493490905, max_scn:{val:1729061875560687001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7667642653}, reuse_lsn:{lsn:17493490905}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17451547865}}, last_submit_lsn:{lsn:17493490783}, last_submit_end_lsn:{lsn:17493490905}, last_submit_log_id:111799518, last_submit_log_pid:17, max_flushed_lsn:{lsn:17493490783}, max_flushed_end_lsn:{lsn:17493490905}, max_flushed_log_pid:17, committed_end_lsn:{lsn:17493490905}, last_slide_log_id:111799518, last_slide_scn:{val:1729061875560687001, v:0}, last_slide_lsn:{lsn:17493490783}, last_slide_end_lsn:{lsn:17493490905}, last_slide_log_pid:17, last_slide_log_accum_checksum:3253153966, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431d4530}, [StateMgr]={this:0x2b0e431d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:111799519, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261853177, palf_role_change_cb:0x2b0e431df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:18}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:18}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494261753074, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494261753074, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262053331, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728249494772393002, v:0}, min_block_id:241, min_block_min_scn:{val:1728207118648006001, v:0}, base_lsn_for_block_gc:{lsn:17380134912}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:17380134912}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:757760}, readable_log_tail:{lsn:757760}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17514344448}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/meta", dir_fd:193, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66347008, block_header_serialize_buf_:0x2b0e431da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17493490905}, readable_log_tail:{lsn:17493490905}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17447239680}, min_scn:{val:1729032503563174001, v:0}, curr_block_id:260, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/log", dir_fd:195, min_block_id:241, max_block_id:261, curr_writable_block_id:260}, logical_block_size_:67104768, curr_block_writable_size_:20853543, block_header_serialize_buf_:0x2b0e431dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261852794, this:0x2b0e431d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:7668488755}, saved_end_lsn:{lsn:7668488755}, last_submit_prepare_req_time_us:1722494261853360, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431d89b0}) [2024-10-16 14:57:55.780027] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=148] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061875730946, last_flushed_end_lsn:{lsn:17493490905}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061875730901, last_flushed_end_lsn:{lsn:17493490905}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061875731148, last_flushed_end_lsn:{lsn:17493490905}}]) [2024-10-16 14:57:55.780038] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=111799519, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061875730946, last_flushed_end_lsn:{lsn:17493490905}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061875730901, last_flushed_end_lsn:{lsn:17493490905}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061875731148, last_flushed_end_lsn:{lsn:17493490905}}]) [2024-10-16 14:57:55.782412] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=18] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:55.783971] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D69-0-0] [lt=20][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178597, log_task={header:{begin_lsn:{lsn:14538505283}, end_lsn:{lsn:14538505405}, log_id:115178597, min_scn:{val:1729061875655181001, v:0}, max_scn:{val:1729061875655181001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538505161}, prev_proposal_id:23, committed_end_lsn:{lsn:14538505283}, data_checksum:1715074492, accum_checksum:1692152029, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061875683914, freeze_ts:1729061875683914, submit_ts:1729061875683928, flushed_ts:1729061875685120, gen_to_freeze cost time:0, gen_to_submit cost time:14, submit_to_flush cost time:1192}, fs_cb_begin_ts=1729061875783963, log_life_time=100049) [2024-10-16 14:57:55.785744] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.788178] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.788196] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.788200] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.788458] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.788543] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.788977] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.789193] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.789399] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.789423] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.789532] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.789556] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:55.797341] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:55.813770] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A11-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061875804955, keepalive_ts:-1}) [2024-10-16 14:57:55.813793] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A11-0-0] [lt=23] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061875804955, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:55.829184] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:57:55.829201] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:55.829215] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52565077, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:55.830926] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=48, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:57:55.830939] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=949, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:57:55.830943] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:57:55.830947] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=-921, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:57:55.830950] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=76, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:57:55.831499] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1003}, {id:1}, {id:1001}]) [2024-10-16 14:57:55.844423] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:55.844895] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=239, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:55.844912] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=68, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:55.844951] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=43] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:55.845199] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F42AF-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=327, avg_log_submit_to_slide_time=536) [2024-10-16 14:57:55.848546] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061875847665000, start_id=1729061875847665000, end_id=1729061875847665001) [2024-10-16 14:57:55.856390] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EEC-0-0] [lt=25] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:55.856845] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9240-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:55.856857] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9240-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10702, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:55.856862] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9240-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=714, avg_log_submit_to_slide_time=10696) [2024-10-16 14:57:55.856894] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:55.861227] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5748, mem_used=12478464) [2024-10-16 14:57:55.877130] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.881633] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=15] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:57:55.884434] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:55.884453] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:55.885043] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3536) [2024-10-16 14:57:55.885081] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3536) [2024-10-16 14:57:55.885210] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10B5-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:55.885221] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10B5-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=577, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:55.885771] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.890306] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:57:55.890482] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4A-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061875560687002, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061873294294000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890098662000, latest_log_ts:{val:1729061873558226001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:55.890621] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4A-0-0] [lt=85] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061875560687002, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061873294294000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:55.890718] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4A-0-0] [lt=33] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061875655181003, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061873294294000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:55.890795] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4A-0-0] [lt=36] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061875655181001, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061873294294000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:55.890825] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4A-0-0] [lt=30] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:55.899506] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=16] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:55.907142] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=5] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:55.907717] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:55.907758] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=8] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061867249743000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:55.907797] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=38] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061867249743000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1003, freeze_info:{latest_snapshot_gc_scn:{val:1729061867249743000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}}) [2024-10-16 14:57:55.920308] INFO [SERVER] refresh_avail_memory (ob_table_load_assigned_memory_manager.cpp:117) [22141][T1002_TntShared][T1002][Y0-0000000000000000-0-0] [lt=16] ObTableLoadAssignedMemoryManager::refresh_avail_memory(MTL_ID()=1002, avail_memory=53687091, chunk_count_=0, avail_sort_memory_=53687091, avail_memory_=53687091) [2024-10-16 14:57:55.920337] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061875918689000, start_id=1729061875918689000, end_id=1729061875918689001) [2024-10-16 14:57:55.922921] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=15] block manager free block(macro_id=[3934](ver=0,mode=0,seq=7663656), io_fd={first_id:7663656, second_id:3934, device_handle:null}) [2024-10-16 14:57:55.922941] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=17] block manager free block(macro_id=[1201](ver=0,mode=0,seq=7663640), io_fd={first_id:7663640, second_id:1201, device_handle:null}) [2024-10-16 14:57:55.922946] INFO [STORAGE.BLKMGR] sweep_one_block (ob_block_manager.cpp:971) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] block manager free block(macro_id=[11594](ver=0,mode=0,seq=7663754), io_fd={first_id:7663754, second_id:11594, device_handle:null}) [2024-10-16 14:57:55.923519] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061875921613000, start_id=1729061875921613000, end_id=1729061875921613001) [2024-10-16 14:57:55.924346] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:55.924799] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:55.925079] INFO [STORAGE.BLKMGR] mark_and_sweep (ob_block_manager.cpp:1053) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=3] finish once mark and sweep(ret=0, alloc_num=3982211, mark_info.count()=3, marker_status={total_block_count:15360, reserved_block_count:2, linked_block_count:17, tmp_file_count:0, data_block_count:7208, shared_data_block_count:161, index_block_count:14, ids_block_count:0, disk_block_count:0, bloomfiter_count:0, hold_count:0, pending_free_count:3, free_count:7863, shared_meta_block_count:95, mark_cost_time:22539, sweep_cost_time:39, start_time:"2024-10-16 14:57:55.900377", last_end_time:"2024-10-16 14:57:55.922955", hold_info:nothing}, map_cnt=7495) [2024-10-16 14:57:55.925437] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=26] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:55.930412] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:55.930454] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=37] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:55.930655] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:55.931118] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3BB-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:55.931139] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3BB-0-0] [lt=20] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=584, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:55.931145] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3BB-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=311, avg_log_submit_to_slide_time=565) [2024-10-16 14:57:55.941772] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7B-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:55.941806] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7B-0-0] [lt=20] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061875655181003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061875655181003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:55.942820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D7B-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061875941883000, start_id=1729061875941883000, end_id=1729061875941883001) [2024-10-16 14:57:55.944815] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3726, mem_used=8318976) [2024-10-16 14:57:55.945419] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7B-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061875945112000, start_id=1729061875945112000, end_id=1729061875945112001) [2024-10-16 14:57:55.945509] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7B-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=27, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:55.945535] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7B-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:55.949789] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:55.949811] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=16] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:55.949825] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=12] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=152) [2024-10-16 14:57:55.953950] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:55.961329] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061875959894000, start_id=1729061875959894000, end_id=1729061875959894001) [2024-10-16 14:57:55.963368] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=50] get number(ret=0, service_type_=0, range=1, base_id=1729061875959894000, start_id=1729061875959894000, end_id=1729061875959894001) [2024-10-16 14:57:55.972264] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=622816, this={dio_aligned_buf:{buf_write_offset:346, buf_padding_size:3750, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111160311, truncate_used_ts:41410978}, log_block_size:67108864, dir_fd:177, io_fd:142}, aligned_buf_len=4096, aligned_block_offset=622592, buf_len=122, total_write_size_=13488, total_write_size_after_dio_=425984, ob_pwrite_used_ts=22280, count=101, each_pwrite_cost=220) [2024-10-16 14:57:55.977213] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.977800] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=5] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061871968938, last_first_register_time_us=1722494150998907, register_time_us=1729061871968938, curr_time_us=1729061875977798) [2024-10-16 14:57:55.977833] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061875977817, last_submit_register_req_time_us=1729061875977817, register_parent_reason=3) [2024-10-16 14:57:55.977842] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:55.985815] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:55.993226] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=16] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:57:55.998936] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:57:56.011781] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [22791][T1004_ObXAHbWor][T1004][Y0-0000000000000000-0-0] [lt=13] XA scheduler heartbeat task statistics(avg_time=1) [2024-10-16 14:57:56.030439] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061876030438, bytes: 4446820134152, bw: 0.395693 MB/s, add_ts: 1000557, add_bytes: 415145 [2024-10-16 14:57:56.030732] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=50] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=288, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:56.030771] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=55, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:56.030787] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:56.030891] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3BE-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:56.030898] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3BE-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=593, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:56.044744] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:56.044873] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:56.044885] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:56.046261] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4497-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:56.046275] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4497-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=663, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:56.048559] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061876047291000, start_id=1729061876047291000, end_id=1729061876047291001) [2024-10-16 14:57:56.056522] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EF0-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:56.056751] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:56.061773] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061876061556000, start_id=1729061876061556000, end_id=1729061876061556001) [2024-10-16 14:57:56.061795] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061876061768], gts:1729061876061556000, latest_srr:[mts=1729061876061768]}) [2024-10-16 14:57:56.061805] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:56.072139] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE788B-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938413, log_task={header:{begin_lsn:{lsn:213058257120}, end_lsn:{lsn:213058257242}, log_id:220938413, min_scn:{val:1729061875655181004, v:0}, max_scn:{val:1729061875655181004, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058256998}, prev_proposal_id:88, committed_end_lsn:{lsn:213058257120}, data_checksum:1715074492, accum_checksum:1703006574, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061875972020, freeze_ts:1729061875972020, submit_ts:1729061875972031, flushed_ts:1729061875972855, gen_to_freeze cost time:0, gen_to_submit cost time:11, submit_to_flush cost time:824}, fs_cb_begin_ts=1729061876072133, log_life_time=100113) [2024-10-16 14:57:56.072388] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:56.072714] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54CA-0-0] [lt=27] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1333) [2024-10-16 14:57:56.077205] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:56.077229] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=23] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:56.077239] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.084091] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D6F-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:56.084306] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:56.084952] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3536) [2024-10-16 14:57:56.085838] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.093959] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:56.094135] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D48-0-0] [lt=8] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061875560687004, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061873056699003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890721265000, latest_log_ts:{val:1729061874059797001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:56.094314] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D48-0-0] [lt=86] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061875560687004, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061873056699003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:56.094470] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=5] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:56.094452] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D48-0-0] [lt=59] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061875655181005, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061873056699003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:56.094590] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D48-0-0] [lt=58] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061875655181004, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061873056699003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:56.094655] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D48-0-0] [lt=63] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:57:56.095064] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:56.095117] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=19] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061870205428000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:56.119684] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] no need to update min/max version(new_version={val:1729061875718837000, v:0}, min_version={val:1729061875718837000, v:0}, max_version={val:1729061875718837000, v:0}) [2024-10-16 14:57:56.122819] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1002, min_version={val:1729061875560687003, v:0}, base_version={val:1729061875560687001, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061875560687003, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061876122805, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061875560687004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061876079154, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061875560687004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061876107210, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:56.123579] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22235][T1002_TenantWea][T1001][YB4A0A000407-00061E9B3B0B77DE-0-0] [lt=25] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.108497524261474609e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.002399163312093151e+03) [2024-10-16 14:57:56.123765] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=44] get number(ret=0, service_type_=0, range=1, base_id=1729061876123249000, start_id=1729061876123249000, end_id=1729061876123249001) [2024-10-16 14:57:56.127976] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:56.127989] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:56.130708] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=245, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:56.130732] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=29, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:56.137481] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3906, mem_used=8318976) [2024-10-16 14:57:56.139193] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [21543][T1_CdcSrv][T1][Y0-0000000000000000-0-0] [lt=11] finish to resize log external storage handler(current_ts=1729061876139190, tenant_max_cpu=3, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:57:56.139370] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1, io_status_sys_log=sys_group_name: SHARED_BLOCK_ReaderWriter_IO, mode: read, size: 40960.00, iops: 1.00, rt: 108.00; ) [2024-10-16 14:57:56.139389] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS CONFIG](tenant_id=1, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:3}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:56.139424] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 145.50; ) [2024-10-16 14:57:56.139433] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:56.139462] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 162.50; sys_group_name: SHARED_BLOCK_ReaderWriter_IO, mode: read, size: 8192.00, iops: 1.00, rt: 150.00; ) [2024-10-16 14:57:56.139471] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:56.139482] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:56.144873] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:56.144887] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:57:56.145108] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:56.145512] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F42B7-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=375, avg_log_submit_to_slide_time=552) [2024-10-16 14:57:56.149589] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:57:56.155895] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15E6-0-0] [lt=16] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:56.155928] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:56.156271] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA5E-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061876156201000, start_id=1729061876156201000, end_id=1729061876156201001) [2024-10-16 14:57:56.156304] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0CE6-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061876156201000, start_id=1729061876156201000, end_id=1729061876156201001) [2024-10-16 14:57:56.156349] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA5C-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061876156201000, start_id=1729061876156201000, end_id=1729061876156201001) [2024-10-16 14:57:56.158839] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15E2-0-0] [lt=13] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:56.159143] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0CE8-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061876158379000, start_id=1729061876158379000, end_id=1729061876158379001) [2024-10-16 14:57:56.159293] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0CEA-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061876158379000, start_id=1729061876158379000, end_id=1729061876158379001) [2024-10-16 14:57:56.159359] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=10] current monitor number(seq_=-1) [2024-10-16 14:57:56.161404] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061876158379000, start_id=1729061876158379000, end_id=1729061876158379001) [2024-10-16 14:57:56.161459] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:56.161472] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:56.166337] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061876165639000, start_id=1729061876165639000, end_id=1729061876165639001) [2024-10-16 14:57:56.170315] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=4] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:56.170333] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:56.170340] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:56.170343] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:56.170347] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:56.170354] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=5] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16728) [2024-10-16 14:57:56.171983] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061876171982, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007618, add_bytes: 0 [2024-10-16 14:57:56.177243] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.182152] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC8A-0-0] [lt=7] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:56.182172] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC8A-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:56.182192] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC8A-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:56.185901] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.197960] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503A-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:56.197979] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503A-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:56.198241] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:56.220654] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061876218687000, start_id=1729061876218687000, end_id=1729061876218687001) [2024-10-16 14:57:56.230528] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:56.230550] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:56.230849] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:56.231039] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3C5-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=368, avg_log_submit_to_slide_time=564) [2024-10-16 14:57:56.238047] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:57:56.238064] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:57:56.238070] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:57:56.238075] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:56.240212] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=86, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}, {obj_id:78893917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:78893918, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893919, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893920, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}]) [2024-10-16 14:57:56.240330] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=116] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:56.245040] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:56.248797] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061876248119000, start_id=1729061876248119000, end_id=1729061876248119001) [2024-10-16 14:57:56.250842] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=10] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:57:56.251639] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061876250235000, start_id=1729061876250235000, end_id=1729061876250235001) [2024-10-16 14:57:56.256649] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53EF4-0-0] [lt=21][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173296, log_task={header:{begin_lsn:{lsn:14352392250}, end_lsn:{lsn:14352392372}, log_id:115173296, min_scn:{val:1729061875655181005, v:0}, max_scn:{val:1729061875655181005, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352392128}, prev_proposal_id:17, committed_end_lsn:{lsn:14352392250}, data_checksum:940009244, accum_checksum:2013401601, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061876156564, freeze_ts:1729061876156564, submit_ts:1729061876156569, flushed_ts:1729061876159705, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:3136}, fs_cb_begin_ts=1729061876256643, log_life_time=100079) [2024-10-16 14:57:56.262775] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DB7-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510344131169, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410622, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:56.269918] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88E-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:56.269947] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88E-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061875655181004, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061875655181004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:56.271280] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA88E-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061876270360000, start_id=1729061876270360000, end_id=1729061876270360001) [2024-10-16 14:57:56.272409] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:56.274797] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88E-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061876274592000, start_id=1729061876274592000, end_id=1729061876274592001) [2024-10-16 14:57:56.274863] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88E-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=9759, avg_group_log_size=542) [2024-10-16 14:57:56.274880] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88E-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9759) [2024-10-16 14:57:56.275089] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=330, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:56.275103] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9759) [2024-10-16 14:57:56.275118] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=173, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:56.275128] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9759) [2024-10-16 14:57:56.275138] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=12, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:56.275213] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [22253][T1002_Compactio][T1002][YB4A0A000407-00061E99741D0D6F-0-0] [lt=11] InfoPoolResizeTask(cost_ts=5) [2024-10-16 14:57:56.275290] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5931-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:56.275297] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5931-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=1170, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:56.275302] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5931-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=588, avg_log_submit_to_slide_time=1139) [2024-10-16 14:57:56.275338] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:56.277288] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.285373] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=389, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:56.285400] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=138, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:56.285443] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:56.285961] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.297235] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D9-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061876288396, keepalive_ts:-1}) [2024-10-16 14:57:56.297256] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79D9-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061876288396, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:56.297552] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850C5-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:56.297573] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850C5-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:56.297585] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850C5-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:56.297609] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850C5-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497562, log_task={header:{begin_lsn:{lsn:207589565222}, end_lsn:{lsn:207589565344}, log_id:218497562, min_scn:{val:1729061876158379001, v:0}, max_scn:{val:1729061876158379001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589565100}, prev_proposal_id:81, committed_end_lsn:{lsn:207589565222}, data_checksum:940009244, accum_checksum:3231982442, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061876197501, freeze_ts:1729061876197501, submit_ts:1729061876197506, flushed_ts:1729061876198266, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:760}, fs_cb_begin_ts=1729061876297592, log_life_time=100091) [2024-10-16 14:57:56.307805] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=17] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:56.312558] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:57:56.312697] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAD-0-0] [lt=14] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061889029444000, latest_log_ts:{val:1729061872445301001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:56.312774] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAD-0-0] [lt=72] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:56.312792] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:56.312806] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=7] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:56.312825] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373687541, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:56.316401] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1132-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061876316069000, start_id=1729061876316069000, end_id=1729061876316069001) [2024-10-16 14:57:56.323822] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061876323360000, start_id=1729061876323360000, end_id=1729061876323360001) [2024-10-16 14:57:56.328672] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=388, mem_used=2079744) [2024-10-16 14:57:56.330854] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:56.333000] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:56.334799] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21656][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3DFF-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510344203193, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410623, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:56.335945] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:56.335957] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28020300, tenant_id=1002) [2024-10-16 14:57:56.335973] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:56.345244] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=215, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:56.345268] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=111, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:56.345307] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:56.345649] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C449F-0-0] [lt=17] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1454) [2024-10-16 14:57:56.348020] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] [WRS] update tenant weak read server version(tenant_id=1004, server_version={version:{val:1729061876061556001, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3, epoch_tstamp:1729061876347974}, version_delta=286463) [2024-10-16 14:57:56.356002] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:56.363961] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E08-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061876362200000, start_id=1729061876362200000, end_id=1729061876362200001) [2024-10-16 14:57:56.364083] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061876362200000, start_id=1729061876362200001, end_id=1729061876362200002) [2024-10-16 14:57:56.364788] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:56.364801] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54641860, tenant_id=1003) [2024-10-16 14:57:56.364811] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:56.365581] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1003, min_version={val:1729061876165639000, v:0}, base_version={val:1729061875959894000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061876165639000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061876365571, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061876165639000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061876272502, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061876165639000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061876324574, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:57:56.366434] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061876365521000, start_id=1729061876365521000, end_id=1729061876365521001) [2024-10-16 14:57:56.372620] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7891-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:56.377350] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.384226] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D75-0-0] [lt=25] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:56.386669] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.395266] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:57:56.404553] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:56.409109] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=16] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:57:56.413218] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601462, offset=9642272}, end_cursor:ObLogCursor{file_id=192, log_id=47601463, offset=9642556}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db728a050, len:3808, log_data_len:284, seq:47601462, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:56.415138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B2-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061876414206000, start_id=1729061876414206000, end_id=1729061876414206001) [2024-10-16 14:57:56.416119] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601463, offset=9642556}, end_cursor:ObLogCursor{file_id=192, log_id=47601464, offset=9642830}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2074050, len:3524, log_data_len:274, seq:47601463, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:56.416524] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22067][T1002_TeRec][T1002][YB4A0A000407-00061E9B225FA4B2-0-0] [lt=13] get_max_applied_scn(scn={val:1729061876316069000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023118472}, palf_committed_end_scn_:{val:1729061876316069000, v:0}, last_check_scn_:{val:1729061876316069000, v:0}, max_applied_cb_scn_:{val:1729061876316069000, v:0}}) [2024-10-16 14:57:56.416538] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [96395][T1002_L0_G2][T1002][YB4A0A000408-00061E9B7FEC8831-0-0] [lt=50] get_min_unreplayed_log_info(lsn={lsn:213058257730}, scn={val:1729061876158379003, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061876373083, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058257730}, next_to_submit_scn_:{val:1729061876158379003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058257608}, end_lsn:{lsn:213058257730}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061876158379002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061876158379002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:598172837, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061876158379002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561424, miss_cnt:1, cache_read_size:152590697712, hit_ratio:"9.999999928346963163e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:57:56.416542] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22067][T1002_TeRec][T1002][YB4A0A000407-00061E9B225FA4B2-0-0] [lt=15] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:56.418196] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601464, offset=9642830}, end_cursor:ObLogCursor{file_id=192, log_id=47601465, offset=9643104}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db731a050, len:3250, log_data_len:274, seq:47601464, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:56.418546] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601465, offset=9643104}, end_cursor:ObLogCursor{file_id=192, log_id=47601466, offset=9643378}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2cec050, len:2976, log_data_len:274, seq:47601465, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:56.418609] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599011, miss_cnt:9, cache_read_size:149777930029, hit_ratio:"9.999999345925574268e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:56.418637] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561424, miss_cnt:1, cache_read_size:152590697712, hit_ratio:"9.999999928346963163e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:56.420646] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061876419664000, start_id=1729061876419664000, end_id=1729061876419664001) [2024-10-16 14:57:56.422992] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get_max_applied_scn(scn={val:1729061876316069000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959719975}, palf_committed_end_scn_:{val:1729061876316069000, v:0}, last_check_scn_:{val:1729061876316069000, v:0}, max_applied_cb_scn_:{val:1729061876316069000, v:0}}) [2024-10-16 14:57:56.423013] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:56.423064] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=39] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:57:56.429480] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:56.445313] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:56.448967] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061876448562000, start_id=1729061876448562000, end_id=1729061876448562001) [2024-10-16 14:57:56.454140] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=45] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:57:56.454155] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=14] try gc retain ctx [2024-10-16 14:57:56.454158] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:56.454785] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061889029444000, latest_log_ts:{val:1729061872445301001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:56.469604] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=13] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:133533428, max_lsn:66390182176, max_scn:{val:1729061876419664000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:33216076378}, reuse_lsn:{lsn:66390182176}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:66348239136}}, last_submit_lsn:{lsn:66390181589}, last_submit_end_lsn:{lsn:66390182176}, last_submit_log_id:133533428, last_submit_log_pid:48, max_flushed_lsn:{lsn:66390181589}, max_flushed_end_lsn:{lsn:66390182176}, max_flushed_log_pid:48, committed_end_lsn:{lsn:66390182176}, last_slide_log_id:133533428, last_slide_scn:{val:1729061876419664000, v:0}, last_slide_lsn:{lsn:66390181589}, last_slide_end_lsn:{lsn:66390182176}, last_slide_log_pid:48, last_slide_log_accum_checksum:1506362191, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0de0acc530}, [StateMgr]={this:0x2b0de0ad0830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:133533429, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261740374, palf_role_change_cb:0x2b0de0ad7230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, reconfig_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, persistent_config_version:{proposal_id:48, config_seq:11}, ms_ack_list:0, resend_config_version:{proposal_id:48, config_seq:11}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494260838312, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494260838312, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0de0acdc70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261941355, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728741813401825001, v:0}, min_block_id:965, min_block_min_scn:{val:1728727703672931001, v:0}, base_lsn_for_block_gc:{lsn:66098196480}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, log_config_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, log_snapshot_meta:{version:1, base_lsn:{lsn:66098196480}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1634304}, readable_log_tail:{lsn:1634304}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66433720320}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/meta", dir_fd:128, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65470464, block_header_serialize_buf_:0x2b0de0ad29f8, flashback_version:0}, log_storage:{log_tail:{lsn:66390182176}, readable_log_tail:{lsn:66390182176}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66366615552}, min_scn:{val:1729057321384354001, v:0}, curr_block_id:989, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/log", dir_fd:134, min_block_id:965, max_block_id:990, curr_writable_block_id:989}, logical_block_size_:67104768, curr_block_writable_size_:43538144, block_header_serialize_buf_:0x2b0de0ad46b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261739251, this:0x2b0de0ad0f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:48, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:47, majority_max_lsn:{lsn:33216819320}, saved_end_lsn:{lsn:33216819320}, last_submit_prepare_req_time_us:1722494261740552, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0de0ad09b0}) [2024-10-16 14:57:56.469750] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=143] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061876420902, last_flushed_end_lsn:{lsn:66390182176}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061876421245, last_flushed_end_lsn:{lsn:66390182176}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061876421187, last_flushed_end_lsn:{lsn:66390182176}}]) [2024-10-16 14:57:56.469761] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=133533429, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061876420902, last_flushed_end_lsn:{lsn:66390182176}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061876421245, last_flushed_end_lsn:{lsn:66390182176}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061876421187, last_flushed_end_lsn:{lsn:66390182176}}]) [2024-10-16 14:57:56.472655] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7893-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:56.472922] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=209, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:56.472938] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:56.473181] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54D2-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:56.473188] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54D2-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10450, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:56.473631] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:56.473662] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=30] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:56.473672] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:56.477412] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.481585] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=37] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:57:56.481608] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:56.481618] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=9] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:57:56.481626] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=7] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:56.483694] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:57:56.483826] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=131] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:56.484839] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8E-0-0] [lt=9] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:56.484857] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8E-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:56.484895] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D77-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:56.486696] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.488504] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257810, offset=39046579}, end_cursor:ObLogCursor{file_id=197, log_id=49257811, offset=39046862}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dc4050, len:589, log_data_len:283, seq:49257810, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:56.490659] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB37D-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061876490274000, start_id=1729061876490274000, end_id=1729061876490274001) [2024-10-16 14:57:56.491244] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] [APPLY STAT CB APPEND COST TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=41, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:57:56.491254] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=1173, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:57:56.491259] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:57:56.491262] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=-579, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:57:56.491265] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=17, stat_interval=5000000, avg (count/cost)=635, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:57:56.491584] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257811, offset=39046862}, end_cursor:ObLogCursor{file_id=197, log_id=49257812, offset=39047144}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72fa050, len:4402, log_data_len:282, seq:49257811, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:56.493655] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257812, offset=39047144}, end_cursor:ObLogCursor{file_id=197, log_id=49257813, offset=39047426}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20b4050, len:4120, log_data_len:282, seq:49257812, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:56.493854] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257813, offset=39047426}, end_cursor:ObLogCursor{file_id=197, log_id=49257814, offset=39047708}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2118050, len:3838, log_data_len:282, seq:49257813, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:56.493902] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650361, miss_cnt:32, cache_read_size:8050435344, hit_ratio:"9.999994892290620108e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:56.493914] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657101, miss_cnt:17, cache_read_size:8050182959, hit_ratio:"9.999997286820628828e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:56.523877] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061876523074000, start_id=1729061876523074000, end_id=1729061876523074001) [2024-10-16 14:57:56.524634] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=25] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061876158379001, v:0}, new_max_version={val:1729061876158379001, v:0}, last_min_version={val:1729061875560687005, v:0}, last_max_version={val:1729061875560687005, v:0}, total_time_us=1616, delta_us=366254) [2024-10-16 14:57:56.524653] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061876158379001, v:0}, min_version={val:1729061876158379001, v:0}, max_version={val:1729061876158379001, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:57:56.530730] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:56.530746] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:56.530970] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:56.533967] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:56.533986] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:56.534003] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444604013, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:56.545064] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:56.545083] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:56.545105] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:56.545114] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:57:56.545493] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:56.545523] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:56.545704] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44A5-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:56.545724] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44A5-0-0] [lt=20] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=570, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:56.556080] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:56.562148] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061876562105000, start_id=1729061876562105000, end_id=1729061876562105001) [2024-10-16 14:57:56.562168] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=16] ts response task statistics(alloc_count=261365030, free_count=130682514) [2024-10-16 14:57:56.564222] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061876564219000, start_id=1729061876564219000, end_id=1729061876564219001) [2024-10-16 14:57:56.566541] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061876566337000, start_id=1729061876566337000, end_id=1729061876566337001) [2024-10-16 14:57:56.566831] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9565) [2024-10-16 14:57:56.568838] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:56.568839] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:56.569052] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:56.569466] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:56.571126] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22697][T1004_TntTransf][T1004][YB4A0A000407-00061E9ABD5C8938-0-0] [lt=12] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:57:56.577475] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.584339] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D79-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:56.584354] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D79-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=50132, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:56.586755] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.596813] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:56.596826] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:56.597384] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1}, {id:1002}]) [2024-10-16 14:57:56.597892] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=205, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:56.597908] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=69, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:56.597941] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:56.598148] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54D5-0-0] [lt=20] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:56.598155] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54D5-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18309, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:56.607610] INFO [COMMON] generate_mod_stat_task (memory_dump.cpp:218) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] task info(*task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:56.607627] INFO [COMMON] handle (memory_dump.cpp:550) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=16] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:57:56.608456] INFO [COMMON] update_check_range (ob_sql_mem_leak_checker.cpp:62) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] update_check_range(min_check_version=652899, max_check_version=652901, global_version=652902) [2024-10-16 14:57:56.610642] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5658, mem_used=10398720) [2024-10-16 14:57:56.620841] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061876619173000, start_id=1729061876619173000, end_id=1729061876619173001) [2024-10-16 14:57:56.636595] INFO handle (memory_dump.cpp:677) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=8] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 205 258 0 1 2 117 1 0 1 4 1 1 0 1 5 38 4 0 1 7 1 2 0 1 8 65 1 0 1 12 18 3 0 1 13 6 1 0 1 14 1 2 0 1 15 1 1 0 1 18 26 3 0 1 20 2 2 0 1 22 1 1 0 500 0 139 248 0 500 7 7 4 0 500 8 50 1 0 500 9 2 1 0 500 10 10 2 0 500 16 3 1 0 500 17 10 7 0 500 22 111 63 0 500 23 61 11 0 508 0 4 13 0 508 8 5 1 0 509 0 3 8 0 509 8 5 1 0 1001 0 131 232 0 1001 2 22 1 0 1001 4 1 1 0 1001 5 23 4 0 1001 8 44 1 0 1001 12 10 3 0 1001 13 3 1 0 1001 14 1 2 0 1001 16 1 1 0 1001 18 14 3 0 1001 20 1 2 0 1002 0 174 258 0 1002 2 21 1 0 1002 4 2 1 0 1002 5 24 4 0 1002 8 50 1 0 1002 12 19 3 0 1002 13 7 1 0 1002 14 1 2 0 1002 18 14 3 0 1002 20 7 2 0 1003 0 136 235 0 1003 2 29 1 0 1003 4 1 1 0 1003 5 23 4 0 1003 8 44 1 0 1003 12 10 3 0 1003 13 3 1 0 1003 14 1 2 0 1003 18 13 3 0 1003 20 1 2 0 1004 0 291 256 0 1004 2 19 1 0 1004 4 2 1 0 1004 5 34 4 0 1004 8 52 1 0 1004 12 19 3 0 1004 13 10 1 0 1004 14 2 2 0 1004 15 2 1 0 1004 16 1 2 0 1004 18 25 3 0 1004 20 11 2 0 cost_time: 28165 [2024-10-16 14:57:56.636633] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=19] [MEMORY] tenant: 1, limit: 2,147,483,648 hold: 1,984,401,408 rpc_hold: 0 cache_hold: 861,929,472 cache_used: 861,929,472 cache_item_count: 411 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 541,540,352 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MEMSTORE_CTX_ID hold_bytes= 245,366,784 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TRANS_CTX_MGR_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PLAN_CACHE_CTX_ID hold_bytes= 79,691,776 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 136,314,880 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= META_OBJ_CTX_ID hold_bytes= 37,748,736 limit= 429,496,720 [MEMORY] ctx_id= TX_CALLBACK_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOB_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PS_CACHE_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TX_DATA_TABLE hold_bytes= 54,546,432 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MDS_DATA_ID hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:56.636847] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= DEFAULT_CTX_ID hold= 541,540,352 used= 428,977,312 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 32 [MEMORY] hold= 56,180,736 used= 56,153,088 count= 27 avg_used= 2,079,744 block_cnt= 27 chunk_cnt= 27 mod=CACHE_MAP_NODE [MEMORY] hold= 41,963,520 used= 41,943,040 count= 1 avg_used= 41,943,040 block_cnt= 1 chunk_cnt= 1 mod=LogGroupBuffer [MEMORY] hold= 30,965,760 used= 27,525,120 count= 420 avg_used= 65,536 block_cnt= 420 chunk_cnt= 50 mod=MinorMergeMgr [MEMORY] hold= 23,511,040 used= 23,433,792 count= 8 avg_used= 2,929,224 block_cnt= 8 chunk_cnt= 7 mod=IoControl [MEMORY] hold= 22,880,256 used= 21,568,040 count= 161 avg_used= 133,962 block_cnt= 161 chunk_cnt= 38 mod=SqlPlanMon [MEMORY] hold= 18,874,368 used= 16,967,424 count= 256 avg_used= 66,279 block_cnt= 256 chunk_cnt= 63 mod=SqlDtlBuf [MEMORY] hold= 17,694,720 used= 16,942,080 count= 240 avg_used= 70,592 block_cnt= 240 chunk_cnt= 30 mod=[T]ObSessionDIB [MEMORY] hold= 16,797,696 used= 16,777,336 count= 1 avg_used= 16,777,336 block_cnt= 1 chunk_cnt= 1 mod=MrgMemPoolChk [MEMORY] hold= 15,034,848 used= 14,886,881 count= 98 avg_used= 151,906 block_cnt= 31 chunk_cnt= 7 mod=OmtTenant [MEMORY] hold= 11,771,904 used= 9,838,592 count= 236 avg_used= 41,688 block_cnt= 236 chunk_cnt= 61 mod=[T]char [MEMORY] hold= 10,911,744 used= 9,699,328 count= 148 avg_used= 65,536 block_cnt= 148 chunk_cnt= 38 mod=MajorMergeMgr [MEMORY] hold= 9,142,272 used= 9,129,920 count= 6 avg_used= 1,521,653 block_cnt= 6 chunk_cnt= 6 mod=MysqlRequesReco [MEMORY] hold= 8,777,728 used= 8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv [MEMORY] hold= 8,409,088 used= 8,388,640 count= 1 avg_used= 8,388,640 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 6,094,848 used= 6,033,408 count= 8 avg_used= 754,176 block_cnt= 8 chunk_cnt= 7 mod=ResourceGroup [MEMORY] hold= 4,964,352 used= 4,959,488 count= 3 avg_used= 1,653,162 block_cnt= 3 chunk_cnt= 3 mod=SqlFltSpanRec [MEMORY] hold= 4,943,872 used= 4,915,456 count= 2 avg_used= 2,457,728 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe [MEMORY] hold= 4,235,264 used= 4,194,304 count= 2 avg_used= 2,097,152 block_cnt= 2 chunk_cnt= 2 mod=SHARE_BLOCK [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMem [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 12 mod=CkptDgnMemCU [MEMORY] hold= 3,772,416 used= 3,751,936 count= 1 avg_used= 3,751,936 block_cnt= 1 chunk_cnt= 1 mod=DiskIteratorSto [MEMORY] hold= 3,723,264 used= 3,698,688 count= 2 avg_used= 1,849,344 block_cnt= 2 chunk_cnt= 2 mod=LogDIOAligned [MEMORY] hold= 3,162,112 used= 3,145,808 count= 2 avg_used= 1,572,904 block_cnt= 2 chunk_cnt= 2 mod=TenantDLMgr [MEMORY] hold= 3,145,728 used= 2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 5 mod=SqlDtlQueue [MEMORY] hold= 2,828,928 used= 2,708,352 count= 604 avg_used= 4,484 block_cnt= 576 chunk_cnt= 72 mod=WorkerMap [MEMORY] hold= 2,777,088 used= 2,760,192 count= 1 avg_used= 2,760,192 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo [MEMORY] hold= 2,415,648 used= 2,387,720 count= 104 avg_used= 22,958 block_cnt= 104 chunk_cnt= 4 mod=TabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=FastFrezCkr [MEMORY] hold= 2,375,680 used= 2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta [MEMORY] hold= 2,248,704 used= 2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb [MEMORY] hold= 2,198,864 used= 416,700 count= 25,277 avg_used= 16 block_cnt= 270 chunk_cnt= 2 mod=Number [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=LogReplayTask [MEMORY] hold= 1,843,200 used= 1,770,696 count= 9 avg_used= 196,744 block_cnt= 9 chunk_cnt= 2 mod=DagMap [MEMORY] hold= 1,777,664 used= 1,770,696 count= 1 avg_used= 1,770,696 block_cnt= 1 chunk_cnt= 1 mod=TableProc [MEMORY] hold= 1,548,288 used= 1,488,096 count= 9 avg_used= 165,344 block_cnt= 9 chunk_cnt= 8 mod=SQLSessionInfo [MEMORY] hold= 1,548,288 used= 1,507,118 count= 5 avg_used= 301,423 block_cnt= 5 chunk_cnt= 1 mod=LDIOSetup [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=TransTimeWheel [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=XATimeWheel [MEMORY] hold= 1,261,568 used= 681,912 count= 77 avg_used= 8,856 block_cnt= 77 chunk_cnt= 31 mod=[T]ObDSActionAr [MEMORY] hold= 1,196,032 used= 1,182,720 count= 2 avg_used= 591,360 block_cnt= 2 chunk_cnt= 2 mod=ServerObjecPool [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsCache [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=RewriteRuleMap [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPlanCac [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsInfo [MEMORY] hold= 1,024,000 used= 1,015,808 count= 1 avg_used= 1,015,808 block_cnt= 1 chunk_cnt= 1 mod=CommitVersions [MEMORY] hold= 983,488 used= 960,256 count= 121 avg_used= 7,936 block_cnt= 121 chunk_cnt= 9 mod=TltMemtablMgr [MEMORY] hold= 958,464 used= 858,728 count= 13 avg_used= 66,056 block_cnt= 13 chunk_cnt= 13 mod=CommSysVarFac [MEMORY] hold= 958,464 used= 950,272 count= 1 avg_used= 950,272 block_cnt= 1 chunk_cnt= 1 mod=IOWorkerLQ [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=ArcSenderQueue [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=TransService [MEMORY] hold= 802,816 used= 394,352 count= 49 avg_used= 8,048 block_cnt= 49 chunk_cnt= 22 mod=[T]ObPerfEventR [MEMORY] hold= 798,400 used= 779,200 count= 100 avg_used= 7,792 block_cnt= 100 chunk_cnt= 20 mod=CkptDgnMemNode [MEMORY] hold= 798,112 used= 789,624 count= 3 avg_used= 263,208 block_cnt= 2 chunk_cnt= 2 mod=PartitTableTask [MEMORY] hold= 786,688 used= 524,352 count= 33 avg_used= 15,889 block_cnt= 33 chunk_cnt= 4 mod=LogAlloc [MEMORY] hold= 765,440 used= 730,112 count= 184 avg_used= 3,968 block_cnt= 184 chunk_cnt= 44 mod=CkpDgn [MEMORY] hold= 663,552 used= 659,200 count= 1 avg_used= 659,200 block_cnt= 1 chunk_cnt= 1 mod=MulLevelQueue [MEMORY] hold= 663,552 used= 655,360 count= 1 avg_used= 655,360 block_cnt= 1 chunk_cnt= 1 mod=FetchLog [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=ElectTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=FrzTrigger [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTF [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DetectorTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MultiVersionGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=OBJLockGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DupTbLease [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTR [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MdsT [MEMORY] hold= 598,016 used= 590,232 count= 1 avg_used= 590,232 block_cnt= 1 chunk_cnt= 1 mod=DagNetIdMap [MEMORY] hold= 458,752 used= 440,928 count= 3 avg_used= 146,976 block_cnt= 3 chunk_cnt= 3 mod=bf_queue [MEMORY] hold= 421,504 used= 404,416 count= 89 avg_used= 4,544 block_cnt= 89 chunk_cnt= 31 mod=[T]ObTraceEvent [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ApplySrv [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ReplaySrv [MEMORY] hold= 406,400 used= 396,800 count= 50 avg_used= 7,936 block_cnt= 50 chunk_cnt= 29 mod=LinearHashMapNo [MEMORY] hold= 399,200 used= 389,600 count= 50 avg_used= 7,792 block_cnt= 50 chunk_cnt= 32 mod=CkptDgnCUNode [MEMORY] hold= 385,024 used= 377,120 count= 1 avg_used= 377,120 block_cnt= 1 chunk_cnt= 1 mod=ClogGe [MEMORY] hold= 338,112 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 44 chunk_cnt= 2 mod=LSMap [MEMORY] hold= 311,296 used= 262,144 count= 256 avg_used= 1,024 block_cnt= 44 chunk_cnt= 1 mod=TabletToLS [MEMORY] hold= 311,104 used= 303,904 count= 6 avg_used= 50,650 block_cnt= 6 chunk_cnt= 2 mod=PoolFreeList [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompactMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=DmlStatHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompStMap [MEMORY] hold= 294,912 used= 261,632 count= 4 avg_used= 65,408 block_cnt= 4 chunk_cnt= 3 mod=MemTblObj [MEMORY] hold= 268,224 used= 261,888 count= 33 avg_used= 7,936 block_cnt= 33 chunk_cnt= 18 mod=CtxBlockPl [MEMORY] hold= 260,224 used= 253,952 count= 31 avg_used= 8,192 block_cnt= 31 chunk_cnt= 22 mod=ParamCharsType [MEMORY] hold= 212,992 used= 212,160 count= 1 avg_used= 212,160 block_cnt= 1 chunk_cnt= 1 mod=MetaMemMgr [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 19 mod=QueryAllocator [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 20 mod=SqlSession [MEMORY] hold= 204,800 used= 196,744 count= 1 avg_used= 196,744 block_cnt= 1 chunk_cnt= 1 mod=DagNetMap [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=TenDLBucket [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=T3MBucket [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=ResourMapLock [MEMORY] hold= 163,776 used= 156,741 count= 2 avg_used= 78,370 block_cnt= 2 chunk_cnt= 2 mod=CsSstableReader [MEMORY] hold= 147,456 used= 139,264 count= 1 avg_used= 139,264 block_cnt= 1 chunk_cnt= 1 mod=RFLTaskQueue [MEMORY] hold= 147,456 used= 138,240 count= 2 avg_used= 69,120 block_cnt= 2 chunk_cnt= 1 mod=LDBlockBitMap [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 11 mod=RawPl [MEMORY] hold= 128,576 used= 33,168 count= 471 avg_used= 70 block_cnt= 175 chunk_cnt= 33 mod=Coro [MEMORY] hold= 128,000 used= 124,928 count= 16 avg_used= 7,808 block_cnt= 16 chunk_cnt= 2 mod=TabletIDSetNode [MEMORY] hold= 122,880 used= 110,608 count= 2 avg_used= 55,304 block_cnt= 2 chunk_cnt= 1 mod=TabletIDSetBkt [MEMORY] hold= 114,688 used= 106,496 count= 1 avg_used= 106,496 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlMgr [MEMORY] hold= 114,688 used= 111,096 count= 1 avg_used= 111,096 block_cnt= 1 chunk_cnt= 1 mod=AsyncFreezeTabl [MEMORY] hold= 106,496 used= 98,432 count= 1 avg_used= 98,432 block_cnt= 1 chunk_cnt= 1 mod=tenant_mf_mgr [MEMORY] hold= 106,496 used= 98,528 count= 1 avg_used= 98,528 block_cnt= 1 chunk_cnt= 1 mod=LSSvr [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=PlanVaIdx [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=DictPl [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 10 mod=SqlSessiVarMap [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=RlePl [MEMORY] hold= 81,920 used= 73,896 count= 1 avg_used= 73,896 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairCache [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 10 mod=ConstPl [MEMORY] hold= 77,824 used= 65,536 count= 64 avg_used= 1,024 block_cnt= 12 chunk_cnt= 3 mod=TxCtxMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushLog [MEMORY] hold= 73,728 used= 66,176 count= 1 avg_used= 66,176 block_cnt= 1 chunk_cnt= 1 mod=Rpc [MEMORY] hold= 73,728 used= 65,536 count= 1 avg_used= 65,536 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LockMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxCtxMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxDataMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=DEVICE_MANAGER [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushMeta [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=StorMetaCacheIO [MEMORY] hold= 73,728 used= 72,736 count= 1 avg_used= 72,736 block_cnt= 1 chunk_cnt= 1 mod=LogSharedQueueT [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 9 mod=IntDiffPl [MEMORY] hold= 72,240 used= 1,024 count= 896 avg_used= 1 block_cnt= 10 chunk_cnt= 1 mod=CharsetUtil [MEMORY] hold= 65,216 used= 60,712 count= 4 avg_used= 15,178 block_cnt= 4 chunk_cnt= 4 mod=HB_SERVICE [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 8 mod=MdsTableMgr [MEMORY] hold= 64,000 used= 62,464 count= 8 avg_used= 7,808 block_cnt= 8 chunk_cnt= 8 mod=HashNodeLCSta [MEMORY] hold= 58,400 used= 55,296 count= 16 avg_used= 3,456 block_cnt= 11 chunk_cnt= 8 mod=CACHE_INST [MEMORY] hold= 57,344 used= 50,496 count= 1 avg_used= 50,496 block_cnt= 1 chunk_cnt= 1 mod=palf_env [MEMORY] hold= 54,000 used= 5,760 count= 240 avg_used= 24 block_cnt= 146 chunk_cnt= 27 mod=[T]MemoryContex [MEMORY] hold= 48,960 used= 44,840 count= 2 avg_used= 22,420 block_cnt= 2 chunk_cnt= 2 mod=DynamicFactor [MEMORY] hold= 48,880 used= 44,760 count= 2 avg_used= 22,380 block_cnt= 2 chunk_cnt= 2 mod=Autoincrement [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 5 mod=ColEqualPl [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 4 mod=HexStrPl [MEMORY] hold= 45,792 used= 35,616 count= 53 avg_used= 672 block_cnt= 51 chunk_cnt= 23 mod=[T]ObWarningBuf [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DagWarnHisBkt [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 1 mod=ObDMMDL [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=StrDiffPl [MEMORY] hold= 32,912 used= 22,464 count= 54 avg_used= 416 block_cnt= 23 chunk_cnt= 7 mod=OMT_Worker [MEMORY] hold= 32,768 used= 17,408 count= 2 avg_used= 8,704 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTTP [MEMORY] hold= 32,768 used= 25,664 count= 1 avg_used= 25,664 block_cnt= 1 chunk_cnt= 1 mod=TSQLSessionMgr [MEMORY] hold= 32,768 used= 24,688 count= 2 avg_used= 12,344 block_cnt= 2 chunk_cnt= 2 mod=TLD_AssignedMgr [MEMORY] hold= 32,704 used= 26,079 count= 2 avg_used= 13,039 block_cnt= 2 chunk_cnt= 2 mod=CGReadInfoMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 3 mod=StrPrefixPl [MEMORY] hold= 32,000 used= 31,232 count= 4 avg_used= 7,808 block_cnt= 4 chunk_cnt= 4 mod=HashNodePlanCac [MEMORY] hold= 24,576 used= 22,592 count= 1 avg_used= 22,592 block_cnt= 1 chunk_cnt= 1 mod=IO_MGR [MEMORY] hold= 24,576 used= 16,384 count= 1 avg_used= 16,384 block_cnt= 1 chunk_cnt= 1 mod=SlogWriteBuffer [MEMORY] hold= 24,368 used= 20,136 count= 2 avg_used= 10,068 block_cnt= 2 chunk_cnt= 2 mod=TabletStats [MEMORY] hold= 24,336 used= 20,104 count= 2 avg_used= 10,052 block_cnt= 2 chunk_cnt= 2 mod=TLD_ResourceMgr [MEMORY] hold= 21,152 used= 16,896 count= 22 avg_used= 768 block_cnt= 15 chunk_cnt= 12 mod=TGTimer [MEMORY] hold= 17,056 used= 8,448 count= 3 avg_used= 2,816 block_cnt= 2 chunk_cnt= 2 mod=BaseLogWriter [MEMORY] hold= 16,384 used= 8,448 count= 1 avg_used= 8,448 block_cnt= 1 chunk_cnt= 1 mod=PalfEnv [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TabletSvrBucket [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=SlogNopLog [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=DiaTabletMap [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_ClientTask [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MedCheckSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_TableCtxMgr [MEMORY] hold= 16,384 used= 11,584 count= 1 avg_used= 11,584 block_cnt= 1 chunk_cnt= 1 mod=LogApplyStatus [MEMORY] hold= 16,384 used= 13,824 count= 1 avg_used= 13,824 block_cnt= 1 chunk_cnt= 1 mod=LogReplayStatus [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=ColSubStrPl [MEMORY] hold= 15,648 used= 15,264 count= 2 avg_used= 7,632 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoNode [MEMORY] hold= 11,984 used= 11,592 count= 2 avg_used= 5,796 block_cnt= 2 chunk_cnt= 1 mod=StopWordSet [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=Scheduler [MEMORY] hold= 9,376 used= 8,984 count= 2 avg_used= 4,492 block_cnt= 2 chunk_cnt= 1 mod=FTPluginMap [MEMORY] hold= 9,248 used= 6,732 count= 13 avg_used= 517 block_cnt= 13 chunk_cnt= 12 mod=SqlSessiQuerSql [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapCn [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapDi [MEMORY] hold= 8,192 used= 7,760 count= 2 avg_used= 3,880 block_cnt= 1 chunk_cnt= 1 mod=HashBucPsSessIn [MEMORY] hold= 8,160 used= 5,512 count= 13 avg_used= 424 block_cnt= 12 chunk_cnt= 7 mod=PackStateMap [MEMORY] hold= 8,144 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 9 mod=SequenceIdMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LockWaitMgr [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LsElectRefInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MatchLsnMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=APPLY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=PurgeThrottle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FetchLogTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ls_id [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=REPLAY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompCkmItems [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=IORunners [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSIter [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=HandleSubmit [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=DASIDCache [MEMORY] hold= 8,128 used= 5,512 count= 13 avg_used= 424 block_cnt= 12 chunk_cnt= 8 mod=SequenceMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ShareBlksMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MdsDebugMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FTPluginHandle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LCLSender [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairs [MEMORY] hold= 8,112 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 8 mod=ContextsMap [MEMORY] hold= 8,096 used= 7,904 count= 1 avg_used= 7,904 block_cnt= 1 chunk_cnt= 1 mod=PsSessionInfo [MEMORY] hold= 8,000 used= 7,808 count= 1 avg_used= 7,808 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsInfo [MEMORY] hold= 7,984 used= 7,792 count= 1 avg_used= 7,792 block_cnt= 1 chunk_cnt= 1 mod=ColUsagNode [MEMORY] hold= 7,920 used= 7,728 count= 1 avg_used= 7,728 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsCache [MEMORY] hold= 7,904 used= 7,712 count= 1 avg_used= 7,712 block_cnt= 1 chunk_cnt= 1 mod=DmlStatNode [MEMORY] hold= 7,824 used= 7,632 count= 1 avg_used= 7,632 block_cnt= 1 chunk_cnt= 1 mod=CheckMapNode [MEMORY] hold= 7,136 used= 66 count= 33 avg_used= 2 block_cnt= 31 chunk_cnt= 11 mod=[T]TSIUseWeak [MEMORY] hold= 6,800 used= 6,216 count= 3 avg_used= 2,072 block_cnt= 2 chunk_cnt= 2 mod=DetectManager [MEMORY] hold= 5,056 used= 4,656 count= 2 avg_used= 2,328 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResGrpIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=PxPoolBkt [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResRuleIdMap [MEMORY] hold= 4,288 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 4,160 used= 3,968 count= 1 avg_used= 3,968 block_cnt= 1 chunk_cnt= 1 mod=TinyAllocator [MEMORY] hold= 3,328 used= 1,792 count= 8 avg_used= 224 block_cnt= 5 chunk_cnt= 2 mod=LogIOTask [MEMORY] hold= 3,328 used= 256 count= 16 avg_used= 16 block_cnt= 16 chunk_cnt= 14 mod=[T]TSILastOper [MEMORY] hold= 3,200 used= 3,008 count= 1 avg_used= 3,008 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlDfc [MEMORY] hold= 2,992 used= 1,920 count= 5 avg_used= 384 block_cnt= 5 chunk_cnt= 4 mod=TableLockNode [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=MedCheckMap [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=IRMMemHashBuck [MEMORY] hold= 2,048 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=LogIOWS [MEMORY] hold= 2,000 used= 1,800 count= 1 avg_used= 1,800 block_cnt= 1 chunk_cnt= 1 mod=PxResMgr [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=MSTXCTX [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TabStatMgrLock [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TXBFRetryId [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=DUP_LS_SET [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GROUP_INDEX_MAP [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GCMemtableMap [MEMORY] hold= 1,344 used= 960 count= 2 avg_used= 480 block_cnt= 1 chunk_cnt= 1 mod=PoolArenaArray [MEMORY] hold= 1,280 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=ModuleInitCtx [MEMORY] hold= 1,248 used= 1,056 count= 1 avg_used= 1,056 block_cnt= 1 chunk_cnt= 1 mod=LOG_HASH_MAP [MEMORY] hold= 1,216 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=LockMemtableMgr [MEMORY] hold= 1,152 used= 960 count= 1 avg_used= 960 block_cnt= 1 chunk_cnt= 1 mod=FTParserMgr [MEMORY] hold= 1,120 used= 120 count= 5 avg_used= 24 block_cnt= 2 chunk_cnt= 1 mod=FreezeTask [MEMORY] hold= 976 used= 776 count= 1 avg_used= 776 block_cnt= 1 chunk_cnt= 1 mod=DepTabletSet [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=CheckPointTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=DiskUsageTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=tenant_rm_mgr [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TLD_TIMER [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=FlushTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TabShellTimer [MEMORY] hold= 848 used= 648 count= 1 avg_used= 648 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfo [MEMORY] hold= 752 used= 552 count= 1 avg_used= 552 block_cnt= 1 chunk_cnt= 1 mod=LSFreeze [MEMORY] hold= 672 used= 288 count= 2 avg_used= 144 block_cnt= 2 chunk_cnt= 2 mod=ObTxLSLogCb [MEMORY] hold= 672 used= 472 count= 1 avg_used= 472 block_cnt= 1 chunk_cnt= 1 mod=Election [MEMORY] hold= 576 used= 384 count= 1 avg_used= 384 block_cnt= 1 chunk_cnt= 1 mod=HAScheduler [MEMORY] hold= 544 used= 144 count= 2 avg_used= 72 block_cnt= 1 chunk_cnt= 1 mod=TntSrvObjPool [MEMORY] hold= 512 used= 120 count= 2 avg_used= 60 block_cnt= 2 chunk_cnt= 2 mod=UserResourceMgr [MEMORY] hold= 448 used= 64 count= 2 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfFSCbNode [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTHandler [MEMORY] hold= 400 used= 208 count= 1 avg_used= 208 block_cnt= 1 chunk_cnt= 1 mod=[T]ObMacroBlock [MEMORY] hold= 320 used= 102 count= 1 avg_used= 102 block_cnt= 1 chunk_cnt= 1 mod=TxLSLogBuf [MEMORY] hold= 304 used= 112 count= 1 avg_used= 112 block_cnt= 1 chunk_cnt= 1 mod=Coordinator [MEMORY] hold= 304 used= 104 count= 1 avg_used= 104 block_cnt= 1 chunk_cnt= 1 mod=RedoLogBuckLock [MEMORY] hold= 256 used= 48 count= 1 avg_used= 48 block_cnt= 1 chunk_cnt= 1 mod=[T]TSITabletTab [MEMORY] hold= 256 used= 64 count= 1 avg_used= 64 block_cnt= 1 chunk_cnt= 1 mod=NegParamIdex [MEMORY] hold= 256 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfoMap [MEMORY] hold= 240 used= 41 count= 1 avg_used= 41 block_cnt= 1 chunk_cnt= 1 mod=KeepAliveBuf [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfRCCbNode [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=RebuildCbNode [MEMORY] hold= 208 used= 16 count= 1 avg_used= 16 block_cnt= 1 chunk_cnt= 1 mod=logservice [MEMORY] hold= 428,977,312 used= 411,783,812 count= 33,080 avg_used= 12,448 mod=SUMMARY [2024-10-16 14:57:56.636911] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=57] [MEMORY] tenant_id= 1 ctx_id= MEMSTORE_CTX_ID hold= 245,366,784 used= 243,449,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 block_cnt= 117 chunk_cnt= 117 mod=Memstore [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 mod=SUMMARY [2024-10-16 14:57:56.636922] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TRANS_CTX_MGR_ID hold= 2,097,152 used= 729,088 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 block_cnt= 1 chunk_cnt= 1 mod=PartTranCtxMgr [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 mod=SUMMARY [2024-10-16 14:57:56.636932] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= PLAN_CACHE_CTX_ID hold= 79,691,776 used= 73,331,136 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 68,663,616 used= 62,143,518 count= 2,832 avg_used= 21,943 block_cnt= 2,832 chunk_cnt= 37 mod=SqlPhyPlan [MEMORY] hold= 3,414,336 used= 3,146,720 count= 326 avg_used= 9,652 block_cnt= 326 chunk_cnt= 37 mod=SqlPlanCache [MEMORY] hold= 1,251,200 used= 1,140,800 count= 575 avg_used= 1,984 block_cnt= 192 chunk_cnt= 37 mod=CreateContext [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 73,331,136 used= 66,432,638 count= 3,735 avg_used= 17,786 mod=SUMMARY [2024-10-16 14:57:56.636943] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= GLIBC hold= 2,097,152 used= 81,040 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 80,944 used= 59,018 count= 263 avg_used= 224 block_cnt= 20 chunk_cnt= 1 mod=PlJit [MEMORY] hold= 96 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PlCodeGen [MEMORY] hold= 81,040 used= 59,050 count= 264 avg_used= 223 mod=SUMMARY [2024-10-16 14:57:56.636950] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= CO_STACK hold= 136,314,880 used= 133,668,864 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 block_cnt= 259 chunk_cnt= 65 mod=CoStack [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 mod=SUMMARY [2024-10-16 14:57:56.636962] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= KVSTORE_CACHE_ID hold= 861,929,472 used= 861,929,472 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 mod=SUMMARY [2024-10-16 14:57:56.636971] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= META_OBJ_CTX_ID hold= 37,748,736 used= 36,290,496 limit= 429,496,720 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 29,491,200 used= 26,236,400 count= 400 avg_used= 65,591 block_cnt= 400 chunk_cnt= 18 mod=L_TabletPool [MEMORY] hold= 6,709,248 used= 5,952,128 count= 91 avg_used= 65,408 block_cnt= 91 chunk_cnt= 15 mod=N_TabletPool [MEMORY] hold= 90,048 used= 84,768 count= 2 avg_used= 42,384 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 36,290,496 used= 32,273,296 count= 493 avg_used= 65,463 mod=SUMMARY [2024-10-16 14:57:56.636979] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TX_CALLBACK_CTX_ID hold= 12,582,912 used= 6,234,176 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 block_cnt= 767 chunk_cnt= 6 mod=MemtableCallbac [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:56.636990] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 1 ctx_id= LOB_CTX_ID hold= 2,097,152 used= 81,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LobAllocator [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 81,856 used= 73,344 count= 2 avg_used= 36,672 mod=SUMMARY [2024-10-16 14:57:56.637002] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= PS_CACHE_CTX_ID hold= 2,097,152 used= 16,288 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 block_cnt= 4 chunk_cnt= 1 mod=SqlPsCache [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 mod=SUMMARY [2024-10-16 14:57:56.637669] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 1 ctx_id= TX_DATA_TABLE hold= 54,546,432 used= 52,779,584 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 49,588,928 used= 48,417,536 count= 6,101 avg_used= 7,936 block_cnt= 6,101 chunk_cnt= 25 mod=TX_DATA_SLICE [MEMORY] hold= 3,182,528 used= 3,153,728 count= 3 avg_used= 1,051,242 block_cnt= 3 chunk_cnt= 3 mod=MEMTABLE_ARENA [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 52,779,584 used= 51,579,200 count= 6,105 avg_used= 8,448 mod=SUMMARY [2024-10-16 14:57:56.637685] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=9] [MEMORY] tenant_id= 1 ctx_id= MDS_DATA_ID hold= 4,194,304 used= 398,272 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 260,096 used= 253,952 count= 32 avg_used= 7,936 block_cnt= 32 chunk_cnt= 2 mod=MdsTable [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 2 mod=TX_OP [MEMORY] hold= 398,272 used= 388,864 count= 49 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:56.637695] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= SCHEMA_SERVICE hold= 2,097,152 used= 8,128 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=NonPartTenCac [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:57:56.637725] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,082,318,848 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 371,380,224 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 14,680,064 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 104,857,600 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 20,971,520 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= RPC_CTX_ID hold_bytes= 6,291,456 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 26,357,760 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 237,785,088 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= UNEXPECTED_IN_500 hold_bytes= 295,800,832 limit= 9,223,372,036,854,775,807 [2024-10-16 14:57:56.637926] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 371,380,224 used= 311,599,488 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 66,584,784 used= 66,551,824 count= 33 avg_used= 2,016,721 block_cnt= 33 chunk_cnt= 33 mod=Compressor [MEMORY] hold= 31,477,760 used= 31,458,304 count= 1 avg_used= 31,458,304 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 23,584,768 used= 23,499,200 count= 9 avg_used= 2,611,022 block_cnt= 9 chunk_cnt= 8 mod=IoControl [MEMORY] hold= 15,925,248 used= 15,247,872 count= 216 avg_used= 70,592 block_cnt= 216 chunk_cnt= 42 mod=[T]ObSessionDIB [MEMORY] hold= 12,873,728 used= 12,856,352 count= 1 avg_used= 12,856,352 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 9,457,664 used= 9,437,784 count= 1 avg_used= 9,437,784 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 5,505,024 used= 5,238,784 count= 32 avg_used= 163,712 block_cnt= 32 chunk_cnt= 22 mod=Iterator= 1 ORDER BY frozen_scn) [2024-10-16 14:57:56.856926] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F00-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:56.857483] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:56.857532] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061874438469000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:56.861329] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5807, mem_used=12478464) [2024-10-16 14:57:56.863102] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22703][T1004_TntShared][T1004][Y0-0000000000000000-0-0] [lt=12] clear dtl interm result cost(us)(clear_cost=932, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:56.877632] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.880564] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061872872026, last_first_register_time_us=1722494157071428, register_time_us=1729061872872026, curr_time_us=1729061876880563) [2024-10-16 14:57:56.880615] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=25] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061876880590, last_submit_register_req_time_us=1729061876880590, register_parent_reason=3) [2024-10-16 14:57:56.880625] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:56.885101] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=252, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:56.885118] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:56.886943] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.905308] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=22] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:56.905720] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=14] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:56.905965] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=20] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:56.906018] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=15] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061871843583000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:56.920621] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:56.920644] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=16] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=216) [2024-10-16 14:57:56.921410] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DDC-0-0] [lt=13] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:57:56.924415] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:56.925512] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:56.933469] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:56.933485] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:56.933725] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:56.934140] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3D9-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:56.934149] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3D9-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=636, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:56.934153] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3D9-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=243, avg_log_submit_to_slide_time=621) [2024-10-16 14:57:56.938716] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE9-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061876906398, keepalive_ts:-1}) [2024-10-16 14:57:56.938738] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBE9-0-0] [lt=21] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061876906398, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:56.944927] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=3776, mem_used=8318976) [2024-10-16 14:57:56.947812] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7C-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:56.947844] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7C-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061876656086003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061876656086003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:56.948857] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D7C-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061876948618000, start_id=1729061876948618000, end_id=1729061876948618001) [2024-10-16 14:57:56.951669] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7C-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061876950754000, start_id=1729061876950754000, end_id=1729061876950754001) [2024-10-16 14:57:56.951739] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7C-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:56.951753] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7C-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:56.956194] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:56.957035] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F02-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:56.957046] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F02-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20379, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:56.957051] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F02-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=583, avg_log_submit_to_slide_time=20374) [2024-10-16 14:57:56.957063] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F02-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173303, log_task={header:{begin_lsn:{lsn:14352393104}, end_lsn:{lsn:14352393226}, log_id:115173303, min_scn:{val:1729061876656086002, v:0}, max_scn:{val:1729061876656086002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352392982}, prev_proposal_id:17, committed_end_lsn:{lsn:14352393104}, data_checksum:940009244, accum_checksum:3763716042, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061876856922, freeze_ts:1729061876856922, submit_ts:1729061876856938, flushed_ts:1729061876857842, gen_to_freeze cost time:0, gen_to_submit cost time:16, submit_to_flush cost time:904}, fs_cb_begin_ts=1729061876957029, log_life_time=100107) [2024-10-16 14:57:56.957106] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:56.960227] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBD-0-0] [lt=18] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:56.960248] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBD-0-0] [lt=18] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:56.964448] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061876963498000, start_id=1729061876963498000, end_id=1729061876963498001) [2024-10-16 14:57:56.964847] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] [COMPACTION DAG STATUS] (start_time=1729061866955418, end_time=1729061876964845, dag_status={}) [2024-10-16 14:57:56.964858] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:56.964865] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:56.964869] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:56.964873] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:56.964887] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=14] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:56.964891] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:56.964894] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:56.964897] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:56.964901] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:56.964904] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964910] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964914] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964918] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964922] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964926] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964930] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964934] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964938] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964941] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964945] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964949] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964953] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964957] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964961] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964964] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964968] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964972] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964976] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964980] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964983] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.964988] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965005] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=16] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965008] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965013] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965016] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965024] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965028] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965032] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965036] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965039] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965043] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965047] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965051] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965054] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965058] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965062] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965066] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965070] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965074] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965077] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965081] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965085] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965088] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965092] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965102] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965105] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965109] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965113] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965117] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965120] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965124] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965128] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965132] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965136] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965139] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965143] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965147] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:56.965151] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:56.965162] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=10] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:56.965165] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:56.965168] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:56.965171] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:56.965173] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:56.965175] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:56.965178] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:56.965180] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:56.965182] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:56.965185] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:56.968861] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061876963498000, start_id=1729061876963498000, end_id=1729061876963498001) [2024-10-16 14:57:56.969506] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33576, start_service_tstamp_=1722494263708085, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:56.972302] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=14] ====== tenant freeze timer task ====== [2024-10-16 14:57:56.972723] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:56.972743] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:57:56.972756] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=11] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:56.977709] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:56.978348] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061872973297, last_first_register_time_us=1722494150998907, register_time_us=1729061872973297, curr_time_us=1729061876978347) [2024-10-16 14:57:56.978383] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061876978366, last_submit_register_req_time_us=1729061876978366, register_parent_reason=3) [2024-10-16 14:57:56.978392] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:57:56.986115] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3667) [2024-10-16 14:57:56.986144] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3667) [2024-10-16 14:57:56.986321] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10C0-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:56.986332] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10C0-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1328, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:56.986980] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.013434] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] Garbage Collector is running(seq_=656700, gc_interval=10000000) [2024-10-16 14:57:57.013467] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=14] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:57:57.013482] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=8] gc_check_member_list_ cost time(ret=0, time_us=33) [2024-10-16 14:57:57.013491] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:57:57.014214] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=4] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:57:57.014233] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=15] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:57:57.014242] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=9] execute_gc cost time(ret=0, time_us=9) [2024-10-16 14:57:57.014246] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:57:57.018365] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:57.018377] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151574520, tenant_id=1) [2024-10-16 14:57:57.018390] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:57.020999] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877020133000, start_id=1729061877020133000, end_id=1729061877020133001) [2024-10-16 14:57:57.027647] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877027614000, start_id=1729061877027614000, end_id=1729061877027614001) [2024-10-16 14:57:57.033535] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061877033534, bytes: 4446820519459, bw: 0.366323 MB/s, add_ts: 1003096, add_bytes: 385307 [2024-10-16 14:57:57.033705] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=460, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:57.033719] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=436, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:57.033733] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:57.033959] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40039-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:57.033966] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40039-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1248, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:57.043060] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:57.043078] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:57.043092] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=291987823, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:57.046177] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:57.046497] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:57.046509] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:57.049096] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061877045509000, start_id=1729061877045509000, end_id=1729061877045509001) [2024-10-16 14:57:57.049196] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Cache wash clean map node details(ret=0, clean_node_count=16, clean_time=0, clean_start_pos=1509936, clean_num=62914) [2024-10-16 14:57:57.049572] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44B5-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:57.049583] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44B5-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1119, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:57.057074] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F04-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:57.057232] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:57.062515] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061877061964000, start_id=1729061877061964000, end_id=1729061877061964001) [2024-10-16 14:57:57.062537] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:57.062553] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061877062550], gts:1729061877061964000, latest_srr:[mts=1729061877062550]}) [2024-10-16 14:57:57.072925] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE789F-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:57:57.073172] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:57.077739] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.084584] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D83-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:57.084817] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:57.087045] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.128005] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:57.128023] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:57.128129] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] [COMPACTION DAG STATUS] (start_time=1729061867126637, end_time=1729061877128128, dag_status={}) [2024-10-16 14:57:57.128155] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=19] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:57.128163] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:57.128167] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:57.128174] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:57.128181] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:57.128184] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:57.128191] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:57.128194] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:57.128200] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:57.128203] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128213] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128217] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128221] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128228] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128232] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128238] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128242] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128249] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128253] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128256] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128262] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128268] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128278] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128312] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=34] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128323] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128339] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=16] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128345] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128351] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128355] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128359] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128366] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128370] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128377] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128381] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128388] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128407] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128411] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128418] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128422] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128429] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128433] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128440] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128444] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128450] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128454] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128461] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128465] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128472] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128476] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128483] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128487] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128491] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128494] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128498] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128504] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128508] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128515] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128521] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128526] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128532] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128537] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128541] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128545] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128549] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128552] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:57.128557] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:57.128564] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:57.128568] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:57.128572] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:57.128575] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:57.128579] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:57.128581] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:57.128585] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:57.128587] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:57.128591] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:57.128594] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:57.133854] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:57.133881] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:57.134990] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:57.137577] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3952, mem_used=8318976) [2024-10-16 14:57:57.141131] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 147.50; ) [2024-10-16 14:57:57.141152] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:57.141191] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 6144.00, iops: 3.99, rt: 165.00; ) [2024-10-16 14:57:57.141199] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:57.141218] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:57.146544] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:57.146558] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:57.146801] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:57.147035] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44B7-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=345, avg_log_submit_to_slide_time=826) [2024-10-16 14:57:57.149617] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=18] gc stale ls task succ [2024-10-16 14:57:57.156261] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:57.156336] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA65-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061877155728000, start_id=1729061877155728000, end_id=1729061877155728001) [2024-10-16 14:57:57.156382] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA67-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061877155728000, start_id=1729061877155728000, end_id=1729061877155728001) [2024-10-16 14:57:57.156543] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA69-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061877155728000, start_id=1729061877155728000, end_id=1729061877155728001) [2024-10-16 14:57:57.156729] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA66-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061877155728000, start_id=1729061877155728000, end_id=1729061877155728001) [2024-10-16 14:57:57.156736] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA68-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061877155728000, start_id=1729061877155728000, end_id=1729061877155728001) [2024-10-16 14:57:57.159440] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=5] current monitor number(seq_=-1) [2024-10-16 14:57:57.164542] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF250-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.108497524261474609e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.002399163312093151e+03) [2024-10-16 14:57:57.164719] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061877163174000, start_id=1729061877163174000, end_id=1729061877163174001) [2024-10-16 14:57:57.164778] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:57.164792] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:57.169146] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061877167992000, start_id=1729061877167992000, end_id=1729061877167992001) [2024-10-16 14:57:57.177805] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.180299] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061877180297, bytes: 0, bw: 0.000000 MB/s, add_ts: 1008315, add_bytes: 0 [2024-10-16 14:57:57.186014] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3667) [2024-10-16 14:57:57.187108] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.198149] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=7] ====== traversal_flush timer task ====== [2024-10-16 14:57:57.198267] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D88-0-0] [lt=9] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061887008855000, latest_log_ts:{val:1729061870498146001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:57.198325] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D88-0-0] [lt=54] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:57:57.198422] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:57.198760] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503B-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:57.198778] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503B-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:57.199889] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=43] success to update min reserved snapshot(reserved_snapshot=1729060074438469000, duration=1800, snapshot_gc_ts=1729061874438469000) [2024-10-16 14:57:57.200436] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=12] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060074438469000) [2024-10-16 14:57:57.200450] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=12] update reserved snapshot finished(cost_ts=564, reserved_snapshot=1729060074438469000) [2024-10-16 14:57:57.211000] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:57.221183] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877220498000, start_id=1729061877220498000, end_id=1729061877220498001) [2024-10-16 14:57:57.227865] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877226831000, start_id=1729061877226831000, end_id=1729061877226831001) [2024-10-16 14:57:57.233590] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:57.233616] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:57.233838] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:57.234061] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3E2-0-0] [lt=21] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=959, avg_log_submit_to_slide_time=1222) [2024-10-16 14:57:57.249347] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877249048000, start_id=1729061877249048000, end_id=1729061877249048001) [2024-10-16 14:57:57.249499] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:57.265098] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21531][T1_TntSharedTim][T1][Y0-0000000000000000-0-0] [lt=24] clear dtl interm result cost(us)(clear_cost=1866, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:57.266213] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21675][T1_L0_G0][T1][YB4A0A000407-00061E99798D0078-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510345134607, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410624, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:57.273306] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:57.273648] INFO [STORAGE] try_shrink (ob_compaction_memory_pool.cpp:522) [22347][T1003_MergeMemP][T1003][Y0-0000000000000000-0-0] [lt=15] Compaction Mem Pool current stat: (ret=0, max_block_num=16, total_block_num=8, Chunk list size: =1) [2024-10-16 14:57:57.277191] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88F-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:57.277218] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA88F-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061877155728000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061877155728000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:57.277870] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.278268] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA88F-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061877277447000, start_id=1729061877277447000, end_id=1729061877277447001) [2024-10-16 14:57:57.281188] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061877280941000, start_id=1729061877280941000, end_id=1729061877280941001) [2024-10-16 14:57:57.281262] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88F-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10588, avg_group_log_size=622) [2024-10-16 14:57:57.281279] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88F-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10588) [2024-10-16 14:57:57.281292] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:142) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA88F-0-0] [lt=6] transaction rpc statistics(total_rt=2209, total_process=21, avg_rt=100) [2024-10-16 14:57:57.281558] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=233, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:57.281570] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10588) [2024-10-16 14:57:57.281580] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=377, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:57.281633] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10588) [2024-10-16 14:57:57.281649] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=13, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:57.281744] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95E6-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:57.281750] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95E6-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1060, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:57.281760] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95E6-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=34, avg_log_gen_to_submit_time=63, avg_log_submit_to_flush_time=793, avg_log_submit_to_slide_time=997) [2024-10-16 14:57:57.281837] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:57.287175] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.298255] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850D9-0-0] [lt=27] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:57.298278] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850D9-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:57.298287] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850D9-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:57.310261] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD103E-0-0] [lt=8] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:57.313132] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB4-0-0] [lt=33] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061877280869, keepalive_ts:-1}) [2024-10-16 14:57:57.313157] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB4-0-0] [lt=24] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061877280869, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:57.313178] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB3-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061877280804, keepalive_ts:-1}) [2024-10-16 14:57:57.313186] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB3-0-0] [lt=8] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061877280804, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:57.323365] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1144-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061877322478000, start_id=1729061877322478000, end_id=1729061877322478001) [2024-10-16 14:57:57.328781] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=459, mem_used=2079744) [2024-10-16 14:57:57.333074] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=12] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:57.333929] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:57.339684] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=44] success to update min reserved snapshot(reserved_snapshot=1729060068730909000, duration=1800, snapshot_gc_ts=1729061868730909000) [2024-10-16 14:57:57.340348] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=14] update reserved snapshot finished(cost_ts=666, reserved_snapshot=1729060068730909000) [2024-10-16 14:57:57.346926] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=202, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:57.346946] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=110, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:57.346969] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:57:57.346986] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:57.352958] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:57.352978] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=15] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:57.352986] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=7] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:57:57.352989] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:57:57.352993] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:57:57.353000] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17118) [2024-10-16 14:57:57.357712] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2359275, replace_num=31457) [2024-10-16 14:57:57.357727] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:57.364897] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061877364003000, start_id=1729061877364003000, end_id=1729061877364003001) [2024-10-16 14:57:57.365623] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] [APPLY STAT CB APPEND COST TIME](cur_stat_count=55, stat_interval=5000000, avg (count/cost)=44, this=0x2b0e07718280, extra_info="") [2024-10-16 14:57:57.365635] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=55, stat_interval=5000000, avg (count/cost)=1120, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:57:57.365639] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=55, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:57:57.365642] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=55, stat_interval=5000000, avg (count/cost)=-689, this=0x2b0e07718910, extra_info="") [2024-10-16 14:57:57.365646] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=55, stat_interval=5000000, avg (count/cost)=475, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:57:57.369208] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061877368238000, start_id=1729061877368238000, end_id=1729061877368238001) [2024-10-16 14:57:57.373171] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78A5-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:57.373451] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB5-0-0] [lt=17] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:57.373471] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB5-0-0] [lt=15] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=21) [2024-10-16 14:57:57.377929] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.384991] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D89-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:57.385261] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=224, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:57.385278] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:57.385318] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:57.387234] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.395333] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:57.396767] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=8] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:57:57.421287] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877420394000, start_id=1729061877420394000, end_id=1729061877420394001) [2024-10-16 14:57:57.422557] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601466, offset=9643378}, end_cursor:ObLogCursor{file_id=192, log_id=47601467, offset=9643662}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2dbc050, len:2702, log_data_len:284, seq:47601466, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:57.424338] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B3-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061877423772000, start_id=1729061877423772000, end_id=1729061877423772001) [2024-10-16 14:57:57.425316] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601467, offset=9643662}, end_cursor:ObLogCursor{file_id=192, log_id=47601468, offset=9643936}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db738a050, len:2418, log_data_len:274, seq:47601467, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:57.427762] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061877427064000, start_id=1729061877427064000, end_id=1729061877427064001) [2024-10-16 14:57:57.428376] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601468, offset=9643936}, end_cursor:ObLogCursor{file_id=192, log_id=47601469, offset=9644210}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2db0050, len:2144, log_data_len:274, seq:47601468, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:57.428683] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=29] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601469, offset=9644210}, end_cursor:ObLogCursor{file_id=192, log_id=47601470, offset=9644484}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2034050, len:1870, log_data_len:274, seq:47601469, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:57.428750] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561434, miss_cnt:1, cache_read_size:152590698932, hit_ratio:"9.999999928346967604e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:57.428758] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599021, miss_cnt:9, cache_read_size:149777931249, hit_ratio:"9.999999345925622007e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:57.449441] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061877449177000, start_id=1729061877449177000, end_id=1729061877449177001) [2024-10-16 14:57:57.449954] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:57.473154] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78A7-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:57.473395] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=185, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:57.473422] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:57.473681] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54E6-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:57.473688] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54E6-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=510, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:57.473712] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:57.473750] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:57.473757] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:57.477990] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.485714] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8F-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:57.485734] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A8F-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:57.485897] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D8B-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:57.486150] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=43881687, this={dio_aligned_buf:{buf_write_offset:1361, buf_padding_size:2735, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35369200, truncate_used_ts:14641122}, log_block_size:67108864, dir_fd:201, io_fd:197}, aligned_buf_len=4096, aligned_block_offset=43880448, buf_len=122, total_write_size_=12182, total_write_size_after_dio_=421888, ob_pwrite_used_ts=23375, count=100, each_pwrite_cost=233) [2024-10-16 14:57:57.487294] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.490686] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:57:57.490699] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:57.490712] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=70842594, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:57.492226] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:57.492235] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=8] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:57.492244] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:57.492249] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:57.492253] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:57.492255] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:57.492258] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:57.492260] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:57.492263] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:57.492265] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:57.492268] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:57.492290] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2A-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:57.492297] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2A-0-0] [lt=5] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:57.494745] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257814, offset=39047708}, end_cursor:ObLogCursor{file_id=197, log_id=49257815, offset=39047991}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72b6050, len:3556, log_data_len:283, seq:49257814, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:57.496780] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB37E-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061877496553000, start_id=1729061877496553000, end_id=1729061877496553001) [2024-10-16 14:57:57.497743] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257815, offset=39047991}, end_cursor:ObLogCursor{file_id=197, log_id=49257816, offset=39048273}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73ce050, len:3273, log_data_len:282, seq:49257815, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:57.502001] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=9] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:57.502557] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:57.502619] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061868730909000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:57:57.503027] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257816, offset=39048273}, end_cursor:ObLogCursor{file_id=197, log_id=49257817, offset=39048555}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73ca050, len:2991, log_data_len:282, seq:49257816, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:57.503164] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257817, offset=39048555}, end_cursor:ObLogCursor{file_id=197, log_id=49257818, offset=39048837}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20a0050, len:2709, log_data_len:282, seq:49257817, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:57.503206] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650371, miss_cnt:32, cache_read_size:8050436564, hit_ratio:"9.999994892291435011e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:57.503222] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657111, miss_cnt:17, cache_read_size:8050184179, hit_ratio:"9.999997286821061815e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:57.518786] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=16][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:57.527337] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=34] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a540f0, tx_id:{txid:199664436}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061877526077, active_ts:1729061877526077, commit_ts:-1, finish_ts:1729061877527176, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061907526072, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061877518718, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:57.528169] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=50] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a5bf70, tx_id:{txid:199664437}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061877527176, active_ts:1729061877527176, commit_ts:-1, finish_ts:1729061877527176, timeout_us:29999996, lock_timeout_us:-1, expire_ts:1729061907527172, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061877518718, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:57:57.533797] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:57.533809] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:57.534070] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:57.543831] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=911101952, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=162639872, wash_ratio=6, tenant_id=1001) [2024-10-16 14:57:57.543883] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=20] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:57.543895] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=11] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:57.543924] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=969699328, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=104042496, wash_ratio=6, tenant_id=1002) [2024-10-16 14:57:57.543966] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:57.543974] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:57:57.544002] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=927879168, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=145862656, wash_ratio=6, tenant_id=1003) [2024-10-16 14:57:57.544044] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=13] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:57.544052] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011871, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:57.544100] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=958939136, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=114802688, wash_ratio=6, tenant_id=1004) [2024-10-16 14:57:57.544146] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:57:57.544162] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=16] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:57:57.546787] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:57.546804] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:57.546826] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:57.546835] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:57.547074] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:57.547132] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:57.547346] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44C3-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:57.547358] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44C3-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=805, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:57.557817] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:57.562931] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061877562093000, start_id=1729061877562093000, end_id=1729061877562093001) [2024-10-16 14:57:57.562960] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=18] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061877562956], gts:1729061877562093000, latest_srr:[mts=1729061877562956]}) [2024-10-16 14:57:57.564893] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061877564217000, start_id=1729061877564217000, end_id=1729061877564217001) [2024-10-16 14:57:57.566116] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1001, new_min_version={val:1729061877280941000, v:0}, new_max_version={val:1729061877280941000, v:0}, last_min_version={val:1729061877163174000, v:0}, last_max_version={val:1729061877163174000, v:0}, total_time_us=2178, delta_us=285173) [2024-10-16 14:57:57.566137] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1001, current_version={val:1729061877280941000, v:0}, min_version={val:1729061877280941000, v:0}, max_version={val:1729061877280941000, v:0}, in_service=true, leader_epoch=33705) [2024-10-16 14:57:57.568855] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.569262] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061877567988000, start_id=1729061877567988000, end_id=1729061877567988001) [2024-10-16 14:57:57.569359] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.569416] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.569436] INFO [SQL] alloc_value (ob_sql_session_mgr.cpp:268) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B74EFEEBC-0-0] [lt=15] alloc_session_count(alloc_total_count=645490000) [2024-10-16 14:57:57.569461] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.569935] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797D5B-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11310) [2024-10-16 14:57:57.578051] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.578684] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=18) [2024-10-16 14:57:57.585083] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D8D-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:57.585096] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D8D-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10343, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:57.587438] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.589219] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCB3F-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061877588479000, start_id=1729061877588479000, end_id=1729061877588479001) [2024-10-16 14:57:57.596888] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:57.596907] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:57.597488] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1001}, {id:1002}]) [2024-10-16 14:57:57.598432] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=206, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:57.598451] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:57.598488] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:57.598637] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44C5-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:57.598644] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44C5-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10673, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:57.610749] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5678, mem_used=10398720) [2024-10-16 14:57:57.612457] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DA-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061877580191, keepalive_ts:-1}) [2024-10-16 14:57:57.612473] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DA-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061877580191, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:57.613746] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=34] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:57.621414] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061877620183000, start_id=1729061877620183000, end_id=1729061877620183001) [2024-10-16 14:57:57.628052] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061877626686000, start_id=1729061877626686000, end_id=1729061877626686001) [2024-10-16 14:57:57.649519] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061877648541000, start_id=1729061877648541000, end_id=1729061877648541001) [2024-10-16 14:57:57.649838] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=258, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:57.649858] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=124, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:57.649874] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:57.650036] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD248-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:57.650048] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD248-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=639, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:57.650250] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:57.653975] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=24] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:57:57.656809] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA6B-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061877656136000, start_id=1729061877656136000, end_id=1729061877656136001) [2024-10-16 14:57:57.656839] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA6A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061877656136000, start_id=1729061877656136000, end_id=1729061877656136001) [2024-10-16 14:57:57.657023] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA6E-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061877656136000, start_id=1729061877656136000, end_id=1729061877656136001) [2024-10-16 14:57:57.657225] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA6D-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061877657186000, start_id=1729061877657186000, end_id=1729061877657186001) [2024-10-16 14:57:57.657697] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=191, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:57.657715] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=75, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:57.657757] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:57.670453] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:57:57.670472] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=17] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:57:57.670479] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:57:57.670484] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:57.672703] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=115, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:787547, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787548, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787549, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:787550, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:57:57.672882] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=176] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:57.672951] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:57.678111] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.679000] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:57:57.680131] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877679743000, start_id=1729061877679743000, end_id=1729061877679743001) [2024-10-16 14:57:57.680245] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=26] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=11860, avg_group_log_size=658) [2024-10-16 14:57:57.680259] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11904) [2024-10-16 14:57:57.685684] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9265-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=245, avg_log_submit_to_slide_time=10342) [2024-10-16 14:57:57.685726] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:57.687507] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=23] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:57.687527] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:57.687534] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.694625] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D3-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:57.694650] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D3-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061877656136000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061877656136000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:57.699670] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D9-0-0] [lt=19] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061877690894, keepalive_ts:-1}) [2024-10-16 14:57:57.699684] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79D9-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061877690894, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:57.699891] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:57.700253] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D3-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061877699811000, start_id=1729061877699811000, end_id=1729061877699811001) [2024-10-16 14:57:57.700538] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11860) [2024-10-16 14:57:57.700743] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD24A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=411, avg_log_submit_to_slide_time=608) [2024-10-16 14:57:57.700811] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:57.727637] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:57.727995] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=8] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:57.733912] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:57.733936] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:57.734323] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:57.750815] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C60-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:57.750841] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C60-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061877588479000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061877588479000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:57.754607] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C60-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061877753679000, start_id=1729061877753679000, end_id=1729061877753679001) [2024-10-16 14:57:57.757889] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:57.759035] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:57.759061] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=14] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:57.759073] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:57.764860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061877764355000, start_id=1729061877764355000, end_id=1729061877764355001) [2024-10-16 14:57:57.769397] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877768588000, start_id=1729061877768588000, end_id=1729061877768588001) [2024-10-16 14:57:57.772613] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=16] gc stale ls task succ [2024-10-16 14:57:57.778170] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.780168] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:57.780183] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:57.780191] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=7] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=54) [2024-10-16 14:57:57.782476] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=19] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:57.786377] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:57.786393] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3655, avg_group_log_size=332) [2024-10-16 14:57:57.786402] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:57.786714] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=789, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:57.786728] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=32, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:57.787567] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.788173] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.788185] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.788297] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.788922] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.789156] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.789404] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.789520] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.790620] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.790701] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.792639] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=14, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:57.792660] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.792676] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:57.793650] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45C92-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=1412, avg_log_submit_to_slide_time=1662) [2024-10-16 14:57:57.798878] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:57.821461] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061877821198000, start_id=1729061877821198000, end_id=1729061877821198001) [2024-10-16 14:57:57.828100] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061877827579000, start_id=1729061877827579000, end_id=1729061877827579001) [2024-10-16 14:57:57.828882] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:207589567296}, scn={val:1729061877657186002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061877798853, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589567296}, next_to_submit_scn_:{val:1729061877657186002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589567174}, end_lsn:{lsn:207589567296}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061877657186001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061877657186001, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:1406838162, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061877657186001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599025, miss_cnt:9, cache_read_size:149777931737, hit_ratio:"9.999999345925640881e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:57:57.829287] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:57.829300] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:57.829314] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=52586501, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:57.831710] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1001}, {id:1003}]) [2024-10-16 14:57:57.847275] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=249, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:57.847291] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=46, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:57.847315] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:57.847597] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F42EA-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=327, avg_log_submit_to_slide_time=875) [2024-10-16 14:57:57.849724] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061877847683000, start_id=1729061877847683000, end_id=1729061877847683001) [2024-10-16 14:57:57.850564] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:57.857697] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F14-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:57.857717] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F14-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:57.861412] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=23] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5868, mem_used=12478464) [2024-10-16 14:57:57.861444] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22136][T1002_TntTransf][T1002][YB4A0A000407-00061E9AB24CC6BE-0-0] [lt=15] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:57:57.878233] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.881354] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061873877670, last_first_register_time_us=1722494157071428, register_time_us=1729061873877670, curr_time_us=1729061877881353) [2024-10-16 14:57:57.881389] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061877881374, last_submit_register_req_time_us=1729061877881374, register_parent_reason=3) [2024-10-16 14:57:57.881400] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:57.883587] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=23] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):114, sequence:1}) [2024-10-16 14:57:57.885708] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:57.885738] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=29] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:57.887632] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.905557] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=13] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:57.912490] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=17] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:57.922076] INFO [SQL] free_value (ob_sql_session_mgr.cpp:321) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B795D8DEF-0-0] [lt=12] free_session_count(free_total_count=645490000) [2024-10-16 14:57:57.924488] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:57.925587] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:57.934038] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:57.934054] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:57.934332] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:57.934529] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3F7-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:57.934535] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3F7-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=559, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:57.934539] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD3F7-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=263, avg_log_submit_to_slide_time=542) [2024-10-16 14:57:57.940725] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15E6-0-0] [lt=9] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:57.945038] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3804, mem_used=8318976) [2024-10-16 14:57:57.954185] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7D-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:57.954201] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7D-0-0] [lt=8] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061877657186002, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061877657186002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:57.955267] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D7D-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061877955011000, start_id=1729061877955011000, end_id=1729061877955011001) [2024-10-16 14:57:57.956662] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=48] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:57.956716] INFO [LIB] stat (utility.h:1165) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:57.957570] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=36] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:57.957639] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:57:57.957649] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=7] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:57.957654] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:57:57.957659] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:57.958052] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:57:57.958064] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=7] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:57:57.958071] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=162) [2024-10-16 14:57:57.958093] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:57:57.959911] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:57:57.960259] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=345] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:57.961557] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7D-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061877961332000, start_id=1729061877961332000, end_id=1729061877961332001) [2024-10-16 14:57:57.964905] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877964573000, start_id=1729061877964573000, end_id=1729061877964573001) [2024-10-16 14:57:57.969430] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061877967751000, start_id=1729061877967751000, end_id=1729061877967751001) [2024-10-16 14:57:57.978292] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.985202] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=29] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:57:57.986539] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=18] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:57.987701] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:57.988216] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:57.988408] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10CB-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3655) [2024-10-16 14:57:57.988460] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3655) [2024-10-16 14:57:57.988487] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:57.988494] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1200, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:57.989005] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=9] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:57.989033] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=28] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:57.989051] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:57.989115] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061889029444000, latest_log_ts:{val:1729061872445301001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:57.989157] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=39] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:57.989176] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:57:57.989183] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061872445301001, v:0}) [2024-10-16 14:57:57.989187] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:57.989191] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:57.989197] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:57.989206] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:57:57.989212] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:57.989217] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:57.997198] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=22] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:57:57.998554] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:207589567474}, scn={val:1729061877657186003, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:57.998580] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:14, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061877998509, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589567530}, next_to_submit_scn_:{val:1729061877657186004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589567418}, end_lsn:{lsn:207589567530}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061877657186003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061877657186003, v:0}, data_checksum:2210724997, flag:0}}, init_mode_version:0, accumulate_checksum:1835374056, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061877657186003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599027, miss_cnt:9, cache_read_size:149777931971, hit_ratio:"9.999999345925649763e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589567474}, scn_:{val:1729061877657186003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061877879504, first_handle_ts_:1729061877998579, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f38a823a0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:57:57.998746] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060070205428000) [2024-10-16 14:57:57.998762] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589567474}, scn_:{val:1729061877657186003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061877879504, first_handle_ts_:1729061877998734, replay_cost_:26, retry_cost_:-1, read_log_buf_:0x2b0f38a823b0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061877998509, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589567530}, next_to_submit_scn_:{val:1729061877657186004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589567418}, end_lsn:{lsn:207589567530}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061877657186003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061877657186003, v:0}, data_checksum:2210724997, flag:0}}, init_mode_version:0, accumulate_checksum:1835374056, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061877657186003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599027, miss_cnt:9, cache_read_size:149777931971, hit_ratio:"9.999999345925649763e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_queue_idx=16, ret=0) [2024-10-16 14:57:58.017532] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=24] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:58.017808] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:58.021508] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061878021468000, start_id=1729061878021468000, end_id=1729061878021468001) [2024-10-16 14:57:58.028236] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=58] get number(ret=0, service_type_=0, range=1, base_id=1729061878026670000, start_id=1729061878026670000, end_id=1729061878026670001) [2024-10-16 14:57:58.034156] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=17] PNIO [ratelimit] time: 1729061878034155, bytes: 4446820921483, bw: 0.383162 MB/s, add_ts: 1000621, add_bytes: 402024 [2024-10-16 14:57:58.034292] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=248, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:58.034307] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=425, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:58.034320] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:58.034539] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40057-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:58.034547] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40057-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=965, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:58.047114] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:58.047129] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:58.049700] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061878048724000, start_id=1729061878048724000, end_id=1729061878048724001) [2024-10-16 14:57:58.050816] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:58.057659] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F18-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:58.057688] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F18-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:58.057694] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F18-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18619, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:58.057698] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F18-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=276, avg_log_submit_to_slide_time=18613) [2024-10-16 14:57:58.057709] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:58.057893] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:58.063304] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061878062469000, start_id=1729061878062469000, end_id=1729061878062469001) [2024-10-16 14:57:58.063327] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=556, get_gts_with_stc_cnt=32, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=30, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:58.063336] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:58.063355] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=531, get_gts_with_stc_cnt=8, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:58.063362] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061878063349], gts:1729061878062469000, latest_srr:[mts=1729061878063349]}) [2024-10-16 14:57:58.073543] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78B1-0-0] [lt=19] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1817) [2024-10-16 14:57:58.073788] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:58.078327] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.081433] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=14) [2024-10-16 14:57:58.085451] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D97-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:58.085644] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:58.087761] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.090045] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=22] new mysql sessid created(conn.sessid_=3221700075, support_ssl=false) [2024-10-16 14:57:58.090072] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=11] sm conn init succ(conn.sessid_=3221700075, sess.client_addr_="10.0.4.7:53448") [2024-10-16 14:57:58.090080] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0e5cdcc030, session_id:3221700075, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:58.090163] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=11] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:57:58.090168] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:295) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=5] mysql login queue(mysql_queue->queue_.size()=1) [2024-10-16 14:57:58.090852] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B7-0-0] [lt=10] MySQL LOGIN(direct_client_ip="10.0.4.7", client_ip=10.0.4.7, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=%, sessid=3221700075, proxy_sessid=13882479996272574487, client_sessid=2107375639, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=4062023, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:57:58.091032] INFO [SERVER] process (obmp_init_db.cpp:53) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85C6-0-0] [lt=10] init db(db_name=oceanbase) [2024-10-16 14:57:58.097874] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [22469][T1003_SSTableGC][T1003][YB4A0A000407-00061E997A7CB787-0-0] [lt=13] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60476}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015306454154001}, old_min_reserved_snapshot=1729060067249743000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060067249743000) [2024-10-16 14:57:58.108340] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [22469][T1003_SSTableGC][T1003][YB4A0A000407-00061E997A7CB787-0-0] [lt=15] SSTableGCTask(cost_ts=10623) [2024-10-16 14:57:58.124405] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=17] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97898426368) [2024-10-16 14:57:58.127934] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:58.127952] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:58.134366] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=188, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:58.134384] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=47, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:58.137672] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4054, mem_used=8318976) [2024-10-16 14:57:58.143108] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=30] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 156.25; ) [2024-10-16 14:57:58.143128] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:58.143166] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 149.75; ) [2024-10-16 14:57:58.143174] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:58.143185] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:58.147459] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:58.147656] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44D7-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:58.147665] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44D7-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=615, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:58.147758] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F42F4-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=13, avg_log_submit_to_flush_time=296, avg_log_submit_to_slide_time=566) [2024-10-16 14:57:58.149682] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:57:58.157259] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA6F-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061878156577000, start_id=1729061878156577000, end_id=1729061878156577001) [2024-10-16 14:57:58.157297] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA70-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061878156577000, start_id=1729061878156577000, end_id=1729061878156577001) [2024-10-16 14:57:58.157448] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA73-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061878156577000, start_id=1729061878156577000, end_id=1729061878156577001) [2024-10-16 14:57:58.157470] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0CFC-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061878156577000, start_id=1729061878156577000, end_id=1729061878156577001) [2024-10-16 14:57:58.157548] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0CFD-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061878156577000, start_id=1729061878156577000, end_id=1729061878156577001) [2024-10-16 14:57:58.158162] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:57:58.165019] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF25F-0-0] [lt=18] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.108497524261474609e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.002399163312093151e+03) [2024-10-16 14:57:58.165178] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061878165018000, start_id=1729061878165018000, end_id=1729061878165018001) [2024-10-16 14:57:58.165245] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:58.165261] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:58.169627] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061878168178000, start_id=1729061878168178000, end_id=1729061878168178001) [2024-10-16 14:57:58.175856] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33576, start_service_tstamp_=1722494263708085, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:58.178351] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.179029] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109203245, max_lsn:17959722162, max_scn:{val:1729061878062469000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7385580775}, reuse_lsn:{lsn:17959722162}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17917779122}}, last_submit_lsn:{lsn:17959722040}, last_submit_end_lsn:{lsn:17959722162}, last_submit_log_id:109203245, last_submit_log_pid:246, max_flushed_lsn:{lsn:17959722040}, max_flushed_end_lsn:{lsn:17959722162}, max_flushed_log_pid:246, committed_end_lsn:{lsn:17959722162}, last_slide_log_id:109203245, last_slide_scn:{val:1729061878062469000, v:0}, last_slide_lsn:{lsn:17959722040}, last_slide_end_lsn:{lsn:17959722162}, last_slide_log_pid:246, last_slide_log_accum_checksum:83511231, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1d4530}, [StateMgr]={this:0x2b0e0f1d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109203246, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261839590, palf_role_change_cb:0x2b0e0f1df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, reconfig_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, persistent_config_version:{proposal_id:246, config_seq:121}, ms_ack_list:0, resend_config_version:{proposal_id:246, config_seq:121}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258131745, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258131745, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262039736, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728994126835081001, v:0}, min_block_id:265, min_block_min_scn:{val:1728952113212346001, v:0}, base_lsn_for_block_gc:{lsn:17849868288}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, log_config_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, log_snapshot_meta:{version:1, base_lsn:{lsn:17849868288}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1798144}, readable_log_tail:{lsn:1798144}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17984077824}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/meta", dir_fd:167, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65306624, block_header_serialize_buf_:0x2b0e0f1da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17959722162}, readable_log_tail:{lsn:17959722162}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17916973056}, min_scn:{val:1729035023717936004, v:0}, curr_block_id:267, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/log", dir_fd:169, min_block_id:265, max_block_id:268, curr_writable_block_id:267}, logical_block_size_:67104768, curr_block_writable_size_:24355662, block_header_serialize_buf_:0x2b0e0f1dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261838799, this:0x2b0e0f1d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:246, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:245, majority_max_lsn:{lsn:7385763025}, saved_end_lsn:{lsn:7385763025}, last_submit_prepare_req_time_us:1722494261839744, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1d89b0}) [2024-10-16 14:57:58.179161] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=129] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878147474, last_flushed_end_lsn:{lsn:17959722162}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878147631, last_flushed_end_lsn:{lsn:17959722162}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878147685, last_flushed_end_lsn:{lsn:17959722162}}]) [2024-10-16 14:57:58.179173] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=109203246, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878147474, last_flushed_end_lsn:{lsn:17959722162}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878147631, last_flushed_end_lsn:{lsn:17959722162}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878147685, last_flushed_end_lsn:{lsn:17959722162}}]) [2024-10-16 14:57:58.187213] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061878187211, bytes: 0, bw: 0.000000 MB/s, add_ts: 1006914, add_bytes: 0 [2024-10-16 14:57:58.187573] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3655) [2024-10-16 14:57:58.187807] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.199469] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503C-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:58.199481] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503C-0-0] [lt=11] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:58.201304] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:57:58.210245] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587F-0-0] [lt=23] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:57:58.210348] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E587F-0-0] [lt=23][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:58.221587] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061878220711000, start_id=1729061878220711000, end_id=1729061878220711001) [2024-10-16 14:57:58.228277] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061878228114000, start_id=1729061878228114000, end_id=1729061878228114001) [2024-10-16 14:57:58.234159] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:58.234198] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=32] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:58.234415] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:58.234629] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD401-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=688, avg_log_submit_to_slide_time=915) [2024-10-16 14:57:58.239270] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=52] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:213058260579}, scn={val:1729061878156577001, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:57:58.239364] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060070205428000) [2024-10-16 14:57:58.239379] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:213058260579}, scn_:{val:1729061878156577001, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061878229329, first_handle_ts_:1729061878239353, replay_cost_:25, retry_cost_:-1, read_log_buf_:0x2b0f38a84060, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061878238506, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058260635}, next_to_submit_scn_:{val:1729061878156577002, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058260523}, end_lsn:{lsn:213058260635}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061878156577001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061878156577001, v:0}, data_checksum:204679066, flag:0}}, init_mode_version:0, accumulate_checksum:488852603, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061878156577001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561443, miss_cnt:1, cache_read_size:152590700617, hit_ratio:"9.999999928346972045e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}, replay_queue_idx=17, ret=0) [2024-10-16 14:57:58.245886] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD10-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:58.247214] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:57:58.247228] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1342) [2024-10-16 14:57:58.249793] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061878249463000, start_id=1729061878249463000, end_id=1729061878249463001) [2024-10-16 14:57:58.251104] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:58.269475] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DB9-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510346137869, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410625, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:58.278411] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.278963] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB9E-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12239, add_timestamp:1729061878278955}) [2024-10-16 14:57:58.279022] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E59-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12239, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061878279009, sql_renew_time:1729061878279009, is_mark_fail:false}]) [2024-10-16 14:57:58.279039] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E59-0-0] [lt=17] success to process renew task(task={tenant_id:1, table_id:12239, add_timestamp:1729061878278955}, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061878279009, sql_renew_time:1729061878279009, is_mark_fail:false}]) [2024-10-16 14:57:58.283401] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB9E-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:57:58.285228] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=13] ls blacklist refresh finish(cost_time=6857) [2024-10-16 14:57:58.285299] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA890-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:58.285325] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA890-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061877657186005, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061877657186005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:58.285978] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] succ to submit log task to replay service(to_submit_lsn={lsn:14538508511}, to_submit_scn={val:1729061878157672001, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061878285950, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538508577}, next_to_submit_scn_:{val:1729061878157672002, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538508455}, end_lsn:{lsn:14538508577}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061878157672001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061878157672001, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:3365791753, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061878157672001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650380, miss_cnt:32, cache_read_size:8050437662, hit_ratio:"9.999994892292168869e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:58.286579] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA890-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061878285981000, start_id=1729061878285981000, end_id=1729061878285981001) [2024-10-16 14:57:58.287872] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.288991] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA890-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061878288756000, start_id=1729061878288756000, end_id=1729061878288756001) [2024-10-16 14:57:58.289053] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA890-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9437, avg_group_log_size=629) [2024-10-16 14:57:58.289062] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA890-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9437) [2024-10-16 14:57:58.289290] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=267, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:58.289303] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9437) [2024-10-16 14:57:58.289313] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=26, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:58.289333] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9437) [2024-10-16 14:57:58.289344] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:58.289558] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95F5-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:58.289564] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95F5-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=521, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:58.289568] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F95F5-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=315, avg_log_submit_to_slide_time=491) [2024-10-16 14:57:58.289639] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:58.295616] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [22470][T1003_Compactio][T1003][YB4A0A000407-00061E99742D0D6F-0-0] [lt=19] InfoPoolResizeTask(cost_ts=6) [2024-10-16 14:57:58.298664] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850ED-0-0] [lt=32] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:58.298683] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850ED-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:57:58.298690] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850ED-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:58.298703] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850ED-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497583, log_task={header:{begin_lsn:{lsn:207589567774}, end_lsn:{lsn:207589567896}, log_id:218497583, min_scn:{val:1729061878156577001, v:0}, max_scn:{val:1729061878156577001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589567652}, prev_proposal_id:81, committed_end_lsn:{lsn:207589567774}, data_checksum:940009244, accum_checksum:3705863086, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061878198590, freeze_ts:1729061878198590, submit_ts:1729061878198594, flushed_ts:1729061878201327, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:2733}, fs_cb_begin_ts=1729061878298696, log_life_time=100106) [2024-10-16 14:57:58.308358] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2055) [22468][T1003_MediumLoo][T1003][YB4A0A000407-00061E9973DF38E1-0-0] [lt=14] start schedule all tablet merge(merge_version=1729015201670298000, medium_ls_tablet_iter_={ls_idx:-1, ls_ids:[{id:1}], tablet_idx:0, tablet_ids_.count():0, schedule_tablet_cnt:0, max_batch_tablet_cnt:50000}) [2024-10-16 14:57:58.308522] INFO [STORAGE] get_adaptive_merge_reason (ob_partition_merge_policy.cpp:1751) [22468][T1003_MediumLoo][T1003][YB4A0A000407-00061E9973DF38E1-0-0] [lt=15] Check tablet adaptive merge reason(ret=0, ls_id={id:1}, tablet_id={id:60476}, reason=0, tablet_analyzer={tablet_stat:{ls_id:1, tablet_id:60476, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, total_tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_small_tenant:false, boost_factor:"1.000000000000000000e+00"}, crazy_medium_flag=false) [2024-10-16 14:57:58.313617] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:58.313630] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:57:58.313650] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=9] cannot_recycle_log_size statistics(cannot_recycle_log_size=373697423, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:58.319537] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:58.319608] WDIAG read_multi_row (ob_table_access_helper.h:238) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:57:58.320443] INFO delete_row (ob_table_access_helper.h:312) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=8] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1002, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061878319617) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061878319617) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:57:58.320500] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=11] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:57:58.320560] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=5] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:57:58.320606] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22221][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=3] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:57:58.329111] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [22468][T1003_MediumLoo][T1003][YB4A0A000407-00061E9973DF38E1-0-0] [lt=21] batch clear tablet stat in ls(ret=0, ls_id={id:1}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:57:58.329133] INFO [STORAGE.COMPACTION] after_schedule_tenant_medium (ob_tenant_tablet_scheduler.cpp:2161) [22468][T1003_MediumLoo][T1003][YB4A0A000407-00061E9973DF38E1-0-0] [lt=17] finish schedule all tablet merge(merge_version=1729015201670298000, schedule_stats_={schedule_dag_cnt:0, submit_clog_cnt:0, finish_cnt:930, wait_rs_validate_cnt:0}, tenant_merge_finish=true, merged_version_=1729015201670298000, is_scan_all_tablet_finish=true, time_guard=) [2024-10-16 14:57:58.329143] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:103) [22468][T1003_MediumLoo][T1003][YB4A0A000407-00061E9973DF38E1-0-0] [lt=9] MediumLoopTask(cost_ts=20809) [2024-10-16 14:57:58.329158] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22468][T1003_MediumLoo][T1003][YB4A0A000407-00061E9973DF38E2-0-0] [lt=7] MediumCheckTask(cost_ts=6) [2024-10-16 14:57:58.329334] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=535, mem_used=2079744) [2024-10-16 14:57:58.331262] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1157-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061878330616000, start_id=1729061878330616000, end_id=1729061878330616001) [2024-10-16 14:57:58.333148] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:58.334570] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:58.339758] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:58.339776] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28020300, tenant_id=1002) [2024-10-16 14:57:58.339791] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:58.347543] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=230, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:58.347559] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:58.347562] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:58.347574] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:58.358240] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:57:58.365254] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061878365010000, start_id=1729061878365010000, end_id=1729061878365010001) [2024-10-16 14:57:58.369759] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061878368179000, start_id=1729061878368179000, end_id=1729061878368179001) [2024-10-16 14:57:58.371133] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:57:58.371146] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54641860, tenant_id=1003) [2024-10-16 14:57:58.371156] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:58.373586] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78B7-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1929, avg_group_log_size=175) [2024-10-16 14:57:58.373805] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2051) [2024-10-16 14:57:58.378449] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.385710] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2D9D-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:58.385900] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=196, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:58.385917] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:58.385946] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:58.387932] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.391828] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=11] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:58.395394] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:57:58.418881] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:58.421903] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061878421569000, start_id=1729061878421569000, end_id=1729061878421569001) [2024-10-16 14:57:58.428410] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061878426798000, start_id=1729061878426798000, end_id=1729061878426798001) [2024-10-16 14:57:58.429093] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:57:58.429831] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601470, offset=9644484}, end_cursor:ObLogCursor{file_id=192, log_id=47601471, offset=9644768}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7336050, len:1596, log_data_len:284, seq:47601470, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:58.431544] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B4-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061878430636000, start_id=1729061878430636000, end_id=1729061878430636001) [2024-10-16 14:57:58.432346] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601471, offset=9644768}, end_cursor:ObLogCursor{file_id=192, log_id=47601472, offset=9645042}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db727e050, len:1312, log_data_len:274, seq:47601471, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:58.434112] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601472, offset=9645042}, end_cursor:ObLogCursor{file_id=192, log_id=47601473, offset=9645316}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7266050, len:1038, log_data_len:274, seq:47601472, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:58.434273] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=29] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601473, offset=9645316}, end_cursor:ObLogCursor{file_id=192, log_id=47601474, offset=9645590}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7332050, len:764, log_data_len:274, seq:47601473, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:58.434332] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=51] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599032, miss_cnt:9, cache_read_size:149777932581, hit_ratio:"9.999999345925674188e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:58.434379] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=75] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561445, miss_cnt:1, cache_read_size:152590700861, hit_ratio:"9.999999928346973155e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:58.447837] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=25, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:57:58.447855] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=17] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=699, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:57:58.447860] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:57:58.447865] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=-641, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:57:58.447870] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB TOTAL TIME](cur_stat_count=30, stat_interval=5000000, avg (count/cost)=83, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:57:58.450042] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061878449675000, start_id=1729061878449675000, end_id=1729061878449675001) [2024-10-16 14:57:58.451401] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:58.473675] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78B9-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:58.473956] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=185, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:58.473979] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:58.474333] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F42FF-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:58.474348] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F42FF-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9453, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:58.474347] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1929, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:58.474380] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=26] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=84, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:58.474397] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=84, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:58.478512] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.486561] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A90-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:58.486579] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A90-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:58.487994] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.504183] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257818, offset=39048837}, end_cursor:ObLogCursor{file_id=197, log_id=49257819, offset=39049120}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73d2050, len:2427, log_data_len:283, seq:49257818, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:58.506296] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB37F-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061878505591000, start_id=1729061878505591000, end_id=1729061878505591001) [2024-10-16 14:57:58.506405] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8FA-0-0] [lt=7] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:58.506426] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8FA-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:58.506444] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB8FA-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:58.507107] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257819, offset=39049120}, end_cursor:ObLogCursor{file_id=197, log_id=49257820, offset=39049402}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e44050, len:2144, log_data_len:282, seq:49257819, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:58.509808] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257820, offset=39049402}, end_cursor:ObLogCursor{file_id=197, log_id=49257821, offset=39049684}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73b6050, len:1862, log_data_len:282, seq:49257820, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:58.510528] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257821, offset=39049684}, end_cursor:ObLogCursor{file_id=197, log_id=49257822, offset=39049966}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2018050, len:1580, log_data_len:282, seq:49257821, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:58.510589] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650382, miss_cnt:32, cache_read_size:8050437906, hit_ratio:"9.999994892292332072e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:58.510621] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657121, miss_cnt:17, cache_read_size:8050185390, hit_ratio:"9.999997286821494802e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:58.511143] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:58.511874] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:58.511901] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=26] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:58.511921] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=16] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:58.511988] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061885590307000, latest_log_ts:{val:1729061868948338001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:58.512033] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=43] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:58.512061] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:57:58.512071] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061868948338001, v:0}) [2024-10-16 14:57:58.512074] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:57:58.512078] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:58.512084] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:57:58.512096] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:57:58.512107] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:58.512112] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:57:58.534078] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:57:58.534097] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:57:58.534123] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444614576, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:57:58.534458] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:58.534468] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:58.534700] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:58.547338] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:58.547358] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:58.547383] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:58.547391] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:58.547632] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:58.547642] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:58.547816] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44E2-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:58.547823] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44E2-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=576, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:58.549187] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=12] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:57:58.549209] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=20] get_min_unreplayed_log_info(lsn={lsn:14538508821}, scn={val:1729061878157672004, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061878510575, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538508821}, next_to_submit_scn_:{val:1729061878157672004, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538508699}, end_lsn:{lsn:14538508821}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061878157672003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061878157672003, v:0}, data_checksum:3348837161, flag:1}}, init_mode_version:0, accumulate_checksum:181173043, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061878157672003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650382, miss_cnt:32, cache_read_size:8050437906, hit_ratio:"9.999994892292332072e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:57:58.549260] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=47] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:57:58.558322] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:57:58.563706] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061878563342000, start_id=1729061878563342000, end_id=1729061878563342001) [2024-10-16 14:57:58.563892] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=17] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061878563882], gts:1729061878563342000, latest_srr:[mts=1729061878563882]}) [2024-10-16 14:57:58.565396] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061878564401000, start_id=1729061878564401000, end_id=1729061878564401001) [2024-10-16 14:57:58.568820] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.569948] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061878569727000, start_id=1729061878569727000, end_id=1729061878569727001) [2024-10-16 14:57:58.570041] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.570060] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.570075] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.570323] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10126) [2024-10-16 14:57:58.570519] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1729061878249463001, v:0}, new_max_version={val:1729061878249463001, v:0}, last_min_version={val:1729061878168178000, v:0}, last_max_version={val:1729061878168178000, v:0}, total_time_us=1408, delta_us=321055) [2024-10-16 14:57:58.570548] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1729061878249463001, v:0}, min_version={val:1729061878249463001, v:0}, max_version={val:1729061878249463001, v:0}, in_service=true, leader_epoch=33576) [2024-10-16 14:57:58.578571] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.579244] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:220938440, max_lsn:213058261123, max_scn:{val:1729061878156577005, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:60464363030}, reuse_lsn:{lsn:213058261123}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:213016318083}}, last_submit_lsn:{lsn:213058261001}, last_submit_end_lsn:{lsn:213058261123}, last_submit_log_id:220938440, last_submit_log_pid:88, max_flushed_lsn:{lsn:213058261001}, max_flushed_end_lsn:{lsn:213058261123}, max_flushed_log_pid:88, committed_end_lsn:{lsn:213058261123}, last_slide_log_id:220938440, last_slide_scn:{val:1729061878156577005, v:0}, last_slide_lsn:{lsn:213058261001}, last_slide_end_lsn:{lsn:213058261123}, last_slide_log_pid:88, last_slide_log_accum_checksum:1328520240, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0dac0530}, [StateMgr]={this:0x2b0e0dac4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, leader:"10.0.4.8:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:60468525403}, scan_disk_log_finished:true, last_check_start_id:220938441, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261639483, palf_role_change_cb:0x2b0e0dacb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, reconfig_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, persistent_config_version:{proposal_id:88, config_seq:21}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061875977817, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061875977817, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0dac1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, accepted_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1729009096591759001, v:0}, min_block_id:3154, min_block_min_scn:{val:1729006455168279002, v:0}, base_lsn_for_block_gc:{lsn:212587905024}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, log_config_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, log_snapshot_meta:{version:1, base_lsn:{lsn:212587905024}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}}, log_meta_storage:{log_tail:{lsn:5767168}, readable_log_tail:{lsn:5767168}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213124743168}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/meta", dir_fd:175, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61337600, block_header_serialize_buf_:0x2b0e0dac69f8, flashback_version:0}, log_storage:{log_tail:{lsn:213058261123}, readable_log_tail:{lsn:213058261123}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213057638400}, min_scn:{val:1729061839666529002, v:0}, curr_block_id:3175, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/log", dir_fd:177, min_block_id:3154, max_block_id:3176, curr_writable_block_id:3175}, logical_block_size_:67104768, curr_block_writable_size_:66482045, block_header_serialize_buf_:0x2b0e0dac86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261637707, this:0x2b0e0dac4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:87, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:86, majority_max_lsn:{lsn:60466038076}, saved_end_lsn:{lsn:60466038076}, last_submit_prepare_req_time_us:1722494261639634, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0dac49b0}) [2024-10-16 14:57:58.579402] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=154] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:58.579425] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=22] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=220938441, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:57:58.579446] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:211147463, max_lsn:204023121147, max_scn:{val:1729061878330616002, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57922170954}, reuse_lsn:{lsn:204023121147}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:203981178107}}, last_submit_lsn:{lsn:204023121025}, last_submit_end_lsn:{lsn:204023121147}, last_submit_log_id:211147463, last_submit_log_pid:130, max_flushed_lsn:{lsn:204023121025}, max_flushed_end_lsn:{lsn:204023121147}, max_flushed_log_pid:130, committed_end_lsn:{lsn:204023121147}, last_slide_log_id:211147463, last_slide_scn:{val:1729061878330616002, v:0}, last_slide_lsn:{lsn:204023121025}, last_slide_end_lsn:{lsn:204023121147}, last_slide_log_pid:130, last_slide_log_accum_checksum:40390665, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1ec530}, [StateMgr]={this:0x2b0e0f1f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:211147464, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261539368, palf_role_change_cb:0x2b0e0f1f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, reconfig_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, persistent_config_version:{proposal_id:130, config_seq:17}, ms_ack_list:0, resend_config_version:{proposal_id:130, config_seq:17}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258231773, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258231773, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, accepted_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261739861, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1729008439619722001, v:0}, min_block_id:3020, min_block_min_scn:{val:1729005678665693002, v:0}, base_lsn_for_block_gc:{lsn:203595866112}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, log_config_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, log_snapshot_meta:{version:1, base_lsn:{lsn:203595866112}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}}, log_meta_storage:{log_tail:{lsn:6397952}, readable_log_tail:{lsn:6397952}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:204065599488}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/meta", dir_fd:145, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:60706816, block_header_serialize_buf_:0x2b0e0f1f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:204023121147}, readable_log_tail:{lsn:204023121147}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:203998494720}, min_scn:{val:1729060878194301003, v:0}, curr_block_id:3040, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/log", dir_fd:173, min_block_id:3020, max_block_id:3041, curr_writable_block_id:3040}, logical_block_size_:67104768, curr_block_writable_size_:42478341, block_header_serialize_buf_:0x2b0e0f1f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261539352, this:0x2b0e0f1f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:130, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:129, majority_max_lsn:{lsn:57923794664}, saved_end_lsn:{lsn:57923794664}, last_submit_prepare_req_time_us:1722494261539574, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1f09b0}) [2024-10-16 14:57:58.579541] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=93] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878547664, last_flushed_end_lsn:{lsn:204023121147}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878547788, last_flushed_end_lsn:{lsn:204023121147}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878548027, last_flushed_end_lsn:{lsn:204023121147}}]) [2024-10-16 14:57:58.579549] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=211147464, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878547664, last_flushed_end_lsn:{lsn:204023121147}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878547788, last_flushed_end_lsn:{lsn:204023121147}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878548027, last_flushed_end_lsn:{lsn:204023121147}}]) [2024-10-16 14:57:58.579565] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:218497586, max_lsn:207589568262, max_scn:{val:1729061878156577004, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57807896114}, reuse_lsn:{lsn:207589568262}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:207547625222}}, last_submit_lsn:{lsn:207589568140}, last_submit_end_lsn:{lsn:207589568262}, last_submit_log_id:218497586, last_submit_log_pid:81, max_flushed_lsn:{lsn:207589568140}, max_flushed_end_lsn:{lsn:207589568262}, max_flushed_log_pid:81, committed_end_lsn:{lsn:207589568262}, last_slide_log_id:218497586, last_slide_scn:{val:1729061878156577004, v:0}, last_slide_lsn:{lsn:207589568140}, last_slide_end_lsn:{lsn:207589568262}, last_slide_log_pid:81, last_slide_log_accum_checksum:2691952799, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0db32530}, [StateMgr]={this:0x2b0e0db36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, leader:"10.0.4.9:2890", leader_epoch:7, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:57814812323}, scan_disk_log_finished:true, last_check_start_id:218497587, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433991656, palf_role_change_cb:0x2b0e0db3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, reconfig_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, persistent_config_version:{proposal_id:81, config_seq:20}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061876978366, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061876978366, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0db33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, accepted_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1729009518284639001, v:0}, min_block_id:3073, min_block_min_scn:{val:1729006819670466001, v:0}, base_lsn_for_block_gc:{lsn:207152418816}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, log_config_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, log_snapshot_meta:{version:1, base_lsn:{lsn:207152418816}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}}, log_meta_storage:{log_tail:{lsn:5660672}, readable_log_tail:{lsn:5660672}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207622152192}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/meta", dir_fd:180, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61444096, block_header_serialize_buf_:0x2b0e0db389f8, flashback_version:0}, log_storage:{log_tail:{lsn:207589568262}, readable_log_tail:{lsn:207589568262}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207555047424}, min_scn:{val:1729060516440786001, v:0}, curr_block_id:3093, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/log", dir_fd:182, min_block_id:3073, max_block_id:3094, curr_writable_block_id:3093}, logical_block_size_:67104768, curr_block_writable_size_:32583930, block_header_serialize_buf_:0x2b0e0db3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433990608, this:0x2b0e0db36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:80, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:79, majority_max_lsn:{lsn:57811903042}, saved_end_lsn:{lsn:57811903042}, last_submit_prepare_req_time_us:1722494433991756, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0db369b0}) [2024-10-16 14:57:58.579670] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=104] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:58.579680] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=218497587, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:57:58.581857] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115178625, max_lsn:14538508821, max_scn:{val:1729061878157672003, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6487749074}, reuse_lsn:{lsn:14538508821}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14496565781}}, last_submit_lsn:{lsn:14538508699}, last_submit_end_lsn:{lsn:14538508821}, last_submit_log_id:115178625, last_submit_log_pid:23, max_flushed_lsn:{lsn:14538508699}, max_flushed_end_lsn:{lsn:14538508821}, max_flushed_log_pid:23, committed_end_lsn:{lsn:14538508821}, last_slide_log_id:115178625, last_slide_scn:{val:1729061878157672003, v:0}, last_slide_lsn:{lsn:14538508699}, last_slide_end_lsn:{lsn:14538508821}, last_slide_log_pid:23, last_slide_log_accum_checksum:181173043, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43cc0530}, [StateMgr]={this:0x2b0e43cc4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, leader:"10.0.4.8:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6488080380}, scan_disk_log_finished:true, last_check_start_id:115178626, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261653045, palf_role_change_cb:0x2b0e43ccb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, reconfig_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, persistent_config_version:{proposal_id:23, config_seq:34}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061877881374, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061877881374, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43cc1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, accepted_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1728259580444569001, v:0}, min_block_id:201, min_block_min_scn:{val:1728204893605525001, v:0}, base_lsn_for_block_gc:{lsn:14427525120}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, log_config_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, log_snapshot_meta:{version:1, base_lsn:{lsn:14427525120}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}}, log_meta_storage:{log_tail:{lsn:1134592}, readable_log_tail:{lsn:1134592}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14561734656}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/meta", dir_fd:166, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65970176, block_header_serialize_buf_:0x2b0e43cc69f8, flashback_version:0}, log_storage:{log_tail:{lsn:14538508821}, readable_log_tail:{lsn:14538508821}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14494629888}, min_scn:{val:1729026040007117003, v:0}, curr_block_id:216, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/log", dir_fd:201, min_block_id:201, max_block_id:217, curr_writable_block_id:216}, logical_block_size_:67104768, curr_block_writable_size_:23225835, block_header_serialize_buf_:0x2b0e43cc86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261652788, this:0x2b0e43cc4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:22, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:21, majority_max_lsn:{lsn:6487893593}, saved_end_lsn:{lsn:6487893593}, last_submit_prepare_req_time_us:1722494261653224, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43cc49b0}) [2024-10-16 14:57:58.581979] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=119] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:58.581988] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=8] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=115178626, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:57:58.585680] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DA1-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:58.586153] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9277-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:58.586161] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9277-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9285, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:58.588052] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.594718] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD017-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061878585311, keepalive_ts:-1}) [2024-10-16 14:57:58.594740] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD017-0-0] [lt=22] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061878585311, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:58.594758] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD018-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061878585355, keepalive_ts:-1}) [2024-10-16 14:57:58.594764] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD018-0-0] [lt=6] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061878585355, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:58.596875] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:58.596892] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:58.597593] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1002}, {id:1003}]) [2024-10-16 14:57:58.599006] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=258, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:58.599019] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=419, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:58.599049] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:58.599269] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54FF-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:58.599276] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D54FF-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=26213, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:58.610864] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5692, mem_used=10398720) [2024-10-16 14:57:58.622002] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061878621395000, start_id=1729061878621395000, end_id=1729061878621395001) [2024-10-16 14:57:58.628499] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061878626610000, start_id=1729061878626610000, end_id=1729061878626610001) [2024-10-16 14:57:58.650016] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061878649822000, start_id=1729061878649822000, end_id=1729061878649822001) [2024-10-16 14:57:58.650325] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=287, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:58.650341] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=36, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:58.650360] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:58.650610] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797D6D-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:58.650618] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797D6D-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1054, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:58.651738] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:58.657653] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA74-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061878657166000, start_id=1729061878657166000, end_id=1729061878657166001) [2024-10-16 14:57:58.657948] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D01-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061878657166000, start_id=1729061878657166000, end_id=1729061878657166001) [2024-10-16 14:57:58.658012] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA75-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061878657166000, start_id=1729061878657166000, end_id=1729061878657166001) [2024-10-16 14:57:58.658039] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA77-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061878657166000, start_id=1729061878657166000, end_id=1729061878657166001) [2024-10-16 14:57:58.658114] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA78-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061878657166000, start_id=1729061878657166000, end_id=1729061878657166001) [2024-10-16 14:57:58.658440] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=206, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:58.658460] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=73, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:58.658489] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:58.668188] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:57:58.668206] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:57:58.668214] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=7] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:57:58.668220] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:58.670359] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=5] Dumping All Cache Objs(alloc_obj_list.count()=108, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}, {obj_id:616254, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616255, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616256, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:616257, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}]) [2024-10-16 14:57:58.670502] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=140] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:57:58.671555] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061878671002000, start_id=1729061878671002000, end_id=1729061878671002001) [2024-10-16 14:57:58.673007] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:58.678618] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.681916] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109593889, max_lsn:13645439197, max_scn:{val:1729061878563342000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6198843846}, reuse_lsn:{lsn:13645439197}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:13603496157}}, last_submit_lsn:{lsn:13645439075}, last_submit_end_lsn:{lsn:13645439197}, last_submit_log_id:109593889, last_submit_log_pid:17, max_flushed_lsn:{lsn:13645439075}, max_flushed_end_lsn:{lsn:13645439197}, max_flushed_log_pid:17, committed_end_lsn:{lsn:13645439197}, last_slide_log_id:109593889, last_slide_scn:{val:1729061878563342000, v:0}, last_slide_lsn:{lsn:13645439075}, last_slide_end_lsn:{lsn:13645439197}, last_slide_log_pid:17, last_slide_log_accum_checksum:1251658779, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431ec530}, [StateMgr]={this:0x2b0e431f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109593890, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261552988, palf_role_change_cb:0x2b0e431f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:14, lock_type:0, added_list:0, removed_list:0, type:"UNLOCK_CONFIG_CHANGE"}, state:0, checking_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:30}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:30}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494257649102, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494257649102, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261753456, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1728296971089337004, v:0}, min_block_id:189, min_block_min_scn:{val:1728240898542283004, v:0}, base_lsn_for_block_gc:{lsn:13555163136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:13555163136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}}, log_meta_storage:{log_tail:{lsn:1011712}, readable_log_tail:{lsn:1011712}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13689372672}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/meta", dir_fd:198, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66093056, block_header_serialize_buf_:0x2b0e431f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:13645439197}, readable_log_tail:{lsn:13645439197}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13622267904}, min_scn:{val:1729042537979987002, v:0}, curr_block_id:203, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/log", dir_fd:200, min_block_id:189, max_block_id:204, curr_writable_block_id:203}, logical_block_size_:67104768, curr_block_writable_size_:43933475, block_header_serialize_buf_:0x2b0e431f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261552391, this:0x2b0e431f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:6198988121}, saved_end_lsn:{lsn:6198988121}, last_submit_prepare_req_time_us:1722494261553147, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431f09b0}) [2024-10-16 14:57:58.682090] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=172] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878634705, last_flushed_end_lsn:{lsn:13645439197}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878634851, last_flushed_end_lsn:{lsn:13645439197}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878635005, last_flushed_end_lsn:{lsn:13645439197}}]) [2024-10-16 14:57:58.682103] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=109593890, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878634705, last_flushed_end_lsn:{lsn:13645439197}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878634851, last_flushed_end_lsn:{lsn:13645439197}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061878635005, last_flushed_end_lsn:{lsn:13645439197}}]) [2024-10-16 14:57:58.686256] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9279-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=222, avg_log_submit_to_slide_time=10157) [2024-10-16 14:57:58.686297] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:58.688117] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.690277] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=23] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:57:58.691079] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061878690983000, start_id=1729061878690983000, end_id=1729061878690983001) [2024-10-16 14:57:58.699955] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:57:58.702517] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D4-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:58.702542] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D4-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061878156577005, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061878156577005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:58.705880] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D4-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061878705797000, start_id=1729061878705797000, end_id=1729061878705797001) [2024-10-16 14:57:58.705945] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D4-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10925, avg_group_log_size=606) [2024-10-16 14:57:58.705955] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D4-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10719) [2024-10-16 14:57:58.706181] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10170) [2024-10-16 14:57:58.707167] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DBB-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510346575560, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410626, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:58.707321] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797D6F-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=867, avg_log_submit_to_slide_time=1067) [2024-10-16 14:57:58.707428] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:58.707960] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DBF-0-0] [lt=17][errcode=-4201] hashtable not init [2024-10-16 14:57:58.707969] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DBF-0-0] [lt=8][errcode=-4016] get cursor info failed(cursor_id=1118, get_sessid()=3221565766) [2024-10-16 14:57:58.708050] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061878707939000, start_id=1729061878707939000, end_id=1729061878707939001) [2024-10-16 14:57:58.724499] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=20] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:58.725164] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22859][T1004_L0_G2][T1004][YB4A0A000409-00061E9A39C4D8CA-0-0] [lt=52] get_min_unreplayed_log_info(lsn={lsn:14352395413}, scn={val:1729061878156577006, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061878658659, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352395413}, next_to_submit_scn_:{val:1729061878156577006, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352395291}, end_lsn:{lsn:14352395413}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061878156577005, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061878156577005, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:15941971, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061878156577005, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657123, miss_cnt:17, cache_read_size:8050185634, hit_ratio:"9.999997286821581399e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:57:58.727700] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:58.729430] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15EA-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:58.729514] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15EC-0-0] [lt=16] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:58.735751] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:57:58.735761] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:57:58.735765] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:57:58.735781] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=2] get_max_applied_scn(scn={val:1729061878563342000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493494990}, palf_committed_end_scn_:{val:1729061878690983000, v:0}, last_check_scn_:{val:1729061878690983000, v:0}, max_applied_cb_scn_:{val:1729061878563342000, v:0}}) [2024-10-16 14:57:58.735797] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:57:58.736431] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=51] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890721265000, latest_log_ts:{val:1729061874059797001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:58.736501] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=39] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1758, avg_group_log_size=159) [2024-10-16 14:57:58.736511] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1767) [2024-10-16 14:57:58.736524] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] get_max_applied_scn(scn={val:1729061878563342000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645439197}, palf_committed_end_scn_:{val:1729061878563342000, v:0}, last_check_scn_:{val:1729061878563342000, v:0}, max_applied_cb_scn_:{val:1729061878563342000, v:0}}) [2024-10-16 14:57:58.736535] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:57:58.737148] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=31] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:58.737787] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=30] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:58.738434] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=37] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:57:58.738786] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1758) [2024-10-16 14:57:58.738899] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=3752431, this={dio_aligned_buf:{buf_write_offset:1313, buf_padding_size:2783, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80962921, truncate_used_ts:29163583}, log_block_size:67108864, dir_fd:148, io_fd:171}, aligned_buf_len=4096, aligned_block_offset=3751936, buf_len=818, total_write_size_=102424, total_write_size_after_dio_=827392, ob_pwrite_used_ts=43618, count=177, each_pwrite_cost=246) [2024-10-16 14:57:58.746996] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=14] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:57:58.756818] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C61-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:58.756851] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C61-0-0] [lt=21] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061878657166000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061878657166000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:58.757863] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C61-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061878757571000, start_id=1729061878757571000, end_id=1729061878757571001) [2024-10-16 14:57:58.758398] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:57:58.759117] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:58.759137] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:58.759144] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=5] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:58.760779] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C61-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061878760740000, start_id=1729061878760740000, end_id=1729061878760740001) [2024-10-16 14:57:58.765406] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061878764932000, start_id=1729061878764932000, end_id=1729061878764932001) [2024-10-16 14:57:58.772673] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:58.778676] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.782562] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=12] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:58.788181] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.788182] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.788198] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.788209] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.788589] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.788592] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.788613] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.788627] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.788817] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.789745] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.789802] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.789817] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:58.819906] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:64, free_obj_hold_cnt:141, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:57:58.822964] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061878822342000, start_id=1729061878822342000, end_id=1729061878822342001) [2024-10-16 14:57:58.823033] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:58.823048] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=4566, avg_group_log_size=415) [2024-10-16 14:57:58.823068] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:58.823403] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=282, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:58.823420] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=33, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:58.823438] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:58.823631] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10D4-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=336, avg_log_submit_to_slide_time=527) [2024-10-16 14:57:58.823802] INFO [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:194) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get wrs ts(ls_id={id:1}, delta=115515, log_service_ts=1729061878707939, min_tx_service_ts=4611686018427387, timestamp={val:1729061878707939000, v:0}) [2024-10-16 14:57:58.823820] INFO [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:214) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=0, Near_To_GTS_Cnt=54, Other_Error_Cnt=0, Submit_Succ_Cnt=46, last_scn="{val:1729061875560687000, v:0}", last_lsn={lsn:66390179283}, last_gts={val:1729061878707939000, v:0}, min_start_scn="{val:0, v:0}", min_start_status=1, sys_ls_end_scn={val:0, v:0}) [2024-10-16 14:57:58.828634] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061878827762000, start_id=1729061878827762000, end_id=1729061878827762001) [2024-10-16 14:57:58.831773] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1003}, {id:1001}]) [2024-10-16 14:57:58.847836] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=240, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:58.847862] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=44, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:58.847877] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:58.848167] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44EB-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=304, avg_log_submit_to_slide_time=545) [2024-10-16 14:57:58.850282] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061878849973000, start_id=1729061878849973000, end_id=1729061878849973001) [2024-10-16 14:57:58.852013] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:58.858123] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F28-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:58.858148] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F28-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:58.858164] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F28-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173322, log_task={header:{begin_lsn:{lsn:14352395413}, end_lsn:{lsn:14352395535}, log_id:115173322, min_scn:{val:1729061878657166000, v:0}, max_scn:{val:1729061878657166000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352395291}, prev_proposal_id:17, committed_end_lsn:{lsn:14352395413}, data_checksum:940009244, accum_checksum:1580464124, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061878758107, freeze_ts:1729061878758108, submit_ts:1729061878758111, flushed_ts:1729061878759345, gen_to_freeze cost time:1, gen_to_submit cost time:4, submit_to_flush cost time:1234}, fs_cb_begin_ts=1729061878858157, log_life_time=100050) [2024-10-16 14:57:58.861526] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5938, mem_used=12478464) [2024-10-16 14:57:58.870663] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4891-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061878838313, keepalive_ts:-1}) [2024-10-16 14:57:58.870680] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4891-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061878838313, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:58.871517] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061878871000000, start_id=1729061878871000000, end_id=1729061878871000001) [2024-10-16 14:57:58.878740] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.886389] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=40, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:58.886404] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:58.888241] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.892029] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdcc030, session_id:3221700075, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061878091319, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:991, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:58.892052] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=17] kill and revert session(conn.sessid_=3221700075, proxy_sessid=13882479996272574487, server_id=1, ret=0) [2024-10-16 14:57:58.892059] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] sock ref clean, do destroy(*s={this:0x2b0e5cdcc030, session_id:3221700075, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061878091319, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:991, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:57:58.892071] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221700075, proxy_sessid=13882479996272574487, conn.tenant_id_=1) [2024-10-16 14:57:58.892076] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221700075, proxy_sessid=13882479996272574487, tenant_id=1, server_id=1, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B34FA85C9-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:11, cur_pkt_pos_:17, pkt_rec[0]:{send:obp_mysql_header_{len_:12134, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{send:obp20_header_:{payload_len:14, pkt_seq:0, req_id:13452658, flag_.flags:42, ext_flags:0}, obp_mysql_header_{len_:554, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[2]:{send:obp20_header_:{payload_len:43, pkt_seq:0, req_id:13452659, flag_.flags:42, ext_flags:0}, obp_mysql_header_{len_:447, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{receive:obp20_header_:{payload_len:162, pkt_seq:0, req_id:13452660, flag_.flags:58, ext_flags:0}, obp_mysql_header_{len_:158, rec_:162, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[4]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452660, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452660, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452660, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452660, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[8]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:13452660, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:35, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:13452660, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:12, seq_:14}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[10]:{receive:obp20_header_:{payload_len:261, pkt_seq:0, req_id:13452661, flag_.flags:62, ext_flags:0}, obp_mysql_header_{len_:257, rec_:261, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[11]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452661, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[12]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452661, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[13]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452661, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[14]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452661, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[15]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452661, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[16]:{send:obp_compess_header_{com_len:568, com_seq:1}, obp20_header_:{payload_len:540, pkt_seq:1, req_id:13452661, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:12, seq_:14}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:57:58.892134] INFO [SERVER] run (obmp_disconnect.cpp:82) [21654][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=11] free session successfully(sessid=3221700075) [2024-10-16 14:57:58.892144] INFO [SERVER] run (obmp_disconnect.cpp:89) [21654][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=10] mark session id unused(sessid=3221700075) [2024-10-16 14:57:58.899055] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE850F9-0-0] [lt=13][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497589, log_task={header:{begin_lsn:{lsn:207589568506}, end_lsn:{lsn:207589568628}, log_id:218497589, min_scn:{val:1729061878657166002, v:0}, max_scn:{val:1729061878657166002, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589568384}, prev_proposal_id:81, committed_end_lsn:{lsn:207589568506}, data_checksum:940009244, accum_checksum:3225551543, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061878798952, freeze_ts:1729061878798953, submit_ts:1729061878798957, flushed_ts:1729061878799854, gen_to_freeze cost time:1, gen_to_submit cost time:5, submit_to_flush cost time:897}, fs_cb_begin_ts=1729061878899048, log_life_time=100096) [2024-10-16 14:57:58.899103] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:58.903529] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D4E-0-0] [lt=17] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061878894663, keepalive_ts:-1}) [2024-10-16 14:57:58.903545] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D4E-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061878894663, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:58.903562] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D4D-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061878894624, keepalive_ts:-1}) [2024-10-16 14:57:58.903567] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D4D-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061878894624, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:58.910715] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=60] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:58.911306] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=13] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:58.911359] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=12] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:58.911352] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061877257554000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:57:58.924591] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:58.925630] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=9] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:58.935928] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=38] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:58.935946] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:58.936200] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:58.936748] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40074-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:58.936767] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40074-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=520, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:58.936771] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40074-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=240, avg_log_submit_to_slide_time=505) [2024-10-16 14:57:58.939692] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBEB-0-0] [lt=27] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061878907427, keepalive_ts:-1}) [2024-10-16 14:57:58.939707] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBEB-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061878907427, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:58.945137] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3842, mem_used=8318976) [2024-10-16 14:57:58.958487] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:57:58.965683] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] submit log success(service_type_=0, last_id=1729061895590307000, limited_id=1729061895590307000) [2024-10-16 14:57:58.965694] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] submit log with lock success(service_type_=0, last_id=1729061895590307000, limited_id=1729061895590307000) [2024-10-16 14:57:58.965708] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061878965510000, start_id=1729061878965510000, end_id=1729061878965510001) [2024-10-16 14:57:58.966211] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] update id meta(ret=0, service_type=0, limited_id=1729061895590307000, latest_scn={val:1729061878965510001, v:0}, *this={tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061895590307000, latest_log_ts:{val:1729061878965510001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:57:58.966246] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=34] handle submit callback(service_type_=0, success=true, used_ts=563, log_ts={val:1729061878965510001, v:0}, limited_id=1729061895590307000) [2024-10-16 14:57:58.966253] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:57:58.966258] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:57:58.966338] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061878965510000, start_id=1729061878965510001, end_id=1729061878965510002) [2024-10-16 14:57:58.966346] INFO [STORAGE.TRANS] wait_gts_elapse (ob_gts_source.cpp:434) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] wait queue push task success(task=0x2b0df706d9b0) [2024-10-16 14:57:58.966362] INFO [STORAGE.TRANS] wait_gts_elapse_commit_version_ (ob_trans_part_ctx.cpp:3072) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=8] need wait gts elapse(ret=0, ret="OB_SUCCESS", *this={this:0x2b0df706c510, ref:3, trans_id:{txid:193381689}, tenant_id:1001, is_exiting:false, trans_expired_time:1729061908964440, cluster_version:17180066304, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=1729061878965631], ctx_create_time:1729061878964452}{self_ls_id:{id:1}, session_id:1, part_trans_action:3, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x2b0e08204030, tx_data_guard_:{tx_data:{tx_id:{txid:193381689}, ref_cnt:1, state:"ELR_COMMIT", commit_version:{val:1729061878965510002, v:0}, start_scn:{val:1729061878965510002, v:0}, end_scn:{val:1729061878965510002, v:0}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:263984625186510464, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:1729061878965510002, v:0}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1729061878965510, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1717560273711016 max_table_version=1717560273711016 trans_version={val:1729061878965510000, v:0} commit_version={val:1729061878965510002, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:193381689} ls_id=1 row_callback[alloc:1, free:0, unsubmit:0] redo[fill:2,sync_succ:2, sync_fail:0] main_list_len=2 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,2,2,2,2,0,0,0)|]}} -{busy_cbs_cnt:1, oldest_busy_cb:{ObTxBaseLogCb:{base_ts:{val:1729061878965510000, v:0}, log_ts:{val:1729061878965510002, v:0}, lsn:{lsn:114618695539}, submit_ts:1729061878965730}, this:0x2b0df706fe80, is_inited_:true, trans_id:{txid:193381689}, ls_id:{id:1}, ctx:0x2b0df706c510, tx_data_guard:{tx_data:NULL}, is_callbacked_:false, is_dynamic_:false, mds_range_:{range_array_.count():0, range_array_:[]}, cb_arg_array_:[{log_type:0x1, arg:null}, {log_type:0x40, arg:null}, {log_type:0x100, arg:null}], first_part_scn_:{val:18446744073709551615, v:3}, extra_cb_:null, need_free_extra_cb_:false, callbacks_.count():1}, final_log_cb:{ObTxBaseLogCb:{base_ts:{val:1729061878965510000, v:0}, log_ts:{val:1729061878965510002, v:0}, lsn:{lsn:114618695539}, submit_ts:1729061878965730}, this:0x2b0df706fe80, is_inited_:true, trans_id:{txid:193381689}, ls_id:{id:1}, ctx:0x2b0df706c510, tx_data_guard:{tx_data:NULL}, is_callbacked_:false, is_dynamic_:false, mds_range_:{range_array_.count():0, range_array_:[]}, cb_arg_array_:[{log_type:0x1, arg:null}, {log_type:0x40, arg:null}, {log_type:0x100, arg:null}], first_part_scn_:{val:18446744073709551615, v:3}, extra_cb_:null, need_free_extra_cb_:false, callbacks_.count():1}, -{info_log_submitted:1 gts_waiting:1 state_log_submitted:1 }-, -{downstream_state:"REDO_COMPLETE", upstream:{id:-1}, participants:[{id:1}], redo_log_no:0, scheduler:"10.0.4.7:2890", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:1, max_applied_log_ts:{val:1729061878965510002, v:0}, max_appling_log_ts:{val:1729061878965510002, v:0}, max_applying_part_log_no:0, max_submitted_seq_no:{branch:0, seq:3}, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, max_durable_lsn:{lsn:114618695539}, commit_parts:[{ls_id:{id:1}, exec_epoch:263984625186510464, transfer_epoch:-1}], }- }-}) [2024-10-16 14:57:58.971517] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7E-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:58.971546] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7E-0-0] [lt=21] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061878657166003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061878657166003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:58.972627] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D7E-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061878971870000, start_id=1729061878971870000, end_id=1729061878971870001) [2024-10-16 14:57:58.975214] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7E-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061878971870000, start_id=1729061878971870002, end_id=1729061878971870003) [2024-10-16 14:57:58.975273] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7E-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:58.975286] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7E-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:58.976797] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:57:58.976807] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=6] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:57:58.976815] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:58.976875] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=12] ====== tenant freeze timer task ====== [2024-10-16 14:57:58.978808] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.982076] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115173324, max_lsn:14352395779, max_scn:{val:1729061878657166002, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6301879685}, reuse_lsn:{lsn:14352395779}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14310452739}}, last_submit_lsn:{lsn:14352395657}, last_submit_end_lsn:{lsn:14352395779}, last_submit_log_id:115173324, last_submit_log_pid:17, max_flushed_lsn:{lsn:14352395657}, max_flushed_end_lsn:{lsn:14352395779}, max_flushed_log_pid:17, committed_end_lsn:{lsn:14352395779}, last_slide_log_id:115173324, last_slide_scn:{val:1729061878657166002, v:0}, last_slide_lsn:{lsn:14352395657}, last_slide_end_lsn:{lsn:14352395779}, last_slide_log_pid:17, last_slide_log_accum_checksum:3935016152, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43d32530}, [StateMgr]={this:0x2b0e43d36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.9:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6302436797}, scan_disk_log_finished:true, last_check_start_id:115173325, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433837169, palf_role_change_cb:0x2b0e43d3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, reconfig_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, persistent_config_version:{proposal_id:17, config_seq:26}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061876880590, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061876880590, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43d33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1728301961314447004, v:0}, min_block_id:199, min_block_min_scn:{val:1728247158357854003, v:0}, base_lsn_for_block_gc:{lsn:14293315584}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:14293315584}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}}, log_meta_storage:{log_tail:{lsn:1064960}, readable_log_tail:{lsn:1064960}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14360420352}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/meta", dir_fd:192, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66039808, block_header_serialize_buf_:0x2b0e43d389f8, flashback_version:0}, log_storage:{log_tail:{lsn:14352395779}, readable_log_tail:{lsn:14352395779}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14293315584}, min_scn:{val:1729013741169122001, v:0}, curr_block_id:213, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/log", dir_fd:207, min_block_id:199, max_block_id:214, curr_writable_block_id:213}, logical_block_size_:67104768, curr_block_writable_size_:8024573, block_header_serialize_buf_:0x2b0e43d3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433836510, this:0x2b0e43d36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:16, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:15, majority_max_lsn:{lsn:6302238429}, saved_end_lsn:{lsn:6302238429}, last_submit_prepare_req_time_us:1722494433837290, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43d369b0}) [2024-10-16 14:57:58.982224] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=144] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:58.982235] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=115173325, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:57:58.988284] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:58.988307] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3979) [2024-10-16 14:57:58.988334] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3979) [2024-10-16 14:57:58.993165] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:58.993190] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=215) [2024-10-16 14:57:59.023069] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061879023055000, start_id=1729061879023055000, end_id=1729061879023055001) [2024-10-16 14:57:59.023665] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10D6-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:57:59.023678] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10D6-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=505, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:57:59.028806] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061879027299000, start_id=1729061879027299000, end_id=1729061879027299001) [2024-10-16 14:57:59.029015] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=3] ====== tenant freeze timer task ====== [2024-10-16 14:57:59.029028] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151574520, tenant_id=1) [2024-10-16 14:57:59.029044] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=10] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:57:59.035915] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=17] PNIO [ratelimit] time: 1729061879035914, bytes: 4446821304204, bw: 0.364350 MB/s, add_ts: 1001759, add_bytes: 382721 [2024-10-16 14:57:59.036179] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=224, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:57:59.036191] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=211, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:57:59.036205] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:57:59.036277] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:57:59.036672] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD41A-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:57:59.036681] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD41A-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=773, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:57:59.037007] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.037036] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=29] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.037061] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=22] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:59.037191] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876316069000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890098662000, latest_log_ts:{val:1729061873558226001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:59.037245] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=50] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876316069000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890098662000, latest_log_ts:{val:1729061873558226001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:57:59.037276] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:57:59.037283] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061873558226001, v:0}) [2024-10-16 14:57:59.037287] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:57:59.037295] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:59.037310] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:57:59.037321] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:59.038027] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.038043] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.038057] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:59.038097] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876316069000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:59.038128] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876316069000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:59.038155] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:57:59.038160] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:59.038169] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:57:59.038177] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:59.038894] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.038909] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.038920] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:59.038953] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876316069000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:59.038983] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876316069000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:59.039006] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:57:59.039011] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:59.039017] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:57:59.039022] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:59.039745] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.039758] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.039771] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:59.039805] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876316069000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:59.039836] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=29] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876316069000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:59.039867] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:57:59.039876] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:59.039884] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:57:59.039893] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:59.039896] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:59.043154] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:59.043175] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=15] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:57:59.043192] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=291995456, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:57:59.047652] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:57:59.047668] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:57:59.050382] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=14] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:57:59.050398] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061879050332000, start_id=1729061879050332000, end_id=1729061879050332001) [2024-10-16 14:57:59.052256] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:59.055090] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Cache wash clean map node details(ret=0, clean_node_count=14, clean_time=0, clean_start_pos=1572850, clean_num=62914) [2024-10-16 14:57:59.058270] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F2C-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:59.058448] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:57:59.058741] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9280-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:57:59.058753] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:59.058755] INFO [LIB] stat (utility.h:1165) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9280-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9525, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:57:59.058765] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107502][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9280-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=334, avg_log_submit_to_slide_time=9520) [2024-10-16 14:57:59.064070] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061879063922000, start_id=1729061879063922000, end_id=1729061879063922001) [2024-10-16 14:57:59.064091] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061879064065], gts:1729061879063922000, latest_srr:[mts=1729061879064065]}) [2024-10-16 14:57:59.064102] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:57:59.071798] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061879071326000, start_id=1729061879071326000, end_id=1729061879071326001) [2024-10-16 14:57:59.074075] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78C5-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:57:59.074273] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=32] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:57:59.078827] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:59.078847] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:57:59.078853] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.085971] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107502][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DAB-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:59.086160] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:59.088344] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.098008] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=51] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:59.098595] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:59.098653] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=25] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061870205428000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:57:59.128007] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:59.128026] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:59.136246] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=179, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:57:59.136265] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=38, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:57:59.137761] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4105, mem_used=8318976) [2024-10-16 14:57:59.145470] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 128.50; ) [2024-10-16 14:57:59.145492] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:59.145528] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 133.25; ) [2024-10-16 14:57:59.145537] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:57:59.145549] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:57:59.148203] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4312-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:57:59.148222] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4312-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=587, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:57:59.149735] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:59.158227] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15E8-0-0] [lt=20] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:59.158337] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA7C-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061879157755000, start_id=1729061879157755000, end_id=1729061879157755001) [2024-10-16 14:57:59.158507] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA7D-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061879157755000, start_id=1729061879157755000, end_id=1729061879157755001) [2024-10-16 14:57:59.158546] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D04-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061879157755000, start_id=1729061879157755000, end_id=1729061879157755001) [2024-10-16 14:57:59.158569] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:57:59.158700] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA7A-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061879157755000, start_id=1729061879157755000, end_id=1729061879157755001) [2024-10-16 14:57:59.159651] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=12] current monitor number(seq_=-1) [2024-10-16 14:57:59.161416] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107539][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=192) [2024-10-16 14:57:59.161445] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107539][][T1004][Y0-0000000000000000-0-0] [lt=27] Init thread local success [2024-10-16 14:57:59.161470] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F2E-0-0] [lt=7] init point thread id with(&point=0x55e3435cb340, idx_=1043, point=[thread id=107539, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107539) [2024-10-16 14:57:59.161591] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] worker thread created(tenant_->id()=1004, group_id_=1, token=1) [2024-10-16 14:57:59.165041] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15E4-0-0] [lt=13] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:59.165189] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA7B-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061879157755000, start_id=1729061879157755000, end_id=1729061879157755001) [2024-10-16 14:57:59.165551] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF26E-0-0] [lt=5] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.108497524261474609e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.002399163312093151e+03) [2024-10-16 14:57:59.165717] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061879165337000, start_id=1729061879165337000, end_id=1729061879165337001) [2024-10-16 14:57:59.165790] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:57:59.165802] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:57:59.178899] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.188101] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3978) [2024-10-16 14:57:59.188401] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.194832] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061879194830, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007619, add_bytes: 0 [2024-10-16 14:57:59.200285] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503D-0-0] [lt=5] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:59.200299] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503D-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:59.223408] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061879222797000, start_id=1729061879222797000, end_id=1729061879222797001) [2024-10-16 14:57:59.224666] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:59.228779] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061879226995000, start_id=1729061879226995000, end_id=1729061879226995001) [2024-10-16 14:57:59.236015] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:57:59.236033] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:57:59.236321] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1758) [2024-10-16 14:57:59.236456] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE4007C-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=472, avg_log_submit_to_slide_time=738) [2024-10-16 14:57:59.247732] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:59.247746] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:59.248016] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:59.248163] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C44F7-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=263, avg_log_submit_to_slide_time=534) [2024-10-16 14:57:59.250421] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061879250293000, start_id=1729061879250293000, end_id=1729061879250293001) [2024-10-16 14:57:59.252503] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] ====== checkpoint timer task ====== [2024-10-16 14:57:59.253222] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.253254] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=31] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.253276] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:57:59.253411] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890721265000, latest_log_ts:{val:1729061874059797001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:59.253458] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:57:59.253480] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:57:59.253490] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061874059797001, v:0}) [2024-10-16 14:57:59.253494] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.253504] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:57:59.253518] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:57:59.253528] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:57:59.254258] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.254275] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.254290] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:57:59.254359] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:59.254394] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=33] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:57:59.254425] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:57:59.254435] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:57:59.254445] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:57:59.254453] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:57:59.255184] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.255198] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.255209] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:57:59.255277] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:59.255305] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:57:59.255329] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:57:59.255334] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:57:59.255341] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:57:59.255346] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:57:59.255376] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:59.256054] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.256069] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:57:59.256082] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:57:59.256150] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:57:59.256182] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061876061556003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:57:59.256208] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:57:59.256213] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:57:59.256222] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:57:59.256230] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:57:59.256234] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:57:59.259256] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E0F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061879258665000, start_id=1729061879258665000, end_id=1729061879258665001) [2024-10-16 14:57:59.272266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061879270172000, start_id=1729061879270172000, end_id=1729061879270172001) [2024-10-16 14:57:59.274793] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DC1-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510347143187, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410627, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:59.275216] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33576, start_service_tstamp_=1722494263708085, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:57:59.275290] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78C9-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938446, log_task={header:{begin_lsn:{lsn:213058261733}, end_lsn:{lsn:213058261855}, log_id:220938446, min_scn:{val:1729061879157755001, v:0}, max_scn:{val:1729061879157755001, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058261611}, prev_proposal_id:88, committed_end_lsn:{lsn:213058261733}, data_checksum:1715074492, accum_checksum:3667171576, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061879174125, freeze_ts:1729061879174126, submit_ts:1729061879174129, flushed_ts:1729061879181087, gen_to_freeze cost time:1, gen_to_submit cost time:4, submit_to_flush cost time:6958}, fs_cb_begin_ts=1729061879275285, log_life_time=101160) [2024-10-16 14:57:59.279002] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.288467] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.292333] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA891-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:59.292357] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA891-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061879157755000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061879157755000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:59.293320] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA891-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061879293165000, start_id=1729061879293165000, end_id=1729061879293165001) [2024-10-16 14:57:59.293797] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=25] dump tenant info(tenant={id:1, tenant_meta:{unit:{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, super_block:{tenant_id:1, replay_start_point:ObLogCursor{file_id=12, log_id=1497048, offset=40358755}, ls_meta_entry:[2738](ver=0,mode=0,seq=7639021), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"3.000000000000000000e+00", unit_max_cpu:"3.000000000000000000e+00", total_worker_cnt:54, min_worker_cnt:32, max_worker_cnt:150, stopped:0, worker_us:1096711289919, recv_hp_rpc_cnt:5717694, recv_np_rpc_cnt:154801912, recv_lp_rpc_cnt:0, recv_mysql_cnt:62154104, recv_task_cnt:6664598, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:19623925251, workers:32, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=7 cnt[3]=0 cnt[4]=0 cnt[5]=228765375 cnt[6]=10625 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541161, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143148686}{group_id:10, queue_size:0, recv_req_cnt:19682125, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143048442}{group_id:20, queue_size:0, recv_req_cnt:401, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015260372185}{group_id:15, queue_size:0, recv_req_cnt:5821, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723554329}{group_id:5, queue_size:0, recv_req_cnt:35746215, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494262337428}{group_id:19, queue_size:0, recv_req_cnt:7245703, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061834968674}{group_id:9, queue_size:0, recv_req_cnt:15758126, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143222379}{group_id:1, queue_size:0, recv_req_cnt:153511372, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061854965187}, rpc_stat_info: pcode=0x51c:cnt=332 pcode=0x1502:cnt=216 pcode=0x717:cnt=183 pcode=0x150b:cnt=60 pcode=0x710:cnt=54, token_change_ts:1722494237022007, tenant_role:1}) [2024-10-16 14:57:59.294315] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=516] dump throttled time info(id_=1, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 10, group: OBCG_LOC_CACHE, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 9, group: OBCG_DETECT_RS, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1, tenant_throttled_time: 0;) [2024-10-16 14:57:59.294329] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] dump tenant info(tenant={id:508, tenant_meta:{unit:{tenant_id:508, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140910089, is_removed:false}, super_block:{tenant_id:508, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:15574754791, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:51691885, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6536500648, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:15, queue_size:0, recv_req_cnt:53884, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060722522359}, rpc_stat_info: pcode=0x5c0:cnt=63, token_change_ts:1722494140914497, tenant_role:0}) [2024-10-16 14:57:59.294791] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=459] dump throttled time info(id_=508, throttled_time_log=group_id: 15, group: OBCG_WR, throttled_time: 0;tenant_id: 508, tenant_throttled_time: 0;) [2024-10-16 14:57:59.294797] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=6] dump tenant info(tenant={id:509, tenant_meta:{unit:{tenant_id:509, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140901337, is_removed:false}, super_block:{tenant_id:509, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:12839166510, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6441838921, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1722494140916657, tenant_role:0}) [2024-10-16 14:57:59.295279] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=480] dump throttled time info(id_=509, throttled_time_log=tenant_id: 509, tenant_throttled_time: 0;) [2024-10-16 14:57:59.295286] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=7] dump tenant info(tenant={id:1001, tenant_meta:{unit:{tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1001, replay_start_point:ObLogCursor{file_id=11, log_id=1059563, offset=59267180}, ls_meta_entry:[7452](ver=0,mode=0,seq=7431948), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:468365433732, recv_hp_rpc_cnt:13589316, recv_np_rpc_cnt:126529805, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2784380092, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=9 cnt[3]=0 cnt[4]=0 cnt[5]=167011910 cnt[6]=3244 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541160, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494145075365}{group_id:20, queue_size:0, recv_req_cnt:306, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015263681355}{group_id:5, queue_size:0, recv_req_cnt:58923703, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726724812431820}{group_id:19, queue_size:0, recv_req_cnt:7662213, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061844973518}{group_id:1, queue_size:0, recv_req_cnt:227493823, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061856295889}, rpc_stat_info: pcode=0x1502:cnt=352 pcode=0x51c:cnt=222 pcode=0x717:cnt=181 pcode=0x710:cnt=91 pcode=0x150b:cnt=60, token_change_ts:1722494144838773, tenant_role:1}) [2024-10-16 14:57:59.295841] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=553] dump throttled time info(id_=1001, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1001, tenant_throttled_time: 0;) [2024-10-16 14:57:59.295861] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=20] dump tenant info(tenant={id:1002, tenant_meta:{unit:{tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1002, replay_start_point:ObLogCursor{file_id=191, log_id=47529733, offset=57901112}, ls_meta_entry:[10485](ver=0,mode=0,seq=7657455), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:998834134051, recv_hp_rpc_cnt:198478889, recv_np_rpc_cnt:379058976, recv_lp_rpc_cnt:0, recv_mysql_cnt:280947956, recv_task_cnt:146658, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:4103776307, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=673 cnt[3]=0 cnt[4]=0 cnt[5]=64429074 cnt[6]=226112 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131355253, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1728209719937557}{group_id:20, queue_size:0, recv_req_cnt:1673, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264869418}{group_id:15, queue_size:0, recv_req_cnt:5733, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723564477}{group_id:5, queue_size:0, recv_req_cnt:401894165, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1727167492226332}{group_id:19, queue_size:0, recv_req_cnt:1399889, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061856166239}{group_id:1, queue_size:0, recv_req_cnt:977161387, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061874601819}, rpc_stat_info: pcode=0x1502:cnt=402 pcode=0x1501:cnt=201 pcode=0x1516:cnt=201 pcode=0x717:cnt=178 pcode=0x150b:cnt=120, token_change_ts:1729004298741539, tenant_role:1}) [2024-10-16 14:57:59.296415] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=552] dump throttled time info(id_=1002, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1002, tenant_throttled_time: 0;) [2024-10-16 14:57:59.296431] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=16] dump tenant info(tenant={id:1003, tenant_meta:{unit:{tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1003, replay_start_point:ObLogCursor{file_id=11, log_id=1067656, offset=59494798}, ls_meta_entry:[3704](ver=0,mode=0,seq=7380334), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:503359569416, recv_hp_rpc_cnt:13609706, recv_np_rpc_cnt:126132525, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2783976861, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=4 cnt[3]=0 cnt[4]=0 cnt[5]=196318329 cnt[6]=6245 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541125, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722517512781421}{group_id:20, queue_size:0, recv_req_cnt:308, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015261556381}{group_id:5, queue_size:0, recv_req_cnt:63971943, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726083164255294}{group_id:19, queue_size:0, recv_req_cnt:7660912, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061862930562}{group_id:1, queue_size:0, recv_req_cnt:228854861, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061872442043}, rpc_stat_info: pcode=0x1502:cnt=344 pcode=0x51c:cnt=254 pcode=0x717:cnt=182 pcode=0x710:cnt=98 pcode=0x150b:cnt=60, token_change_ts:1722494150252707, tenant_role:1}) [2024-10-16 14:57:59.296968] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=535] dump throttled time info(id_=1003, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1003, tenant_throttled_time: 0;) [2024-10-16 14:57:59.296978] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=10] dump tenant info(tenant={id:1004, tenant_meta:{unit:{tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1004, replay_start_point:ObLogCursor{file_id=197, log_id=49173916, offset=15367427}, ls_meta_entry:[12150](ver=0,mode=0,seq=7655944), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:19, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:603328173518, recv_hp_rpc_cnt:12341804, recv_np_rpc_cnt:113755795, recv_lp_rpc_cnt:0, recv_mysql_cnt:11227, recv_task_cnt:1052, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2741361379, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=99 cnt[3]=0 cnt[4]=0 cnt[5]=72426608 cnt[6]=155203 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131355111, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494154950936}{group_id:20, queue_size:0, recv_req_cnt:798, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264132685}{group_id:15, queue_size:0, recv_req_cnt:5423, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723614999}{group_id:21, queue_size:0, recv_req_cnt:15, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269726417109}{group_id:5, queue_size:0, recv_req_cnt:31029687, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1724705543331737}{group_id:19, queue_size:0, recv_req_cnt:1325449, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061845553743}{group_id:16, queue_size:0, recv_req_cnt:18, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269721623294}{group_id:1, queue_size:0, recv_req_cnt:509701069, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:2, nesting_worker_cnt:0, token_change:1729061879161274}, rpc_stat_info: pcode=0x1502:cnt=400 pcode=0x1501:cnt=201 pcode=0x1516:cnt=194 pcode=0x717:cnt=186 pcode=0x150b:cnt=120, token_change_ts:1725266206494273, tenant_role:1}) [2024-10-16 14:57:59.297461] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=482] dump throttled time info(id_=1004, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 21, group: OBCG_UNLOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 16, group: OBCG_TRANSFER, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1004, tenant_throttled_time: 0;) [2024-10-16 14:57:59.297896] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1E04-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061879297124000, start_id=1729061879297124000, end_id=1729061879297124001) [2024-10-16 14:57:59.299278] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85101-0-0] [lt=33] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:57:59.299510] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:59.299725] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA891-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061879299309000, start_id=1729061879299309000, end_id=1729061879299309001) [2024-10-16 14:57:59.299792] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA891-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10791, avg_group_log_size=634) [2024-10-16 14:57:59.299805] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA891-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10792) [2024-10-16 14:57:59.299815] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D550D-0-0] [lt=18] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:57:59.300020] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=201, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:57:59.300034] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10791) [2024-10-16 14:57:59.300045] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=205, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:57:59.300062] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10792) [2024-10-16 14:57:59.300071] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:57:59.300218] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9606-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:57:59.300227] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9606-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=646, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:57:59.300231] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9606-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=14, avg_log_gen_to_submit_time=44, avg_log_submit_to_flush_time=429, avg_log_submit_to_slide_time=602) [2024-10-16 14:57:59.300299] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=75] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:59.310442] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=20] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:57:59.311079] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985DE-0-0] [lt=10] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=193) [2024-10-16 14:57:59.311155] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061879310940000, start_id=1729061879310940000, end_id=1729061879310940001) [2024-10-16 14:57:59.313016] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD103F-0-0] [lt=17] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:57:59.329408] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=599, mem_used=2079744) [2024-10-16 14:57:59.332533] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B26-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061879331616000, start_id=1729061879331616000, end_id=1729061879331616001) [2024-10-16 14:57:59.335655] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21657][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE89A-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510347204050, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410628, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:57:59.335722] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:57:59.336806] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:57:59.346457] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1168-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061879345611000, start_id=1729061879345611000, end_id=1729061879345611001) [2024-10-16 14:57:59.348092] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=210, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:57:59.348109] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=38, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:57:59.348174] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:59.348183] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:57:59.349884] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=11][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:59.358590] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F32-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173327, log_task={header:{begin_lsn:{lsn:14352396023}, end_lsn:{lsn:14352396145}, log_id:115173327, min_scn:{val:1729061879157755000, v:0}, max_scn:{val:1729061879157755000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352395901}, prev_proposal_id:17, committed_end_lsn:{lsn:14352396023}, data_checksum:940009244, accum_checksum:3674682518, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061879258524, freeze_ts:1729061879258524, submit_ts:1729061879258527, flushed_ts:1729061879259532, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:1005}, fs_cb_begin_ts=1729061879358580, log_life_time=100056) [2024-10-16 14:57:59.358707] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:57:59.365743] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061879364746000, start_id=1729061879364746000, end_id=1729061879364746001) [2024-10-16 14:57:59.374275] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78CB-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:59.374470] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:57:59.376641] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB6-0-0] [lt=15] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:57:59.376657] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB6-0-0] [lt=12] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=21) [2024-10-16 14:57:59.379160] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.383144] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789DF-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061879382610000, start_id=1729061879382610000, end_id=1729061879382610001) [2024-10-16 14:57:59.386286] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DB1-0-0] [lt=35] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:59.386493] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=185, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:57:59.386516] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:57:59.386580] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:57:59.388534] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.395453] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:57:59.400109] INFO [SERVER] refresh_avail_memory (ob_table_load_assigned_memory_manager.cpp:117) [22703][T1004_TntShared][T1004][Y0-0000000000000000-0-0] [lt=17] ObTableLoadAssignedMemoryManager::refresh_avail_memory(MTL_ID()=1004, avail_memory=53687091, chunk_count_=0, avail_sort_memory_=53687091, avail_memory_=53687091) [2024-10-16 14:57:59.401997] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85103-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1333, avg_group_log_size=121) [2024-10-16 14:57:59.422737] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] no need to update min/max version(new_version={val:1729061879023055000, v:0}, min_version={val:1729061879023055000, v:0}, max_version={val:1729061879023055000, v:0}) [2024-10-16 14:57:59.428948] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061879427864000, start_id=1729061879427864000, end_id=1729061879427864001) [2024-10-16 14:57:59.436634] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601474, offset=9645590}, end_cursor:ObLogCursor{file_id=192, log_id=47601475, offset=9645874}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd208c050, len:490, log_data_len:284, seq:47601474, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:59.438345] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B5-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061879437382000, start_id=1729061879437382000, end_id=1729061879437382001) [2024-10-16 14:57:59.439207] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601475, offset=9645874}, end_cursor:ObLogCursor{file_id=192, log_id=47601476, offset=9646148}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7386050, len:4302, log_data_len:274, seq:47601475, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:59.441070] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601476, offset=9646148}, end_cursor:ObLogCursor{file_id=192, log_id=47601477, offset=9646422}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd205c050, len:4028, log_data_len:274, seq:47601476, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:59.441223] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601477, offset=9646422}, end_cursor:ObLogCursor{file_id=192, log_id=47601478, offset=9646696}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73e6050, len:3754, log_data_len:274, seq:47601477, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:59.441273] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561455, miss_cnt:1, cache_read_size:152590702081, hit_ratio:"9.999999928346978706e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:57:59.441286] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599042, miss_cnt:9, cache_read_size:149777933792, hit_ratio:"9.999999345925721927e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:57:59.450537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061879449970000, start_id=1729061879449970000, end_id=1729061879449970001) [2024-10-16 14:57:59.455674] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:59.461000] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] [COMPACTION DAG STATUS] (start_time=1729061869458099, end_time=1729061879460999, dag_status={}) [2024-10-16 14:57:59.461016] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=14] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:59.461027] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:59.461034] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:59.461038] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:59.461044] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:59.461048] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:59.461054] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:59.461057] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:59.461063] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:57:59.461067] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461073] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461080] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461084] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461096] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461100] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461104] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461108] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461113] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461117] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461121] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461125] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461129] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461132] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461136] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461140] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461144] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461148] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461152] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461155] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461159] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461163] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461167] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461170] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461175] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461179] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461182] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461186] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461189] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461193] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461197] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461201] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461204] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461208] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461212] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461216] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461220] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461224] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461228] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461231] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461235] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461239] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461243] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461252] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461255] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461259] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461263] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461266] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461270] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461274] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461278] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461281] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461285] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461289] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461292] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461296] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461300] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461304] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:57:59.461308] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:57:59.461311] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:57:59.461315] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:57:59.461317] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:57:59.461320] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:57:59.461322] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:57:59.461324] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:57:59.461327] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:57:59.461329] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:57:59.461331] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:57:59.461343] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:57:59.466144] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21810][T1001_TntShared][T1001][Y0-0000000000000000-0-0] [lt=14] clear dtl interm result cost(us)(clear_cost=15, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:57:59.472392] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061879472245000, start_id=1729061879472245000, end_id=1729061879472245001) [2024-10-16 14:57:59.474351] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78CD-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:57:59.474535] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=203, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:57:59.474554] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=670, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:57:59.474780] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5510-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:57:59.474787] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5510-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30011, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:57:59.474792] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:59.474833] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=51, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:57:59.474842] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:57:59.479222] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.487333] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A91-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:57:59.487350] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A91-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:57:59.488590] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.489955] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8D0-0-0] [lt=7] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:57:59.489974] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8D0-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:59.489997] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8D0-0-0] [lt=20] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:57:59.490738] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:57:59.490752] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:57:59.490763] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70863259, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:59.492566] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:59.492575] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=8] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:57:59.492584] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:57:59.492589] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:57:59.492593] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:57:59.492596] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:57:59.492598] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:57:59.492608] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:57:59.492610] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:57:59.492613] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:57:59.492616] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:57:59.492633] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2B-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:59.492640] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2B-0-0] [lt=6] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:57:59.508658] INFO [STORAGE.TRANS] refresh_elr_tenant_config_ (ob_tx_elr_util.cpp:47) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C89-0-0] [lt=15] refresh tenant config success(tenant_id=1004, *this={last_refresh_ts:1729061879507676, can_tenant_elr:true}) [2024-10-16 14:57:59.508821] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C89-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061879508732000, start_id=1729061879508732000, end_id=1729061879508732001) [2024-10-16 14:57:59.514073] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257822, offset=39049966}, end_cursor:ObLogCursor{file_id=197, log_id=49257823, offset=39050249}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db735e050, len:1298, log_data_len:283, seq:49257822, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:59.516133] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB380-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061879514054000, start_id=1729061879514054000, end_id=1729061879514054001) [2024-10-16 14:57:59.517179] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257823, offset=39050249}, end_cursor:ObLogCursor{file_id=197, log_id=49257824, offset=39050531}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20a8050, len:1015, log_data_len:282, seq:49257823, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:59.519175] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257824, offset=39050531}, end_cursor:ObLogCursor{file_id=197, log_id=49257825, offset=39050813}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db729e050, len:733, log_data_len:282, seq:49257824, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:59.519381] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257825, offset=39050813}, end_cursor:ObLogCursor{file_id=197, log_id=49257826, offset=39051095}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20d8050, len:451, log_data_len:282, seq:49257825, flush_finish:false, flush_ret:0}) [2024-10-16 14:57:59.519438] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=50] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650392, miss_cnt:32, cache_read_size:8050439126, hit_ratio:"9.999994892293146975e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:57:59.519445] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657131, miss_cnt:17, cache_read_size:8050186610, hit_ratio:"9.999997286821927789e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:57:59.523250] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=9] replay status is not follower(min_unreplayed_lsn={lsn:7385763025}, base_lsn={lsn:7314419712}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:57:59.523297] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=47] get_replay_process success(id={id:1}, replayed_log_size=71343313, unreplayed_log_size=0) [2024-10-16 14:57:59.523304] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:57923794664}, base_lsn={lsn:57911414784}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:57:59.523328] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=24] get_replay_process success(id={id:1001}, replayed_log_size=12379880, unreplayed_log_size=0) [2024-10-16 14:57:59.523335] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=152596866253, unreplayed_log_size=0) [2024-10-16 14:57:59.523342] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=6] get_replay_process success(id={id:1003}, replayed_log_size=149812364103, unreplayed_log_size=0) [2024-10-16 14:57:59.523357] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1002, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=288479, last_replayed_log_size(MB)=288479, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:57:59.536128] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:57:59.536142] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:57:59.536395] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:57:59.536635] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=22, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:57:59.536647] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=1003, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:57:59.536651] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:57:59.536654] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=-562, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:57:59.536657] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=25, stat_interval=5000000, avg (count/cost)=463, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:57:59.547873] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=28, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:57:59.547890] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:57:59.547914] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:57:59.547922] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:57:59.548212] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2194) [2024-10-16 14:57:59.548490] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F431F-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:59.548499] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F431F-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=610, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:59.560253] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2390732, replace_num=31457) [2024-10-16 14:57:59.560271] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:57:59.564453] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061879563510000, start_id=1729061879563510000, end_id=1729061879563510001) [2024-10-16 14:57:59.564471] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] ts response task statistics(alloc_count=261365154, free_count=130682576) [2024-10-16 14:57:59.564484] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=4] gts statistics(tenant_id=1001, gts_rpc_cnt=11, get_gts_cache_cnt=490, get_gts_with_stc_cnt=75, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=73, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:59.564654] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=518, get_gts_with_stc_cnt=6, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:57:59.565890] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061879565577000, start_id=1729061879565577000, end_id=1729061879565577001) [2024-10-16 14:57:59.568761] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.568795] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.568807] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.568828] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.579281] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.586227] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DB5-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:57:59.586657] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B928B-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:57:59.586678] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B928B-0-0] [lt=21] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=453, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:57:59.588640] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.597160] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=2][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:59.597176] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:57:59.597693] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1002}, {id:1}]) [2024-10-16 14:57:59.599468] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85107-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:57:59.599477] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85107-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=33676, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:57:59.599648] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=348, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:57:59.599664] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=66, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:57:59.599703] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:57:59.605394] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4812-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061879573136, keepalive_ts:-1}) [2024-10-16 14:57:59.605407] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4812-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061879573136, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:59.606070] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B30-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061879605504000, start_id=1729061879605504000, end_id=1729061879605504001) [2024-10-16 14:57:59.606589] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=42, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:57:59.606605] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=16] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=994, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:57:59.606609] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:57:59.606613] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=-619, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:57:59.606616] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=417, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:57:59.610024] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1E03-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061879609785000, start_id=1729061879609785000, end_id=1729061879609785001) [2024-10-16 14:57:59.610503] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797D7C-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10566) [2024-10-16 14:57:59.610951] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5712, mem_used=10398720) [2024-10-16 14:57:59.623573] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061879623567000, start_id=1729061879623567000, end_id=1729061879623567001) [2024-10-16 14:57:59.624198] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061879258665001, v:0}, new_max_version={val:1729061879258665001, v:0}, last_min_version={val:1729061879023055000, v:0}, last_max_version={val:1729061879023055000, v:0}, total_time_us=1371, delta_us=365532) [2024-10-16 14:57:59.624217] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061879258665001, v:0}, min_version={val:1729061879258665001, v:0}, max_version={val:1729061879258665001, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:57:59.626712] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:57:59.629073] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061879628867000, start_id=1729061879628867000, end_id=1729061879628867001) [2024-10-16 14:57:59.650834] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061879650572000, start_id=1729061879650572000, end_id=1729061879650572001) [2024-10-16 14:57:59.651102] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:57:59.651282] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=380, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:57:59.651303] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=78, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:57:59.651310] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:57:59.651443] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD269-0-0] [lt=24] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:57:59.651450] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD269-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=820, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:57:59.652626] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=17] BlockGCTimerTask success(ret=0, cost_time_us=9, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:57:59.655964] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:59.658466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA7F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061879658070000, start_id=1729061879658070000, end_id=1729061879658070001) [2024-10-16 14:57:59.658530] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA80-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061879658070000, start_id=1729061879658070000, end_id=1729061879658070001) [2024-10-16 14:57:59.658595] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA81-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061879658070000, start_id=1729061879658070000, end_id=1729061879658070001) [2024-10-16 14:57:59.658663] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA7E-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061879658070000, start_id=1729061879658070000, end_id=1729061879658070001) [2024-10-16 14:57:59.658807] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA82-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061879658070000, start_id=1729061879658070000, end_id=1729061879658070001) [2024-10-16 14:57:59.659121] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=358, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:57:59.659136] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=529, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:57:59.659186] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:57:59.672552] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061879671830000, start_id=1729061879671830000, end_id=1729061879671830001) [2024-10-16 14:57:59.673060] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:59.679332] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.686831] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B928D-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=222, avg_log_submit_to_slide_time=454) [2024-10-16 14:57:59.686838] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:59.689164] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.699975] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:57:59.706199] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=10] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:57:59.706940] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061879706003000, start_id=1729061879706003000, end_id=1729061879706003001) [2024-10-16 14:57:59.708963] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D5-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:57:59.708985] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D5-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061879157755004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061879157755004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:59.713168] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D5-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061879713071000, start_id=1729061879713071000, end_id=1729061879713071001) [2024-10-16 14:57:59.713255] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D5-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=11071, avg_group_log_size=651) [2024-10-16 14:57:59.713273] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D5-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11071) [2024-10-16 14:57:59.713483] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11071) [2024-10-16 14:57:59.713857] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD26C-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=486, avg_log_submit_to_slide_time=723) [2024-10-16 14:57:59.713893] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:57:59.716193] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:57:59.716214] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=216) [2024-10-16 14:57:59.724839] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B23-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061879724025000, start_id=1729061879724025000, end_id=1729061879724025001) [2024-10-16 14:57:59.727743] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:59.742540] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=18] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:57:59.759155] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:57:59.759188] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=16] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:57:59.759200] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=11] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:57:59.760343] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:57:59.763742] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C62-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:59.763772] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C62-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061879658070001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061879658070001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:59.764820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C62-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061879764446000, start_id=1729061879764446000, end_id=1729061879764446001) [2024-10-16 14:57:59.766018] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061879764446000, start_id=1729061879764446001, end_id=1729061879764446002) [2024-10-16 14:57:59.766154] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=15][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:57:59.768500] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15EE-0-0] [lt=4] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:57:59.768895] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C62-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061879768304000, start_id=1729061879768304000, end_id=1729061879768304001) [2024-10-16 14:57:59.771609] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=22] LogLoopThread round_cost_time(us)(round_cost_time=19) [2024-10-16 14:57:59.772811] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=17] gc stale ls task succ [2024-10-16 14:57:59.779396] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.782653] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:57:59.788063] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.788062] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.788203] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.788351] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.788479] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.788630] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.788715] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.788825] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.788840] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.788857] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.789184] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.789615] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:57:59.816541] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A12-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061879807735, keepalive_ts:-1}) [2024-10-16 14:57:59.816561] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A12-0-0] [lt=20] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061879807735, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:57:59.822956] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061879623567000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199381, in_cluster_service=true, cluster_version={val:1729061879258665001, v:0}, min_cluster_version={val:1729061879258665001, v:0}, max_cluster_version={val:1729061879258665001, v:0}, get_cluster_version_err=0, cluster_version_delta=564283, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65249079, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=377095663, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:57:59.824261] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=48] get number(ret=0, service_type_=0, range=1, base_id=1729061879823327000, start_id=1729061879823327000, end_id=1729061879823327001) [2024-10-16 14:57:59.824345] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:57:59.824362] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=4593, avg_group_log_size=382) [2024-10-16 14:57:59.824375] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:57:59.824810] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=223, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:57:59.824829] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=36, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:57:59.824850] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=7, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:57:59.824922] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10E0-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=342, avg_log_submit_to_slide_time=515) [2024-10-16 14:57:59.829375] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:57:59.829391] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:57:59.829405] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52607149, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:57:59.829620] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061879827691000, start_id=1729061879827691000, end_id=1729061879827691001) [2024-10-16 14:57:59.833093] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1002}, {id:1}]) [2024-10-16 14:57:59.836276] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2070, avg_group_log_size=207) [2024-10-16 14:57:59.836287] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2061) [2024-10-16 14:57:59.836530] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2070) [2024-10-16 14:57:59.848634] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=542, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:57:59.848659] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=86, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:57:59.848685] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:57:59.848812] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C450A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=38, avg_log_submit_to_flush_time=651, avg_log_submit_to_slide_time=867) [2024-10-16 14:57:59.851441] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061879850020000, start_id=1729061879850020000, end_id=1729061879850020001) [2024-10-16 14:57:59.856296] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:57:59.860261] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=4] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:59.860872] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:59.860922] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061874438469000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:57:59.861145] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F3C-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:57:59.861162] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F3C-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:57:59.861632] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=5995, mem_used=12478464) [2024-10-16 14:57:59.872678] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061879871687000, start_id=1729061879871687000, end_id=1729061879871687001) [2024-10-16 14:57:59.879459] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.886907] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:57:59.886929] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:57:59.888025] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=21] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:57:59.889246] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.899878] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=34526511, this={dio_aligned_buf:{buf_write_offset:1449, buf_padding_size:2647, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111299748, truncate_used_ts:40614854}, log_block_size:67108864, dir_fd:182, io_fd:127}, aligned_buf_len=4096, aligned_block_offset=34525184, buf_len=122, total_write_size_=12294, total_write_size_after_dio_=425984, ob_pwrite_used_ts=22781, count=101, each_pwrite_cost=225) [2024-10-16 14:57:59.900092] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:57:59.908947] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=40] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:57:59.909542] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:57:59.909595] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061871843583000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:57:59.915023] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=17] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:57:59.924684] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=24] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:57:59.925712] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=17] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:57:59.936367] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:57:59.936383] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:57:59.936934] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE4008F-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:57:59.936946] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE4008F-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=623, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:57:59.936951] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE4008F-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=316, avg_log_submit_to_slide_time=604) [2024-10-16 14:57:59.945630] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26142][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB25CC6BB-0-0] [lt=21] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:57:59.945644] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3879, mem_used=8318976) [2024-10-16 14:57:59.960423] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:57:59.966117] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061879965459000, start_id=1729061879965459000, end_id=1729061879965459001) [2024-10-16 14:57:59.979518] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.980557] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061875977817, last_first_register_time_us=1722494150998907, register_time_us=1729061875977817, curr_time_us=1729061879980555) [2024-10-16 14:57:59.980592] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061879980574, last_submit_register_req_time_us=1729061879980574, register_parent_reason=3) [2024-10-16 14:57:59.980608] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=6] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:57:59.981127] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7F-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:57:59.981144] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D7F-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061879658070003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061879658070003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:57:59.983310] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D7F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061879982234000, start_id=1729061879982234000, end_id=1729061879982234001) [2024-10-16 14:57:59.986371] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC736-0-0] [lt=17] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:57:59.986388] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC736-0-0] [lt=13] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:57:59.987017] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7F-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061879986024000, start_id=1729061879986024000, end_id=1729061879986024001) [2024-10-16 14:57:59.987089] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7F-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:57:59.987107] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D7F-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:57:59.989306] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:57:59.989530] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4593) [2024-10-16 14:57:59.989572] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=24] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4593) [2024-10-16 14:58:00.006352] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=16] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:00.017116] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=7] ====== traversal_flush timer task ====== [2024-10-16 14:58:00.017217] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAF-0-0] [lt=14] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061879965459000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061895590307000, latest_log_ts:{val:1729061878965510001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:00.017278] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DAF-0-0] [lt=57] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:00.023935] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061880023878000, start_id=1729061880023878000, end_id=1729061880023878001) [2024-10-16 14:58:00.025543] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10E2-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:00.025572] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10E2-0-0] [lt=28] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=631, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:00.029217] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061880028615000, start_id=1729061880028615000, end_id=1729061880028615001) [2024-10-16 14:58:00.029817] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=41] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:00.036485] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061880036484, bytes: 4446821692279, bw: 0.369886 MB/s, add_ts: 1000570, add_bytes: 388075 [2024-10-16 14:58:00.036500] INFO pn_print_diag_info (group.c:715) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO client:0x2b0dcaf6ea98_"10.0.4.9:2890"_"10.0.4.9:2890"_141_1722494489083454_1, write_queue=0/0, write=437417235/305975553705, read=437417235/101215961308, doing=0, done=437417235, write_time=438163967235, read_time=26173786, process_time=50614688 [2024-10-16 14:58:00.036507] INFO pn_print_diag_info (group.c:715) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO client:0x2b0dcaf6e048_"10.0.4.8:2890"_"10.0.4.8:2890"_135_1722494324847028_1, write_queue=1/409, write=448067453/266104675460, read=448067453/103733683738, doing=1, done=448067453, write_time=449134067575, read_time=27596936, process_time=53592666 [2024-10-16 14:58:00.036511] INFO pn_print_diag_info (group.c:715) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO client:0x2b0dcaf6f4e8_"10.0.4.7:2890"_"10.0.4.7:2890"_153_1722494143221904_1, write_queue=0/0, write=56852861/31557537718, read=56852861/13543544995, doing=0, done=56852861, write_time=56632544270, read_time=3200797, process_time=5485060 [2024-10-16 14:58:00.036515] INFO pn_print_diag_info (group.c:727) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x2b0dcaed6a58_"10.0.4.9:51572"_152_1722494490514961, write_queue=0/0, write=586900370/218995102326, read=586900370/380774669264, doing=0, done=586900370, write_time=586595660315, read_time=96174216, process_time=123928384 [2024-10-16 14:58:00.036519] INFO pn_print_diag_info (group.c:727) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x2b0dcaed6048_"10.0.4.8:36170"_151_1722494326889766, write_queue=0/0, write=645002032/242075644702, read=645002032/480820435746, doing=0, done=645002032, write_time=645145970079, read_time=132781796, process_time=160739681 [2024-10-16 14:58:00.036522] INFO pn_print_diag_info (group.c:727) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x2b0dcafb4048_"10.0.4.7:34816"_154_1722494143222245, write_queue=0/0, write=56852861/13543544995, read=56852861/31557537718, doing=0, done=56852861, write_time=56811607985, read_time=3598802, process_time=5798961 [2024-10-16 14:58:00.036525] INFO pn_print_diag_info (group.c:734) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO client_send:942366836/603661574727, client_queue_time=135891757511, cnt=3, server_send:1288787706/474627133136, server_queue_time=85616180567, cnt=3 [2024-10-16 14:58:00.036691] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=291, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:00.036711] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=44, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:00.036724] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:00.036735] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:00.037077] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD435-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:00.037085] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD435-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=657, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:00.048480] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:00.048499] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:00.050261] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1004, min_version={val:1729061879564517001, v:0}, base_version={val:1729061879157755004, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061879564517001, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061880050244, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061879564517002, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061880006326, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061879564517001, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061880028920, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:00.051269] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061880051087000, start_id=1729061880051087000, end_id=1729061880051087001) [2024-10-16 14:58:00.052122] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:58:00.052188] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1002 and flag like "%DUPLICATE%", tenant_id=1002, exec_tenant_id=1001, need_member_list=false) [2024-10-16 14:58:00.056589] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:00.059079] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F40-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:00.059301] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:00.059568] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9294-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:00.059584] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9294-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20228, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:00.064846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061880063974000, start_id=1729061880063974000, end_id=1729061880063974001) [2024-10-16 14:58:00.064876] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=21] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:00.064902] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=526, get_gts_with_stc_cnt=81, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=77, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:00.064915] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061880064897], gts:1729061880063974000, latest_srr:[mts=1729061880064897]}) [2024-10-16 14:58:00.072834] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061880072703000, start_id=1729061880072703000, end_id=1729061880072703001) [2024-10-16 14:58:00.074671] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78D9-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:00.074919] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:00.079572] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.086525] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DBF-0-0] [lt=36] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:00.086720] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:00.089341] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.092548] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:58:00.093285] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=13] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:00.093313] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:00.093335] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:00.093437] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061887008855000, latest_log_ts:{val:1729061870498146001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:00.093485] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=45] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:00.093496] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:58:00.093505] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061870498146001, v:0}) [2024-10-16 14:58:00.093509] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:00.093515] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:00.093522] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:00.093535] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:58:00.093544] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:00.093549] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:00.128084] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:00.128107] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:00.136782] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=257, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:00.136806] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=51, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:00.137826] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4166, mem_used=8318976) [2024-10-16 14:58:00.147300] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 153.00; ) [2024-10-16 14:58:00.147322] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:00.147366] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 132.25; ) [2024-10-16 14:58:00.147375] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:00.147386] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:00.149107] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4331-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:00.149143] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4331-0-0] [lt=35] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=914, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:00.149762] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:00.159464] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA85-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061880159095000, start_id=1729061880159095000, end_id=1729061880159095001) [2024-10-16 14:58:00.159467] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA86-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061880159095000, start_id=1729061880159095000, end_id=1729061880159095001) [2024-10-16 14:58:00.159481] INFO [STORAGE.TRANS] handle_request (ob_timestamp_service.cpp:175) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA86-0-0] [lt=12] handle gts request statistics(total_rt=2485, total_cnt=448, avg_rt=5.534521158129176222e+00, avg_cnt=4.479999999999999716e+01) [2024-10-16 14:58:00.159563] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA83-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061880159095000, start_id=1729061880159095000, end_id=1729061880159095001) [2024-10-16 14:58:00.159587] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA87-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061880159095000, start_id=1729061880159095000, end_id=1729061880159095001) [2024-10-16 14:58:00.159672] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA84-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061880159095000, start_id=1729061880159095000, end_id=1729061880159095001) [2024-10-16 14:58:00.159778] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=9] current monitor number(seq_=-1) [2024-10-16 14:58:00.160190] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD43A-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=787, avg_log_submit_to_slide_time=18423) [2024-10-16 14:58:00.160238] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:00.160505] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:58:00.166084] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF27D-0-0] [lt=15] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.108497524261474609e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=305, Throttle Trigger(MB)=183, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.002399163312093151e+03) [2024-10-16 14:58:00.166244] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061880165479000, start_id=1729061880165479000, end_id=1729061880165479001) [2024-10-16 14:58:00.166310] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=25, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:00.166325] INFO [LIB] stat (utility.h:1165) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:00.166838] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=10] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:58:00.167099] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=3] get_max_applied_scn(scn={val:1729061880028615000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618707132}, palf_committed_end_scn_:{val:1729061880165479000, v:0}, last_check_scn_:{val:1729061880165479000, v:0}, max_applied_cb_scn_:{val:1729061880028615000, v:0}}) [2024-10-16 14:58:00.167123] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:58:00.167179] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107502][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=14] worker thread exit(tenant_->id()=1004, workers_.get_size()=2) [2024-10-16 14:58:00.167192] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107502][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=10] tenant thread end_run(id_=1004, ret=0, thread_count_=191) [2024-10-16 14:58:00.179617] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.189402] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.189458] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4594) [2024-10-16 14:58:00.201140] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503E-0-0] [lt=8] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:00.201162] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503E-0-0] [lt=20] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:00.202448] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061880202447, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007617, add_bytes: 0 [2024-10-16 14:58:00.202462] INFO pn_print_diag_info (group.c:734) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO client_send:0/0, client_queue_time=0, cnt=0, server_send:0/0, server_queue_time=0, cnt=0 [2024-10-16 14:58:00.211502] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=45, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:58:00.211513] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=768, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:58:00.211517] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:58:00.211521] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=-504, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:58:00.211524] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6BFDAA-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=309, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:58:00.224016] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061880223774000, start_id=1729061880223774000, end_id=1729061880223774001) [2024-10-16 14:58:00.229347] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061880228985000, start_id=1729061880228985000, end_id=1729061880228985001) [2024-10-16 14:58:00.236491] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:00.236510] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:00.236731] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1948) [2024-10-16 14:58:00.236997] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD43C-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=357, avg_log_submit_to_slide_time=629) [2024-10-16 14:58:00.243985] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:468) [22402][T1003_SSTableDe][T1003][Y0-0000000000000000-0-0] [lt=9] skip defragment task(ret=0, need_defragment=false) [2024-10-16 14:58:00.244003] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [22402][T1003_SSTableDe][T1003][Y0-0000000000000000-0-0] [lt=17] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=9) [2024-10-16 14:58:00.244009] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [22402][T1003_SSTableDe][T1003][Y0-0000000000000000-0-0] [lt=5] successfully defragment data blocks(rewrite_cnt=0) [2024-10-16 14:58:00.244014] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [22402][T1003_SSTableDe][T1003][Y0-0000000000000000-0-0] [lt=4] tablet skip filter destructed(total_cnt=0, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105368325, total_skip_cnt=310297, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:58:00.249124] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4517-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=22, avg_log_submit_to_flush_time=549, avg_log_submit_to_slide_time=837) [2024-10-16 14:58:00.251247] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061880251211000, start_id=1729061880251211000, end_id=1729061880251211001) [2024-10-16 14:58:00.256960] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:00.271995] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061880072703000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199287, in_cluster_service=true, cluster_version={val:1729061879850020000, v:0}, min_cluster_version={val:1729061879850020000, v:0}, max_cluster_version={val:1729061879850020000, v:0}, get_cluster_version_err=0, cluster_version_delta=421970, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244395, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=489341793, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:00.272954] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061880272359000, start_id=1729061880272359000, end_id=1729061880272359001) [2024-10-16 14:58:00.278071] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DC3-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510348146464, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410629, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:00.279681] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.281208] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=43] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:58:00.281219] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx [2024-10-16 14:58:00.281223] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:00.281872] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061895590307000, latest_log_ts:{val:1729061878965510001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:00.289461] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.299910] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85115-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:00.299934] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85115-0-0] [lt=17] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:58:00.300204] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:00.302023] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA892-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:00.302047] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA892-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061880159095001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061880159095001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:00.303178] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA892-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061880303084000, start_id=1729061880303084000, end_id=1729061880303084001) [2024-10-16 14:58:00.306873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA892-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061880305890000, start_id=1729061880305890000, end_id=1729061880305890001) [2024-10-16 14:58:00.307001] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA892-0-0] [lt=24] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10507, avg_group_log_size=618) [2024-10-16 14:58:00.307014] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA892-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10506) [2024-10-16 14:58:00.307346] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=208, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:00.307363] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10507) [2024-10-16 14:58:00.307375] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=41, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:00.307394] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10506) [2024-10-16 14:58:00.307421] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:00.308199] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9617-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=3, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:00.308209] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9617-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=594, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:00.308214] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9617-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=269, avg_log_submit_to_slide_time=560) [2024-10-16 14:58:00.308284] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:00.313758] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:58:00.313776] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:00.313795] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373708300, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:00.329500] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=661, mem_used=2079744) [2024-10-16 14:58:00.335789] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=7] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:00.340813] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:58:00.340835] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=15] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=10485760, ctx.total_memstore_used_=44040192, ctx.total_memstore_hold_=44040192, memstore_freeze_trigger_limit_=28020300, tenant_id=1002) [2024-10-16 14:58:00.340853] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:00.348615] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=3435, avg_group_log_size=286) [2024-10-16 14:58:00.348632] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3435) [2024-10-16 14:58:00.348930] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3435) [2024-10-16 14:58:00.348955] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=478, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:00.348965] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:00.348987] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:00.348993] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:00.353904] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1178-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061880353690000, start_id=1729061880353690000, end_id=1729061880353690001) [2024-10-16 14:58:00.360591] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=8] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:58:00.365461] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=47] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061880165479000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199975, in_cluster_service=true, cluster_version={val:1729061879965459000, v:0}, min_cluster_version={val:1729061879965459000, v:0}, max_cluster_version={val:1729061879965459000, v:0}, get_cluster_version_err=0, cluster_version_delta=399995, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244642, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=254761625, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:00.366373] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061880366362000, start_id=1729061880366362000, end_id=1729061880366362001) [2024-10-16 14:58:00.367066] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33705, start_service_tstamp_=1722494263684720, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:00.372566] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:00.372588] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=17] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54641860, tenant_id=1003) [2024-10-16 14:58:00.372605] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:00.374986] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78DF-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:00.375206] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:00.375262] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:00.379742] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.381163] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:00.381182] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=218) [2024-10-16 14:58:00.384879] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=12] blacklist_loop exec finished(cost_time=35, is_enabled=true, send_cnt=0) [2024-10-16 14:58:00.386633] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DC5-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:00.386832] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=209, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:00.386851] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:00.386896] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:00.389740] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.391479] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22443][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=26][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:00.391531] WDIAG read_multi_row (ob_table_access_helper.h:238) [22443][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=15][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:00.392285] INFO delete_row (ob_table_access_helper.h:312) [22443][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1003, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061880391542) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061880391542) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:58:00.395478] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:58:00.396844] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=10] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:58:00.399532] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DA-0-0] [lt=16] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061880390691, keepalive_ts:-1}) [2024-10-16 14:58:00.399552] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DA-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061880390691, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:00.424361] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] submit log success(service_type_=0, last_id=1729061897008855000, limited_id=1729061897008855000) [2024-10-16 14:58:00.424376] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] submit log with lock success(service_type_=0, last_id=1729061897008855000, limited_id=1729061897008855000) [2024-10-16 14:58:00.424380] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061880424064000, start_id=1729061880424064000, end_id=1729061880424064001) [2024-10-16 14:58:00.425011] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061897008855000, latest_scn={val:1729061880424064001, v:0}, *this={tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061897008855000, latest_log_ts:{val:1729061880424064001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:58:00.425048] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=37] handle submit callback(service_type_=0, success=true, used_ts=689, log_ts={val:1729061880424064001, v:0}, limited_id=1729061897008855000) [2024-10-16 14:58:00.425056] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:58:00.425062] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=5] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:58:00.425281] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061880425130000, start_id=1729061880425130000, end_id=1729061880425130001) [2024-10-16 14:58:00.429957] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061880429331000, start_id=1729061880429331000, end_id=1729061880429331001) [2024-10-16 14:58:00.431619] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=25] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:00.436952] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:00.442755] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601478, offset=9646696}, end_cursor:ObLogCursor{file_id=192, log_id=47601479, offset=9646980}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e84050, len:3480, log_data_len:284, seq:47601478, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:00.444508] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22067][T1002_TeRec][T1002][YB4A0A000407-00061E9B225FA4B6-0-0] [lt=22] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=0, MAX_GAP=900000000000, new_sync_scn={val:1729061879345611000, v:0}, new_replayable_scn={val:1729061879345611000, v:0}, new_readable_scn={val:1729061879345611000, v:0}, old_tenant_info={tenant_id:1002, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061878330616000, v:0}, replayable_scn:{val:1729061878330616000, v:0}, standby_scn:{val:1729061878330616000, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"NOARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:58:00.444708] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B6-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061880442148000, start_id=1729061880442148000, end_id=1729061880442148001) [2024-10-16 14:58:00.445721] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601479, offset=9646980}, end_cursor:ObLogCursor{file_id=192, log_id=47601480, offset=9647254}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2120050, len:3196, log_data_len:274, seq:47601479, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:00.447882] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601480, offset=9647254}, end_cursor:ObLogCursor{file_id=192, log_id=47601481, offset=9647528}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73da050, len:2922, log_data_len:274, seq:47601480, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:00.448065] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601481, offset=9647528}, end_cursor:ObLogCursor{file_id=192, log_id=47601482, offset=9647802}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73f2050, len:2648, log_data_len:274, seq:47601481, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:00.448120] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561465, miss_cnt:1, cache_read_size:152590703292, hit_ratio:"9.999999928346983147e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:00.448154] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599052, miss_cnt:9, cache_read_size:149777935012, hit_ratio:"9.999999345925768557e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:00.451330] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061880451163000, start_id=1729061880451163000, end_id=1729061880451163001) [2024-10-16 14:58:00.457353] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:00.473095] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061880472178000, start_id=1729061880472178000, end_id=1729061880472178001) [2024-10-16 14:58:00.474903] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78E1-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:00.475160] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=188, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:00.475180] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:00.475431] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F433C-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:00.475440] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F433C-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=460, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:00.475464] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:00.475521] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=66, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:00.475532] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:00.479807] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.488323] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A92-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:00.488359] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A92-0-0] [lt=34] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:00.489805] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.500053] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85119-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:58:00.506433] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=44] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:00.507045] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=19] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:00.507093] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061878735149000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:58:00.520357] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257826, offset=39051095}, end_cursor:ObLogCursor{file_id=197, log_id=49257827, offset=39051378}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7376050, len:4265, log_data_len:283, seq:49257826, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:00.522448] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB381-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061880521662000, start_id=1729061880521662000, end_id=1729061880521662001) [2024-10-16 14:58:00.523408] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=5] get_max_applied_scn(scn={val:1729061880424064002, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390198076}, palf_committed_end_scn_:{val:1729061880424064002, v:0}, last_check_scn_:{val:1729061880424064002, v:0}, max_applied_cb_scn_:{val:1729061880424064002, v:0}}) [2024-10-16 14:58:00.523430] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:58:00.523490] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257827, offset=39051378}, end_cursor:ObLogCursor{file_id=197, log_id=49257828, offset=39051660}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7322050, len:3982, log_data_len:282, seq:49257827, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:00.525642] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257828, offset=39051660}, end_cursor:ObLogCursor{file_id=197, log_id=49257829, offset=39051942}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2144050, len:3700, log_data_len:282, seq:49257828, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:00.525839] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257829, offset=39051942}, end_cursor:ObLogCursor{file_id=197, log_id=49257830, offset=39052224}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd207c050, len:3418, log_data_len:282, seq:49257829, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:00.525897] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657141, miss_cnt:17, cache_read_size:8050187830, hit_ratio:"9.999997286822360776e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:00.525907] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650402, miss_cnt:32, cache_read_size:8050440337, hit_ratio:"9.999994892293961879e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:00.534187] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:58:00.534209] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=16] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:58:00.534226] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444626411, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:00.536723] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:58:00.536740] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:58:00.536940] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:00.544268] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=911101952, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=162639872, wash_ratio=6, tenant_id=1001) [2024-10-16 14:58:00.544329] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=29] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:00.544342] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=12] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:00.544376] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=969699328, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=104042496, wash_ratio=6, tenant_id=1002) [2024-10-16 14:58:00.544411] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:00.544419] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:58:00.544451] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=927879168, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=145862656, wash_ratio=6, tenant_id=1003) [2024-10-16 14:58:00.544493] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:00.544499] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011872, drift_size_=0, pre_drift_size=1043719, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:00.544532] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=956841984, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=116899840, wash_ratio=6, tenant_id=1004) [2024-10-16 14:58:00.544569] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:00.544576] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:00.548709] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:00.548733] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:00.548769] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:00.548778] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:00.549675] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F433E-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:00.549686] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F433E-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=878, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:00.550284] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C451F-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2461) [2024-10-16 14:58:00.550573] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061880063974002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=486592, in_cluster_service=true, cluster_version={val:1729061880063974001, v:0}, min_cluster_version={val:1729061880063974001, v:0}, max_cluster_version={val:1729061880063974001, v:0}, get_cluster_version_err=0, cluster_version_delta=486592, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228422, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=425942087, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:00.560676] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:58:00.565336] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061880564734000, start_id=1729061880564734000, end_id=1729061880564734001) [2024-10-16 14:58:00.565363] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061880565358], gts:1729061880564734000, latest_srr:[mts=1729061880565358]}) [2024-10-16 14:58:00.566619] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061880565796000, start_id=1729061880565796000, end_id=1729061880565796001) [2024-10-16 14:58:00.568539] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.568620] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.568622] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.568848] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.576241] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22405][T1003_DupTbLeas][T1003][Y0-0000000000000000-0-0] [lt=10][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:58:00.576303] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22405][T1003_DupTbLeas][T1003][Y0-0000000000000000-0-0] [lt=21][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1003 and flag like "%DUPLICATE%", tenant_id=1003, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:58:00.579819] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.586747] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DC9-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:00.587246] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B929F-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:00.587255] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B929F-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20170, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:00.589864] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.597237] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:00.597262] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:00.597797] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1002}, {id:1003}]) [2024-10-16 14:58:00.600490] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=235, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:00.600516] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:00.600566] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:00.600719] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4521-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:00.600734] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4521-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=499, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:00.611046] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5731, mem_used=10398720) [2024-10-16 14:58:00.618269] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=19] BlockGCTimerTask success(ret=0, cost_time_us=8, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:58:00.618709] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=15][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:00.624386] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=45] get number(ret=0, service_type_=0, range=1, base_id=1729061880622842000, start_id=1729061880622842000, end_id=1729061880622842001) [2024-10-16 14:58:00.625217] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061880223774001, v:0}, new_max_version={val:1729061880223774001, v:0}, last_min_version={val:1729061880223774000, v:0}, last_max_version={val:1729061880223774000, v:0}, total_time_us=1753, delta_us=401442) [2024-10-16 14:58:00.625237] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061880223774001, v:0}, min_version={val:1729061880223774001, v:0}, max_version={val:1729061880223774001, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:58:00.628799] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061880159095003, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=469698, in_cluster_service=true, cluster_version={val:1729061880063974000, v:0}, min_cluster_version={val:1729061880063974000, v:0}, max_cluster_version={val:1729061880063974000, v:0}, get_cluster_version_err=0, cluster_version_delta=564819, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244343, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=551958632, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:00.629791] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061880628827000, start_id=1729061880628827000, end_id=1729061880628827001) [2024-10-16 14:58:00.651479] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061880645667000, start_id=1729061880645667000, end_id=1729061880645667001) [2024-10-16 14:58:00.651849] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:00.652374] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=310, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:00.652390] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=159, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:00.652388] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797D8C-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10428) [2024-10-16 14:58:00.652428] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:00.652441] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=696, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:00.652445] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:00.659907] INFO pn_print_diag_info (group.c:715) [21190][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO client:0x2b0dcae96048_"10.0.4.9:2890"_"10.0.4.9:2890"_138_1722494489055968_1, write_queue=0/0, write=437429476/305996143886, read=437429476/101222119284, doing=0, done=437429476, write_time=438170840361, read_time=26747434, process_time=49148563 [2024-10-16 14:58:00.659923] INFO pn_print_diag_info (group.c:715) [21190][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO client:0x2b0dcaebaa98_"10.0.4.8:2890"_"10.0.4.8:2890"_139_1722494324849444_1, write_queue=0/0, write=448034883/266113938531, read=448034883/103728968791, doing=0, done=448034883, write_time=449092006701, read_time=28233376, process_time=51363008 [2024-10-16 14:58:00.659931] INFO pn_print_diag_info (group.c:715) [21190][pnio1][T0][Y0-0000000000000000-0-0] [lt=7] PNIO client:0x2b0dcafb54a8_"10.0.4.7:2890"_"10.0.4.7:2890"_156_1722494143237088_1, write_queue=0/0, write=56866547/31581277750, read=56866547/13548697393, doing=0, done=56866548, write_time=56654108653, read_time=3297341, process_time=5703840 [2024-10-16 14:58:00.659935] INFO pn_print_diag_info (group.c:727) [21190][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO server:0x2b0dcaec8a58_"10.0.4.9:51578"_181_1722494490520165, write_queue=0/0, write=586896158/219120923664, read=586896159/380775796483, doing=1, done=586896158, write_time=586622642866, read_time=96270687, process_time=122221934 [2024-10-16 14:58:00.659939] INFO pn_print_diag_info (group.c:727) [21190][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO server:0x2b0dfb62d468_"10.0.4.8:36178"_158_1722494326962968, write_queue=0/0, write=644996261/242061660917, read=644996261/480829775313, doing=0, done=644996261, write_time=645147437965, read_time=120625017, process_time=150071678 [2024-10-16 14:58:00.659942] INFO pn_print_diag_info (group.c:727) [21190][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x2b0dfb62c048_"10.0.4.7:34820"_157_1722494143237240, write_queue=0/0, write=56866547/13548697393, read=56866547/31581277750, doing=0, done=56866547, write_time=56818473623, read_time=3728877, process_time=5996073 [2024-10-16 14:58:00.659945] INFO pn_print_diag_info (group.c:734) [21190][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO client_send:942360443/603714615413, client_queue_time=135926633522, cnt=3, server_send:1288791801/474743218634, server_queue_time=85735713023, cnt=3 [2024-10-16 14:58:00.660063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA88-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061880659983000, start_id=1729061880659983000, end_id=1729061880659983001) [2024-10-16 14:58:00.662812] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA8C-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061880662090000, start_id=1729061880662090000, end_id=1729061880662090001) [2024-10-16 14:58:00.662935] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D15-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061880662090000, start_id=1729061880662090000, end_id=1729061880662090001) [2024-10-16 14:58:00.664620] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA8B-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061880664207000, start_id=1729061880664207000, end_id=1729061880664207001) [2024-10-16 14:58:00.666892] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA89-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061880665248000, start_id=1729061880665248000, end_id=1729061880665248001) [2024-10-16 14:58:00.667258] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:00.668472] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=211, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:00.668501] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=103, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:00.668534] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:00.673164] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061880670087000, start_id=1729061880670087000, end_id=1729061880670087001) [2024-10-16 14:58:00.674770] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:00.679880] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.687223] INFO pn_print_diag_info (group.c:715) [21191][pnio1][T0][Y0-0000000000000000-0-0] [lt=6] PNIO client:0x2b0dcaeba048_"10.0.4.9:2890"_"10.0.4.9:2890"_149_1722494489154647_1, write_queue=0/0, write=437407532/306003105962, read=437407532/101215338070, doing=0, done=437407532, write_time=438167424729, read_time=26649601, process_time=47018313 [2024-10-16 14:58:00.687252] INFO pn_print_diag_info (group.c:715) [21191][pnio1][T0][Y0-0000000000000000-0-0] [lt=27] PNIO client:0x2b0dcafb4a58_"10.0.4.8:2890"_"10.0.4.8:2890"_136_1722494324847151_1, write_queue=0/0, write=448051620/266078181143, read=448051620/103730861826, doing=0, done=448051620, write_time=449013440980, read_time=27926044, process_time=54206665 [2024-10-16 14:58:00.687257] INFO pn_print_diag_info (group.c:715) [21191][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO client:0x2b0dcae96a98_"10.0.4.7:2890"_"10.0.4.7:2890"_185_1722494146923218_1, write_queue=0/0, write=56871668/31568381177, read=56871668/13549390139, doing=0, done=56871668, write_time=56671471094, read_time=3218742, process_time=7153895 [2024-10-16 14:58:00.687261] INFO pn_print_diag_info (group.c:727) [21191][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x2b0dcaec9468_"10.0.4.9:51584"_184_1722494490560083, write_queue=0/0, write=586888196/219105914212, read=586888196/380790158518, doing=0, done=586888196, write_time=586607063409, read_time=97999231, process_time=125094423 [2024-10-16 14:58:00.687251] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92A1-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=256, avg_log_submit_to_slide_time=20156) [2024-10-16 14:58:00.687265] INFO pn_print_diag_info (group.c:727) [21191][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x2b0dcaec8048_"10.0.4.8:36180"_161_1722494326965496, write_queue=0/0, write=644986748/242076490042, read=644986749/480775306494, doing=1, done=644986748, write_time=645122239028, read_time=130602709, process_time=160124862 [2024-10-16 14:58:00.687268] INFO pn_print_diag_info (group.c:727) [21191][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x2b0dcae974e8_"10.0.4.7:34850"_186_1722494146923428, write_queue=0/0, write=56871668/13549390139, read=56871668/31568381177, doing=0, done=56871668, write_time=56827756713, read_time=3624016, process_time=5799196 [2024-10-16 14:58:00.687271] INFO pn_print_diag_info (group.c:734) [21191][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO client_send:942360091/603672357099, client_queue_time=135505468569, cnt=3, server_send:1288779138/474744796781, server_queue_time=85587158985, cnt=3 [2024-10-16 14:58:00.687293] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:00.689690] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=20] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:58:00.689704] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] try gc retain ctx [2024-10-16 14:58:00.689708] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:00.689885] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:00.689905] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=20] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:00.689911] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.690355] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061897008855000, latest_log_ts:{val:1729061880424064001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:00.700039] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:58:00.709698] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22141][T1002_TntShared][T1002][Y0-0000000000000000-0-0] [lt=15] clear dtl interm result cost(us)(clear_cost=929, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:58:00.717725] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D6-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:00.717756] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D6-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061880159095003, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061880159095003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:00.721226] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D6-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061880720674000, start_id=1729061880720674000, end_id=1729061880720674001) [2024-10-16 14:58:00.721311] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D6-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9330, avg_group_log_size=666) [2024-10-16 14:58:00.721322] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D6-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9330) [2024-10-16 14:58:00.721628] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9330) [2024-10-16 14:58:00.721887] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797D8E-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=613, avg_log_submit_to_slide_time=760) [2024-10-16 14:58:00.721936] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:00.727830] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:00.736724] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:00.736745] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=14] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=218) [2024-10-16 14:58:00.759220] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:00.759252] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=19] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:00.759260] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:00.760758] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:58:00.771657] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C63-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:00.771687] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C63-0-0] [lt=19] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061880160163004, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061880160163004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:00.772332] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get_max_applied_scn(scn={val:1729061880670087000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264932257}, palf_committed_end_scn_:{val:1729061880731308000, v:0}, last_check_scn_:{val:1729061880731308000, v:0}, max_applied_cb_scn_:{val:1729061880670087000, v:0}}) [2024-10-16 14:58:00.772356] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:58:00.772846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C63-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061880772332000, start_id=1729061880772332000, end_id=1729061880772332001) [2024-10-16 14:58:00.772870] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:00.776842] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C63-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061880776572000, start_id=1729061880776572000, end_id=1729061880776572001) [2024-10-16 14:58:00.779942] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.782620] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=21] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:00.787792] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.788137] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.788150] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.788401] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.788626] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.788736] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.788741] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.788855] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.788870] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.788981] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.789638] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:00.789912] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.824547] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061880824099000, start_id=1729061880824099000, end_id=1729061880824099001) [2024-10-16 14:58:00.824639] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:00.824654] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=3786, avg_group_log_size=315) [2024-10-16 14:58:00.824664] INFO [LIB] stat (utility.h:1165) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:00.824949] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=229, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:00.824969] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=26, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:00.824985] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:00.825179] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10EC-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=27, avg_log_gen_to_submit_time=57, avg_log_submit_to_flush_time=275, avg_log_submit_to_slide_time=619) [2024-10-16 14:58:00.829940] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061880829388000, start_id=1729061880829388000, end_id=1729061880829388001) [2024-10-16 14:58:00.833198] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1003}, {id:1}]) [2024-10-16 14:58:00.836178] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:00.836824] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:00.836845] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:00.837060] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:00.837374] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=7] [APPLY STAT CB APPEND COST TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=45, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:58:00.837388] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=14] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=650, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:58:00.837393] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:58:00.837398] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=-475, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:58:00.837402] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB TOTAL TIME](cur_stat_count=16, stat_interval=5000000, avg (count/cost)=220, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:58:00.851688] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061880850366000, start_id=1729061880850366000, end_id=1729061880850366001) [2024-10-16 14:58:00.851961] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=261, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:00.851981] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=196, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:00.852007] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=499, avg_log_submit_to_slide_time=882) [2024-10-16 14:58:00.852018] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:00.861625] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F50-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:00.861646] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F50-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:00.861754] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6057, mem_used=12478464) [2024-10-16 14:58:00.867532] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1581383680, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:00.868416] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061880867708000, start_id=1729061880867708000, end_id=1729061880867708001) [2024-10-16 14:58:00.873241] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=51] get number(ret=0, service_type_=0, range=1, base_id=1729061880872987000, start_id=1729061880872987000, end_id=1729061880872987001) [2024-10-16 14:58:00.875087] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78E9-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938462, log_task={header:{begin_lsn:{lsn:213058263676}, end_lsn:{lsn:213058263798}, log_id:220938462, min_scn:{val:1729061880662090001, v:0}, max_scn:{val:1729061880662090001, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058263554}, prev_proposal_id:88, committed_end_lsn:{lsn:213058263676}, data_checksum:1715074492, accum_checksum:209291002, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061880775012, freeze_ts:1729061880775012, submit_ts:1729061880775017, flushed_ts:1729061880775300, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:283}, fs_cb_begin_ts=1729061880875081, log_life_time=100069) [2024-10-16 14:58:00.879987] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.887562] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:00.887575] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:00.889942] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.890896] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:58:00.891081] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4B-0-0] [lt=8] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061880564734001, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061878330616000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890098662000, latest_log_ts:{val:1729061873558226001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:00.891216] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4B-0-0] [lt=84] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061880564734001, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061878330616000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:00.891282] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4B-0-0] [lt=31] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061880662090002, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061878330616000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:00.891346] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4B-0-0] [lt=31] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061880668315001, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061878330616000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:00.891375] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4B-0-0] [lt=28] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:58:00.900746] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:00.919213] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=15] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:00.924777] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:00.925762] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=10] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:00.936879] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:00.936896] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:00.937473] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE400AE-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:00.937484] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE400AE-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=603, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:00.940682] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBEC-0-0] [lt=34] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061880908413, keepalive_ts:-1}) [2024-10-16 14:58:00.940706] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBEC-0-0] [lt=23] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061880908413, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:00.942518] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15E8-0-0] [lt=12] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:00.945763] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3912, mem_used=8318976) [2024-10-16 14:58:00.949205] INFO pn_print_diag_info (group.c:715) [21192][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO client:0x2b0dfb6c2a98_"10.0.4.9:2890"_"10.0.4.9:2890"_137_1722494489055846_1, write_queue=1/353, write=437428798/305947693864, read=437428798/101221258804, doing=1, done=437428798, write_time=438147577113, read_time=27325079, process_time=49225740 [2024-10-16 14:58:00.949215] INFO pn_print_diag_info (group.c:715) [21192][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO client:0x2b0dcaebb4e8_"10.0.4.8:2890"_"10.0.4.8:2890"_140_1722494324870779_1, write_queue=0/0, write=448047404/266106857734, read=448047404/103730404558, doing=0, done=448047404, write_time=449043577593, read_time=28656930, process_time=46770129 [2024-10-16 14:58:00.949220] INFO pn_print_diag_info (group.c:715) [21192][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO client:0x2b0dfb6c2048_"10.0.4.7:2890"_"10.0.4.7:2890"_147_1722494143917958_1, write_queue=0/0, write=56853611/31567926886, read=56853611/13544507106, doing=0, done=56853611, write_time=56627670825, read_time=3301729, process_time=6350529 [2024-10-16 14:58:00.949224] INFO pn_print_diag_info (group.c:727) [21192][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x2b0dcaed7468_"10.0.4.9:51576"_162_1722494490518716, write_queue=0/0, write=586927507/218995394166, read=586927507/380767530834, doing=0, done=586927507, write_time=586662257521, read_time=97563621, process_time=124253063 [2024-10-16 14:58:00.949228] INFO pn_print_diag_info (group.c:727) [21192][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x2b0dfb62ca58_"10.0.4.8:36176"_155_1722494326891429, write_queue=0/0, write=645012223/242077563930, read=645012223/480811899892, doing=0, done=645012223, write_time=645151946436, read_time=127207076, process_time=156956546 [2024-10-16 14:58:00.949231] INFO pn_print_diag_info (group.c:727) [21192][pnio1][T0][Y0-0000000000000000-0-0] [lt=2] PNIO server:0x2b0dfb6c34e8_"10.0.4.7:34838"_150_1722494143918150, write_queue=0/0, write=56853611/13544507106, read=56853611/31567926886, doing=0, done=56853611, write_time=56795251094, read_time=3779057, process_time=6019839 [2024-10-16 14:58:00.949234] INFO pn_print_diag_info (group.c:734) [21192][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO client_send:942359361/603645609459, client_queue_time=135674106796, cnt=3, server_send:1288826061/474629634812, server_queue_time=85582750118, cnt=3 [2024-10-16 14:58:00.949874] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:00.949892] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:00.949903] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=9] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=153) [2024-10-16 14:58:00.960830] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:58:00.979081] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:58:00.980049] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.984959] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061876978366, last_first_register_time_us=1722494150998907, register_time_us=1729061876978366, curr_time_us=1729061880984958) [2024-10-16 14:58:00.984986] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=12] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061880984971, last_submit_register_req_time_us=1729061880984971, register_parent_reason=3) [2024-10-16 14:58:00.984993] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=6] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:00.985051] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061876880590, last_first_register_time_us=1722494157071428, register_time_us=1729061876880590, curr_time_us=1729061880985050) [2024-10-16 14:58:00.985066] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=6] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061880985057, last_submit_register_req_time_us=1729061880985057, register_parent_reason=3) [2024-10-16 14:58:00.985070] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=5] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:00.985610] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:00.985621] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:58:00.985631] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:00.990006] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:00.990230] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D80-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:00.990254] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D80-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061880662090004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061880662090004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:00.990327] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10ED-0-0] [lt=19] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3908) [2024-10-16 14:58:00.991290] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D80-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061880991241000, start_id=1729061880991241000, end_id=1729061880991241001) [2024-10-16 14:58:01.004074] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=15] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:58:01.004397] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D80-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061881003841000, start_id=1729061881003841000, end_id=1729061881003841001) [2024-10-16 14:58:01.004465] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D80-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:01.004479] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D80-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:01.010378] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=16] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:200070}) [2024-10-16 14:58:01.010404] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=23] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:200070}, max_saved_version=0) [2024-10-16 14:58:01.010411] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:200070}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.010426] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0efb1f4200, key:{tablet_id:{id:200070}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881010423}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881010423, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f1f144dd0, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0efb1f4200, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f1f144dd0, ref_cnt:2, is_inited:true, tablet_id:{id:200070}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0efb1f4200, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:200070}}}) [2024-10-16 14:58:01.010477] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=51] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:200070}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.010486] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:200070}, table_store_addr={ptr:0x2b0e0c0f268c, addr:[7667511-10399-0-2305843633057837056-1](offset=143360,size=581,type=2,seq=1)}, this=0x2b0e0c0f20a0) [2024-10-16 14:58:01.010533] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=9] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:1152921504606846989}) [2024-10-16 14:58:01.010542] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846989}, max_saved_version=0) [2024-10-16 14:58:01.010553] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=10] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846989}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.010558] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=4] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e44a46200, key:{tablet_id:{id:1152921504606846989}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881010558}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881010558, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f16e831b0, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e44a46200, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f16e831b0, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846989}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e44a46200, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:1152921504606846989}}}) [2024-10-16 14:58:01.010584] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=25] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846989}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.010589] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=4] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846989}, table_store_addr={ptr:0x2b0e695b46b0, addr:[7667511-10399-0-2305843631984111616-1](offset=159744,size=580,type=2,seq=1)}, this=0x2b0e695b40a0) [2024-10-16 14:58:01.010619] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=4] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:1152921504606846990}) [2024-10-16 14:58:01.010625] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=5] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846990}, max_saved_version=0) [2024-10-16 14:58:01.010628] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=3] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846990}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.010633] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=4] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e4565f8b0, key:{tablet_id:{id:1152921504606846990}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881010632}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881010632, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f16e829f0, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e4565f8b0, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f16e829f0, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846990}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e4565f8b0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:1152921504606846990}}}) [2024-10-16 14:58:01.010660] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=27] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846990}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.010665] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=4] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846990}, table_store_addr={ptr:0x2b0df1ca66b0, addr:[7667511-10399-0-2305843631984103424-1](offset=151552,size=580,type=2,seq=1)}, this=0x2b0df1ca60a0) [2024-10-16 14:58:01.010698] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846991}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328873906000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:58:01.010752] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=10] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:1152921504606846991}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:58:01.010764] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=8] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:1152921504606846991}) [2024-10-16 14:58:01.010772] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=3] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846991}, max_saved_version=0) [2024-10-16 14:58:01.010775] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=3] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846991}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.010780] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=4] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e44a48240, key:{tablet_id:{id:1152921504606846991}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881010780}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881010780, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f16e82fc0, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e44a48240, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f16e82fc0, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846991}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e44a48240, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:1152921504606846991}}}) [2024-10-16 14:58:01.010808] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=27] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846991}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.010813] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=3] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846991}, table_store_addr={ptr:0x2b0e6b1a26b0, addr:[7667511-10399-0-2305843631984066560-1](offset=114688,size=580,type=2,seq=1)}, this=0x2b0e6b1a20a0) [2024-10-16 14:58:01.010834] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846992}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328873906000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:58:01.010863] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=8] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:1152921504606846992}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:58:01.010874] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:1152921504606846992}) [2024-10-16 14:58:01.010881] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=6] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846992}, max_saved_version=0) [2024-10-16 14:58:01.010884] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=3] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846992}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.010892] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=6] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e44aa4890, key:{tablet_id:{id:1152921504606846992}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881010892}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881010892, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f16e83780, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e44aa4890, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f16e83780, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846992}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e44aa4890, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:1152921504606846992}}}) [2024-10-16 14:58:01.010919] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=27] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846992}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.010924] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=4] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846992}, table_store_addr={ptr:0x2b0e6aa4c6b0, addr:[7667511-10399-0-2305843630910394368-1](offset=184320,size=579,type=2,seq=1)}, this=0x2b0e6aa4c0a0) [2024-10-16 14:58:01.010944] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846993}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328873906000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:58:01.010964] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:1152921504606846993}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:58:01.010974] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:1152921504606846993}) [2024-10-16 14:58:01.010982] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=7] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846993}, max_saved_version=0) [2024-10-16 14:58:01.010989] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=6] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846993}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.010994] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=3] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e4566b220, key:{tablet_id:{id:1152921504606846993}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881010993}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881010993, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f16ffbb60, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e4566b220, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f16ffbb60, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846993}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e4566b220, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:1152921504606846993}}}) [2024-10-16 14:58:01.011020] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=26] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846993}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.011025] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1184-0-0] [lt=4] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846993}, table_store_addr={ptr:0x2b0e6656c6b0, addr:[7667511-10399-0-2305843630910427136-1](offset=217088,size=579,type=2,seq=1)}, this=0x2b0e6656c0a0) [2024-10-16 14:58:01.011781] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=6] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:200067}) [2024-10-16 14:58:01.011798] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=12] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:200067}, max_saved_version=0) [2024-10-16 14:58:01.011804] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=5] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:200067}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.011814] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=5] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0efb1e90a0, key:{tablet_id:{id:200067}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881011812}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881011812, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f16ffafc0, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0efb1e90a0, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f16ffafc0, ref_cnt:2, is_inited:true, tablet_id:{id:200067}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0efb1e90a0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:200067}}}) [2024-10-16 14:58:01.011854] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=39] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:200067}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.011861] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=5] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:200067}, table_store_addr={ptr:0x2b0e6a20468c, addr:[7667511-10399-0-2305843634131644416-1](offset=208896,size=582,type=2,seq=1)}, this=0x2b0e6a2040a0) [2024-10-16 14:58:01.011897] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=10] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846982}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328401377000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:58:01.011934] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=9] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:1152921504606846982}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:58:01.011947] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=8] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:1152921504606846982}) [2024-10-16 14:58:01.011956] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=6] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846982}, max_saved_version=0) [2024-10-16 14:58:01.011962] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=7] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846982}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.011968] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=3] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e4565e890, key:{tablet_id:{id:1152921504606846982}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881011967}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881011967, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f16e85590, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e4565e890, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f16e85590, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846982}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e4565e890, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:1152921504606846982}}}) [2024-10-16 14:58:01.012001] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=32] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846982}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.012006] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=5] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846982}, table_store_addr={ptr:0x2b0df1c5e6d4, addr:[7667511-10399-0-2305843631984001024-1](offset=49152,size=580,type=2,seq=1)}, this=0x2b0df1c5e0a0) [2024-10-16 14:58:01.012033] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=7] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:1152921504606846983}) [2024-10-16 14:58:01.012039] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=5] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846983}, max_saved_version=0) [2024-10-16 14:58:01.012042] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=4] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846983}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.012051] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=3] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e6b40c990, key:{tablet_id:{id:1152921504606846983}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881012051}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881012051, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f16e84800, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e6b40c990, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f16e84800, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846983}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e6b40c990, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:1152921504606846983}}}) [2024-10-16 14:58:01.012075] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=23] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846983}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.012079] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=4] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846983}, table_store_addr={ptr:0x2b0e6b6ee6b0, addr:[7667511-10399-0-2305843630910267392-1](offset=57344,size=579,type=2,seq=1)}, this=0x2b0e6b6ee0a0) [2024-10-16 14:58:01.012102] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=4] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:1152921504606846984}) [2024-10-16 14:58:01.012106] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=3] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846984}, max_saved_version=0) [2024-10-16 14:58:01.012109] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=4] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846984}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.012114] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=3] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e44a42990, key:{tablet_id:{id:1152921504606846984}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881012114}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881012114, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f2a0b4be0, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e44a42990, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f2a0b4be0, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846984}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e44a42990, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:1152921504606846984}}}) [2024-10-16 14:58:01.012137] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=22] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846984}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.012141] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=4] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846984}, table_store_addr={ptr:0x2b0df1d906b0, addr:[7667511-10399-0-2305843633057800192-1](offset=106496,size=581,type=2,seq=1)}, this=0x2b0df1d900a0) [2024-10-16 14:58:01.012158] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:197) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=4] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846985}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1717560328401377000, delete_commit_version:-1}, snapshot_version=9223372036854775807) [2024-10-16 14:58:01.012181] INFO [STORAGE] check_schema_version_with_cache (ob_tablet.cpp:7156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=5] refresh ddl data cache(ret=0, tablet_meta_.ls_id_={id:1001}, tablet_id={id:1152921504606846985}, ddl_data_cache_={redefined:false, schema_version:0, snapshot_version:0}) [2024-10-16 14:58:01.012188] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=4] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1001}, tablet_meta.tablet_id_={id:1152921504606846985}) [2024-10-16 14:58:01.012192] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=3] success to init medium clog recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846985}, max_saved_version=0) [2024-10-16 14:58:01.012195] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=3] success to init storage recorder(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846985}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.012200] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=4] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778253675001, v:0}, new_clog_checkpoint_scn_:{val:1729061778253675001, v:0}, for_replay_:false, for_inc_direct_load_:false}, ret=0, ls_id={id:1001}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e44aa68d0, key:{tablet_id:{id:1152921504606846985}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778253675001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881012199}, ls_id_:{id:1001}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881012199, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2611a0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f2a0b4fc0, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e44aa68d0, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1001}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f2a0b4fc0, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846985}, freezer:0x2b0e2611a0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e44aa68d0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26114ed0}, ls_id:{id:1001}, tablet_id:{id:1152921504606846985}}}) [2024-10-16 14:58:01.012223] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=23] succeeded to create memtable for tablet(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846985}, clog_checkpoint_scn={val:1729061778253675001, v:0}, schema_version=0, for_replay=false) [2024-10-16 14:58:01.012227] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1185-0-0] [lt=4] table store update memtable success(ret=0, ls_id={id:1001}, tablet_id={id:1152921504606846985}, table_store_addr={ptr:0x2b0e0c0ce6b0, addr:[7663857-6566-0-2305843633058488320-1](offset=794624,size=581,type=2,seq=1)}, this=0x2b0e0c0ce0a0) [2024-10-16 14:58:01.024589] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061881023961000, start_id=1729061881023961000, end_id=1729061881023961001) [2024-10-16 14:58:01.024965] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4372) [2024-10-16 14:58:01.025307] WDIAG [STORAGE.TRANS] update_logic_clock_ (ob_tx_free_route.cpp:527) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1186-0-0] [lt=4][errcode=0] logic clock is fast more than 1 day(logic_clock=1729061881012338, tx={this:0x2b0ec37cd910, tx_id:{txid:392223473}, state:3, addr:"10.0.4.7:2890", tenant_id:1002, session_id:3221515920, assoc_session_id:3221515920, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:659781}, op_sn:3, alloc_ts:1729061880351451, active_ts:1729061881009108, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:1729062481009108, coord_id:{id:-1}, parts:[{id:{id:1001}, addr:"10.0.4.7:2890", epoch:36834321025869743, first_scn:{branch:0, seq:659781}, last_scn:{branch:0, seq:661895}, last_touch_ts:1729061881009113}], exec_info_reap_ts:1729061881009112, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061880349331, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:58:01.025414] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1186-0-0] [lt=42] get number(ret=0, service_type_=0, range=1, base_id=1729061881025146000, start_id=1729061881025146000, end_id=1729061881025146001) [2024-10-16 14:58:01.025567] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1186-0-0] [lt=10] rpc statistics(total_trans_msg_count=4, total_batch_msg_count=1) [2024-10-16 14:58:01.025607] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45CB7-0-0] [lt=26] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:01.025616] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45CB7-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=633, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:01.030059] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061881029864000, start_id=1729061881029864000, end_id=1729061881029864001) [2024-10-16 14:58:01.030215] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:58:01.030223] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=6] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151574520, tenant_id=1) [2024-10-16 14:58:01.030236] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:01.030961] INFO [RPC] process (ob_batch_processor.cpp:106) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCB3F-0-0] [lt=6] batch rpc statistics(clog_batch_nodelay_cnt=0, clog_batch_cnt=0, trx_batch_cnt=1, sql_batch_cnt=0) [2024-10-16 14:58:01.031310] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1002}, tablet_meta.tablet_id_={id:200065}) [2024-10-16 14:58:01.031328] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] success to init medium clog recorder(ret=0, ls_id={id:1002}, tablet_id={id:200065}, max_saved_version=0) [2024-10-16 14:58:01.031334] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] success to init storage recorder(ret=0, ls_id={id:1002}, tablet_id={id:200065}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.031356] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778311878001, v:0}, new_clog_checkpoint_scn_:{val:1729061778311878001, v:0}, for_replay_:true, for_inc_direct_load_:false}, ret=0, ls_id={id:1002}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e6b411220, key:{tablet_id:{id:200065}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778311878001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881031353}, ls_id_:{id:1002}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881031353, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2618e0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f2a0b5970, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e6b411220, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1002}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f2a0b5970, ref_cnt:2, is_inited:true, tablet_id:{id:200065}, freezer:0x2b0e2618e0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e6b411220, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26188ed0}, ls_id:{id:1002}, tablet_id:{id:200065}}}) [2024-10-16 14:58:01.031401] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=45] succeeded to create memtable for tablet(ret=0, ls_id={id:1002}, tablet_id={id:200065}, clog_checkpoint_scn={val:1729061778311878001, v:0}, schema_version=0, for_replay=true) [2024-10-16 14:58:01.031410] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] table store update memtable success(ret=0, ls_id={id:1002}, tablet_id={id:200065}, table_store_addr={ptr:0x2b0e6b67068c, addr:[7663857-6566-0-2305843504209911808-1](offset=1236992,size=461,type=2,seq=1)}, this=0x2b0e6b6700a0) [2024-10-16 14:58:01.036187] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107548][][T1002][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1002, ret=0, thread_count_=191) [2024-10-16 14:58:01.036206] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107548][][T1002][Y0-0000000000000000-0-0] [lt=18] Init thread local success [2024-10-16 14:58:01.036230] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107548][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4350-0-0] [lt=4] init point thread id with(&point=0x55e3435cb580, idx_=1052, point=[thread id=107548, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107548) [2024-10-16 14:58:01.036398] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=19] worker thread created(tenant_->id()=1002, group_id_=1, token=1) [2024-10-16 14:58:01.036844] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO [ratelimit] time: 1729061881036843, bytes: 4446822080556, bw: 0.370157 MB/s, add_ts: 1000359, add_bytes: 388277 [2024-10-16 14:58:01.038445] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:01.038471] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=255, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:01.038479] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=31, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:01.038484] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:01.038616] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD453-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=268, avg_log_submit_to_slide_time=570) [2024-10-16 14:58:01.038774] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE400B0-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:01.038780] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE400B0-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=616, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:01.043282] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:58:01.043296] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:01.043309] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=292003835, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:58:01.051686] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061881050282000, start_id=1729061881050282000, end_id=1729061881050282001) [2024-10-16 14:58:01.061726] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F54-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:01.062049] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:01.062257] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD455-0-0] [lt=25] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:01.062266] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD455-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9932, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:01.065726] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061881065386000, start_id=1729061881065386000, end_id=1729061881065386001) [2024-10-16 14:58:01.065747] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:01.065761] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061881065758], gts:1729061881065386000, latest_srr:[mts=1729061881065758]}) [2024-10-16 14:58:01.067821] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:01.068376] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061881067488000, start_id=1729061881067488000, end_id=1729061881067488001) [2024-10-16 14:58:01.070873] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Cache wash clean map node details(ret=0, clean_node_count=13, clean_time=0, clean_start_pos=1635764, clean_num=62914) [2024-10-16 14:58:01.073317] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061881072310000, start_id=1729061881072310000, end_id=1729061881072310001) [2024-10-16 14:58:01.075115] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78ED-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=2257) [2024-10-16 14:58:01.075340] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:01.080107] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.087257] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:01.090079] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.094723] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:58:01.094902] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D49-0-0] [lt=11] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061880724989003, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061878062469003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890721265000, latest_log_ts:{val:1729061874059797001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:01.095080] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D49-0-0] [lt=86] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061880724989003, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061878062469003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:01.095228] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D49-0-0] [lt=71] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061880662090005, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061878062469003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:01.095397] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D49-0-0] [lt=78] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061880662090003, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061878062469003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:01.095463] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D49-0-0] [lt=63] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:58:01.123833] INFO [SQL.PC] need_check_schema_version (ob_plan_cache_value.cpp:2237) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B710F161F-0-0] [lt=14] need check schema(new_schema_version=1724725354537200, cached_tenant_schema_version=1724725354537200, is_contain_synonym()=false, contain_sys_name_table_=true, is_contain_tmp_tbl()=false, is_contain_sys_pl_object()=false, pc_ctx.sql_ctx_.session_info_->get_has_temp_table_flag()=false, need_check=true, constructed_sql_=SELECT * FROM __all_ls_meta_table WHERE tenant_id = ? ORDER BY tenant_id, ls_id, svr_ip, svr_port) [2024-10-16 14:58:01.128105] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:01.128126] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:01.137874] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4206, mem_used=8318976) [2024-10-16 14:58:01.138537] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=206, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:01.138556] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=33, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:01.149806] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:58:01.152010] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 145.75; ) [2024-10-16 14:58:01.152030] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:01.152068] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.98, rt: 174.75; ) [2024-10-16 14:58:01.152073] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:01.152081] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:01.159867] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=8] current monitor number(seq_=-1) [2024-10-16 14:58:01.159990] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D18-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061881159563000, start_id=1729061881159563000, end_id=1729061881159563001) [2024-10-16 14:58:01.160193] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA91-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061881159563000, start_id=1729061881159563000, end_id=1729061881159563001) [2024-10-16 14:58:01.160240] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA8F-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061881159563000, start_id=1729061881159563000, end_id=1729061881159563001) [2024-10-16 14:58:01.160433] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D19-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061881159563000, start_id=1729061881159563000, end_id=1729061881159563001) [2024-10-16 14:58:01.160438] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA90-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061881159563000, start_id=1729061881159563000, end_id=1729061881159563001) [2024-10-16 14:58:01.160881] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:58:01.162299] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92AA-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=349, avg_log_submit_to_slide_time=9933) [2024-10-16 14:58:01.162722] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:01.170429] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=7] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:01.170446] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:01.170454] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=7] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:58:01.170458] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:58:01.170461] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:58:01.170464] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16729) [2024-10-16 14:58:01.180171] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.181785] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=47] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=25, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:01.181804] INFO [LIB] stat (utility.h:1165) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:01.187056] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DD5-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1333, avg_group_log_size=121) [2024-10-16 14:58:01.191706] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.196486] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8] set tenant work area memory(tenant_id=1001, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:58:01.196541] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=14] set tenant work area memory(tenant_id=1002, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:58:01.196583] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8] set tenant work area memory(tenant_id=1003, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:58:01.196638] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6] set tenant work area memory(tenant_id=1004, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:58:01.196757] EDIAG [SHARE.CONFIG] check_500_tenant_hold (ob_server_config.cpp:438) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] the hold memory of tenant_500 is over the reserved memory(hold=1082318848, reserved=1073741824) BACKTRACE:0x1a5796b1 0x6847325 0x6a4dfba 0x6a4d9a4 0x6a4d8e0 0x6a4d707 0x1479497f 0x11dea49b 0x6809ea1 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d [2024-10-16 14:58:01.196773] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:32) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=15] === Run print tenant memory usage task === [2024-10-16 14:58:01.196805] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:102) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4] ====== tenants memory info ====== === TENANTS MEMORY INFO === divisive_memory_used= 220,811,264 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,082,318,848 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 19,554,304 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 17,457,152 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 1 now= 1,729,061,881,195,459 active_memstore_used= 247,463,936 total_memstore_used= 245,366,784 total_memstore_hold= 245,366,784 memstore_freeze_trigger_limit= 151,574,520 memstore_limit= 858,993,440 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 1,984,401,408 max_mem_memstore_can_get_now= 757,872,634 memstore_alloc_pos= 20,002,635,776 memstore_frozen_pos= 19,755,171,840 memstore_reclaimed_pos= 19,757,268,992 [TENANT_MEMORY] tenant_id= 1,001 now= 1,729,061,881,195,459 active_memstore_used= 50,331,648 total_memstore_used= 46,137,344 total_memstore_hold= 46,137,344 memstore_freeze_trigger_limit= 54,012,700 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 911,101,952 max_mem_memstore_can_get_now= 270,063,594 memstore_alloc_pos= 32,719,765,504 memstore_frozen_pos= 32,669,433,856 memstore_reclaimed_pos= 32,673,628,160 [TENANT_MEMORY] tenant_id= 1,002 now= 1,729,061,881,195,459 active_memstore_used= 46,137,344 total_memstore_used= 46,137,344 total_memstore_hold= 46,137,344 memstore_freeze_trigger_limit= 28,020,300 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 973,893,632 max_mem_memstore_can_get_now= 140,101,594 memstore_alloc_pos= 465,953,619,968 memstore_frozen_pos= 465,907,482,624 memstore_reclaimed_pos= 465,907,482,624 [TENANT_MEMORY] tenant_id= 1,003 now= 1,729,061,881,195,459 active_memstore_used= 60,817,408 total_memstore_used= 60,817,408 total_memstore_hold= 60,817,408 memstore_freeze_trigger_limit= 54,641,860 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 927,879,168 max_mem_memstore_can_get_now= 273,209,308 memstore_alloc_pos= 34,273,755,136 memstore_frozen_pos= 34,212,937,728 memstore_reclaimed_pos= 34,212,937,728 [TENANT_MEMORY] tenant_id= 1,004 now= 1,729,061,881,195,459 active_memstore_used= 39,845,888 total_memstore_used= 39,845,888 total_memstore_hold= 39,845,888 memstore_freeze_trigger_limit= 27,628,320 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 956,841,984 max_mem_memstore_can_get_now= 138,141,688 memstore_alloc_pos= 1,602,224,128 memstore_frozen_pos= 1,562,378,240 memstore_reclaimed_pos= 1,562,378,240 [2024-10-16 14:58:01.196866] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:114) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8] [CHUNK_MGR] limit= 12,884,901,888 hold= 7,745,781,760 total_hold= 8,636,071,936 used= 6,873,448,448 freelists_hold= 872,333,312 total_maps= 388,959 total_unmaps= 385,795 large_maps= 301 large_unmaps= 0 huge_maps= 200 huge_unmaps= 187 memalign=0 resident_size= 8,394,637,312 virtual_memory_used= 9,717,915,648 [CHUNK_MGR] 2 MB_CACHE: hold= 132,120,576 free= 63 pushes= 6,196,542,334 pops= 6,196,542,271 maps= 388,458 unmaps= 385,608 [CHUNK_MGR] 4 MB_CACHE: hold= 167,239,680 free= 73 pushes= 6,821,154 pops= 6,821,081 maps= 172 unmaps= 0 [CHUNK_MGR] 6 MB_CACHE: hold= 193,880,064 free= 46 pushes= 6,920 pops= 6,874 maps= 59 unmaps= 0 [CHUNK_MGR] 8 MB_CACHE: hold= 25,247,744 free= 4 pushes= 16,248,858 pops= 16,248,854 maps= 11 unmaps= 0 [CHUNK_MGR] 10 MB_CACHE: hold= 109,318,144 free= 13 pushes= 123 pops= 110 maps= 30 unmaps= 0 [CHUNK_MGR] 12 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [CHUNK_MGR] 14 MB_CACHE: hold= 26,435,584 free= 2 pushes= 4 pops= 2 maps= 9 unmaps= 0 [CHUNK_MGR] 16 MB_CACHE: hold= 33,316,864 free= 2 pushes= 4 pops= 2 maps= 2 unmaps= 0 [CHUNK_MGR] 18 MB_CACHE: hold= 184,774,656 free= 11 pushes= 993 pops= 982 maps= 18 unmaps= 0 [CHUNK_MGR] 20 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [2024-10-16 14:58:01.196902] INFO print (ob_malloc_time_monitor.cpp:39) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4] [MALLOC_TIME_MONITOR] show the distribution of ob_malloc's cost_time [MALLOC_TIME_MONITOR] [ 0, 10): delta_total_cost_time= 59901, delta_count= 128252, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 10, 100): delta_total_cost_time= 2447, delta_count= 160, avg_cost_time= 15 [MALLOC_TIME_MONITOR] [ 100, 1000): delta_total_cost_time= 635, delta_count= 1, avg_cost_time= 635 [MALLOC_TIME_MONITOR] [ 1000, 10000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 10000, 100000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 100000, 1000000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 1000000, 9223372036854775807): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [2024-10-16 14:58:01.201938] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503F-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:01.201955] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD503F-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:01.210064] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=4] PNIO [ratelimit] time: 1729061881210063, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007616, add_bytes: 0 [2024-10-16 14:58:01.210571] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5880-0-0] [lt=26] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:58:01.210675] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5880-0-0] [lt=16][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:01.224836] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3AD140-0-0] [lt=27] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.733881430625915527e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=866, Throttle Trigger(MB)=520, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.150066919627531092e+01) [2024-10-16 14:58:01.224991] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061881223108000, start_id=1729061881223108000, end_id=1729061881223108001) [2024-10-16 14:58:01.225071] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3907) [2024-10-16 14:58:01.230190] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061881229371000, start_id=1729061881229371000, end_id=1729061881229371001) [2024-10-16 14:58:01.238288] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:01.238311] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:01.238584] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:01.238986] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD45A-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=307, avg_log_submit_to_slide_time=620) [2024-10-16 14:58:01.240214] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:58:01.240437] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:58:01.240448] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=8] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:58:01.240455] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:58:01.240460] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:01.242699] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=86, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}, {obj_id:78893917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:78893918, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893919, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893920, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}]) [2024-10-16 14:58:01.242812] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=111] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:01.246494] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD11-0-0] [lt=8][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:01.251759] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061881250917000, start_id=1729061881250917000, end_id=1729061881250917001) [2024-10-16 14:58:01.251982] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:01.251998] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:01.252314] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:58:01.252481] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4534-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:01.252490] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4534-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=906, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:01.253194] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061881251973000, start_id=1729061881251973000, end_id=1729061881251973001) [2024-10-16 14:58:01.253709] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4535-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=490, avg_log_submit_to_slide_time=1137) [2024-10-16 14:58:01.267677] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] no need to update min/max version(new_version={val:1729061880867708000, v:0}, min_version={val:1729061880867708000, v:0}, max_version={val:1729061880867708000, v:0}) [2024-10-16 14:58:01.271166] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:01.273318] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061881272897000, start_id=1729061881272897000, end_id=1729061881272897001) [2024-10-16 14:58:01.279054] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769AB9F-0-0] [lt=18] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12287, add_timestamp:1729061881279043}) [2024-10-16 14:58:01.279076] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5A-0-0] [lt=8] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12287, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:01.279226] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5A-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12287, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881279213, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881279213, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881279213, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:01.279246] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5A-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:12287, add_timestamp:1729061881279043}, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881279213, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881279213, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881279213, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:01.280230] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.283478] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769AB9F-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:01.285076] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79A-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349153469, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410630, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.286712] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=14] start to print blacklist info [2024-10-16 14:58:01.286858] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=12] ls blacklist refresh finish(cost_time=8408) [2024-10-16 14:58:01.288741] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:45338084276}, base_lsn={lsn:45161508864}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:58:01.288790] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=48] get_replay_process success(id={id:1}, replayed_log_size=176575412, unreplayed_log_size=0) [2024-10-16 14:58:01.288808] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1003, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=168, last_replayed_log_size(MB)=168, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:58:01.291755] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.300490] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85129-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:01.300511] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85129-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:01.300737] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:01.310594] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA893-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:01.310621] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA893-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061881159563001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061881159563001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:01.311707] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA893-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061881310984000, start_id=1729061881310984000, end_id=1729061881310984001) [2024-10-16 14:58:01.312845] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=6] ====== traversal_flush timer task ====== [2024-10-16 14:58:01.312949] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAE-0-0] [lt=11] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061889029444000, latest_log_ts:{val:1729061872445301001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:01.313008] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAE-0-0] [lt=55] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:01.315280] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB6-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061881283014, keepalive_ts:-1}) [2024-10-16 14:58:01.315299] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB6-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061881283014, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:01.315344] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB7-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061881283061, keepalive_ts:-1}) [2024-10-16 14:58:01.315350] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB7-0-0] [lt=6] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061881283061, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:01.315763] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1040-0-0] [lt=20] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:58:01.317259] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA893-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061881316277000, start_id=1729061881316277000, end_id=1729061881316277001) [2024-10-16 14:58:01.317344] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA893-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10121, avg_group_log_size=595) [2024-10-16 14:58:01.317357] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA893-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10121) [2024-10-16 14:58:01.317685] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=222, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:01.317697] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10121) [2024-10-16 14:58:01.317707] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=35, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:01.317719] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10121) [2024-10-16 14:58:01.317767] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=45] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:01.317919] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9628-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=3, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:01.317947] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9628-0-0] [lt=28] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=735, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:01.317952] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9628-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=430, avg_log_submit_to_slide_time=700) [2024-10-16 14:58:01.318069] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:01.329671] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=24] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=752, mem_used=2079744) [2024-10-16 14:58:01.335845] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=19] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:01.342851] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=21] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:01.352024] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21659][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3E02-0-0] [lt=8][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349220418, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410631, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.352131] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:58:01.352143] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1098) [2024-10-16 14:58:01.352444] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=323, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:01.352460] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=148, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:01.352471] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:58:01.353454] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4538-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=3542) [2024-10-16 14:58:01.361236] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=9] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:58:01.361633] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:01.367210] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E09-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061881365889000, start_id=1729061881365889000, end_id=1729061881365889001) [2024-10-16 14:58:01.368443] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061881367313000, start_id=1729061881367313000, end_id=1729061881367313001) [2024-10-16 14:58:01.373060] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [WRS] update tenant weak read server version(tenant_id=1001, server_version={version:{val:1729061881251973000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061881373039}, version_delta=121086) [2024-10-16 14:58:01.375348] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78F3-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=13, avg_log_batch_cnt=0, total_group_log_size=2266, avg_group_log_size=174) [2024-10-16 14:58:01.375568] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2266) [2024-10-16 14:58:01.377004] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB7-0-0] [lt=11] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:58:01.377018] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB7-0-0] [lt=9] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=14) [2024-10-16 14:58:01.378727] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C118B-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061881377949000, start_id=1729061881377949000, end_id=1729061881377949001) [2024-10-16 14:58:01.380294] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.387280] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DD9-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:01.388005] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=261, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:01.388022] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:01.388057] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:01.395524] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:01.395660] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.415625] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=12] LogLoopThread round_cost_time(us)(round_cost_time=11) [2024-10-16 14:58:01.428339] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061881426629000, start_id=1729061881426629000, end_id=1729061881426629001) [2024-10-16 14:58:01.430226] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061881428488000, start_id=1729061881428488000, end_id=1729061881428488001) [2024-10-16 14:58:01.437712] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_max_applied_scn(scn={val:1729061881065386000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959728037}, palf_committed_end_scn_:{val:1729061881065386001, v:0}, last_check_scn_:{val:1729061881065386001, v:0}, max_applied_cb_scn_:{val:1729061881065386000, v:0}}) [2024-10-16 14:58:01.437735] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:58:01.437783] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=41] get_max_applied_scn(scn={val:1729061881065386000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023126885}, palf_committed_end_scn_:{val:1729061881065386001, v:0}, last_check_scn_:{val:1729061881065386001, v:0}, max_applied_cb_scn_:{val:1729061881065386000, v:0}}) [2024-10-16 14:58:01.437791] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:58:01.437815] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:58:01.437823] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=5] get_min_unreplayed_log_info(lsn={lsn:213058265576}, scn={val:1729061881159563004, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061881375789, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058265576}, next_to_submit_scn_:{val:1729061881159563004, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058265454}, end_lsn:{lsn:213058265576}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061881159563003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061881159563003, v:0}, data_checksum:3348837161, flag:1}}, init_mode_version:0, accumulate_checksum:3101873755, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061881159563003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561478, miss_cnt:1, cache_read_size:152590705558, hit_ratio:"9.999999928346989808e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:58:01.438743] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:01.454409] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061881452639000, start_id=1729061881452639000, end_id=1729061881452639001) [2024-10-16 14:58:01.455438] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:01.463093] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=15] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:01.465636] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:58:01.465650] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=13] try gc retain ctx [2024-10-16 14:58:01.465653] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:58:01.466249] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061889029444000, latest_log_ts:{val:1729061872445301001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:01.467395] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5B-0-0] [lt=17] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12274, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:01.467678] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5B-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12274, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881467666, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881467666, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881467666, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:01.467697] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5B-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:12274, add_timestamp:1729061881466948}, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881467666, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881467666, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881467666, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:01.468145] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB290-0-0] [lt=10][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:58:01.468344] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=8] Slow write(duration=13817, write_len=4096) [2024-10-16 14:58:01.468353] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=8] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601482, offset=9647802}, end_cursor:ObLogCursor{file_id=192, log_id=47601483, offset=9648086}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e70050, len:2374, log_data_len:284, seq:47601482, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:01.471031] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B7-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061881469445000, start_id=1729061881469445000, end_id=1729061881469445001) [2024-10-16 14:58:01.472746] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=51] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:01.472953] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash time detail, (compute_wash_size_time=85, refresh_score_time=70, wash_time=0) [2024-10-16 14:58:01.473329] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB290-0-0] [lt=14][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:58:01.473788] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061881472352000, start_id=1729061881472352000, end_id=1729061881472352001) [2024-10-16 14:58:01.474888] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601483, offset=9648086}, end_cursor:ObLogCursor{file_id=192, log_id=47601484, offset=9648360}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20fc050, len:2090, log_data_len:274, seq:47601483, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:01.475447] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78F5-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:01.475635] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=328, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:01.475651] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=28, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:01.475853] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D553B-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:01.475866] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D553B-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=15936, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:01.476752] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=2266, hit_cnt=13, read_cnt=13, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:01.476774] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561479, miss_cnt:1, cache_read_size:152590705680, hit_ratio:"9.999999928346990918e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:01.476816] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=45] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=23, stat_interval=1000000, avg (count/cost)=29, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:01.476824] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=23, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:01.479138] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601484, offset=9648360}, end_cursor:ObLogCursor{file_id=192, log_id=47601485, offset=9648634}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21ec050, len:1816, log_data_len:274, seq:47601484, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:01.479369] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601485, offset=9648634}, end_cursor:ObLogCursor{file_id=192, log_id=47601486, offset=9648908}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7366050, len:1542, log_data_len:274, seq:47601485, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:01.479416] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599062, miss_cnt:9, cache_read_size:149777936232, hit_ratio:"9.999999345925816296e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:01.480354] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.483929] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:58:01.483952] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=21] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:58:01.483957] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:58:01.483962] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=3] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:01.486133] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=7] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:58:01.486266] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=130] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:01.489070] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A93-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:01.489087] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A93-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:01.490835] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:01.490848] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:58:01.490861] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=70883887, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:01.492838] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:01.492848] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:58:01.492857] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:58:01.492855] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2C-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:01.492866] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:58:01.492870] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:58:01.492867] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2C-0-0] [lt=10] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:01.492872] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:58:01.492875] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:58:01.492877] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:58:01.492880] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:58:01.492882] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:58:01.492885] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:58:01.495703] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.500563] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8512D-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:01.516346] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] [COMPACTION DAG STATUS] (start_time=1729061871515088, end_time=1729061881516345, dag_status={}) [2024-10-16 14:58:01.516363] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=15] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:01.516370] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:01.516374] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:01.516378] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:01.516384] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:01.516388] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:01.516394] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:01.516397] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:01.516401] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:01.516404] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516413] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516417] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516424] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516429] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516435] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516439] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516446] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516450] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516457] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516460] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516467] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516472] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516484] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516488] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516494] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516499] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516505] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516510] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516517] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516520] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516527] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516531] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516538] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516542] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516549] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516553] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516559] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516563] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516567] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516576] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516580] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516583] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516587] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516591] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516595] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516599] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516608] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516611] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516615] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516619] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516623] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516627] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516630] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516634] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516638] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516641] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516645] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516649] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516652] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516656] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516660] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516664] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516667] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516671] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516674] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516678] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516682] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:01.516686] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:58:01.516690] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:58:01.516694] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:58:01.516696] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:58:01.516700] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:58:01.516703] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:58:01.516705] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:58:01.516707] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:58:01.516710] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:58:01.516712] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:58:01.516715] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:58:01.527610] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257830, offset=39052224}, end_cursor:ObLogCursor{file_id=197, log_id=49257831, offset=39052507}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2068050, len:3136, log_data_len:283, seq:49257830, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:01.529558] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB382-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061881528872000, start_id=1729061881528872000, end_id=1729061881528872001) [2024-10-16 14:58:01.530208] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] [APPLY STAT CB APPEND COST TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=47, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:58:01.530220] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=940, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:58:01.530224] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:58:01.530228] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=-482, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:58:01.530231] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=505, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:58:01.530514] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257831, offset=39052507}, end_cursor:ObLogCursor{file_id=197, log_id=49257832, offset=39052789}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db735a050, len:2853, log_data_len:282, seq:49257831, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:01.532395] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257832, offset=39052789}, end_cursor:ObLogCursor{file_id=197, log_id=49257833, offset=39053071}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72c6050, len:2571, log_data_len:282, seq:49257832, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:01.532552] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257833, offset=39053071}, end_cursor:ObLogCursor{file_id=197, log_id=49257834, offset=39053353}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2128050, len:2289, log_data_len:282, seq:49257833, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:01.532594] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650412, miss_cnt:32, cache_read_size:8050441557, hit_ratio:"9.999994892294777893e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:01.532610] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657150, miss_cnt:17, cache_read_size:8050188928, hit_ratio:"9.999997286822750464e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:01.538485] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:01.538498] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:01.538718] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:01.552308] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:01.552342] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=25] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:01.552371] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3542, avg_group_log_size=322) [2024-10-16 14:58:01.552380] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3542) [2024-10-16 14:58:01.552674] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=976) [2024-10-16 14:58:01.552773] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F435E-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:01.552783] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F435E-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1900, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:01.561296] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:01.566085] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061881565247000, start_id=1729061881565247000, end_id=1729061881565247001) [2024-10-16 14:58:01.566267] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061881566257], gts:1729061881565247000, latest_srr:[mts=1729061881566257]}) [2024-10-16 14:58:01.568743] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061881568416000, start_id=1729061881568416000, end_id=1729061881568416001) [2024-10-16 14:58:01.568843] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.569489] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.569612] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.569644] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.580417] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.587336] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DDD-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:01.587781] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92B3-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:01.587791] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92B3-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1567, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:01.595763] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.597175] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:01.597192] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:01.597733] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=7] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:58:01.597867] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=21] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1001}, {id:1}]) [2024-10-16 14:58:01.600796] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=203, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:01.600814] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:01.600851] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:01.601019] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D553E-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:01.601026] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D553E-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=476, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:01.611181] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=37] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5745, mem_used=10398720) [2024-10-16 14:58:01.615335] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DB-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061881582874, keepalive_ts:-1}) [2024-10-16 14:58:01.615357] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DB-0-0] [lt=21] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061881582874, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:01.627767] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] no need to update min/max version(new_version={val:1729061881223108000, v:0}, min_version={val:1729061881223108000, v:0}, max_version={val:1729061881223108000, v:0}) [2024-10-16 14:58:01.630558] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061881629705000, start_id=1729061881629705000, end_id=1729061881629705001) [2024-10-16 14:58:01.631258] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061881159563002, v:0}, new_max_version={val:1729061881159563002, v:0}, last_min_version={val:1729061881025146000, v:0}, last_max_version={val:1729061881025146000, v:0}, total_time_us=1599, delta_us=471694) [2024-10-16 14:58:01.631275] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061881159563002, v:0}, min_version={val:1729061881159563002, v:0}, max_version={val:1729061881159563002, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:58:01.642417] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:01.654563] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061881651022000, start_id=1729061881651022000, end_id=1729061881651022001) [2024-10-16 14:58:01.655042] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=247, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:01.655059] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=66, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:01.655699] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:01.655875] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD288-0-0] [lt=22] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:01.655884] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD288-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=808, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:01.656988] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797D9C-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10681) [2024-10-16 14:58:01.660416] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0D20-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061881660290000, start_id=1729061881660290000, end_id=1729061881660290001) [2024-10-16 14:58:01.660417] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D1D-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061881660290000, start_id=1729061881660290000, end_id=1729061881660290001) [2024-10-16 14:58:01.660509] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA93-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061881660290000, start_id=1729061881660290000, end_id=1729061881660290001) [2024-10-16 14:58:01.660584] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D21-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061881660290000, start_id=1729061881660290000, end_id=1729061881660290001) [2024-10-16 14:58:01.660618] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA94-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061881660290000, start_id=1729061881660290000, end_id=1729061881660290001) [2024-10-16 14:58:01.663711] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26147][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB18CC6EE-0-0] [lt=25] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:58:01.673212] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:01.673978] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061881673390000, start_id=1729061881673390000, end_id=1729061881673390001) [2024-10-16 14:58:01.675481] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:01.680439] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.687928] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92B5-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=297, avg_log_submit_to_slide_time=1569) [2024-10-16 14:58:01.688355] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:01.695823] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.700096] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:58:01.724620] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D7-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:01.724650] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D7-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061881660290001, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061881660290001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:01.728577] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D7-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061881728354000, start_id=1729061881728354000, end_id=1729061881728354001) [2024-10-16 14:58:01.728654] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D7-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=10088, avg_group_log_size=630) [2024-10-16 14:58:01.728664] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D7-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10088) [2024-10-16 14:58:01.729064] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061881728354000, start_id=1729061881728354000, end_id=1729061881728354001) [2024-10-16 14:58:01.729149] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:01.729461] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10088) [2024-10-16 14:58:01.729497] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=391, avg_log_submit_to_slide_time=721) [2024-10-16 14:58:01.729553] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:01.731740] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15EE-0-0] [lt=7] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:01.748330] INFO [STORAGE.TRANS] post_msg (ob_trans_rpc.cpp:526) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B401E406B-0-0] [lt=9] post txn free route msg(ret=0, msg={type:82, request_id:20890117, req_sess_id:3221769135, tx_id:{txid:392223473}, ret:-6213, sender:"10.0.4.7:2890", receiver:"10.0.4.8:2890"}) [2024-10-16 14:58:01.752854] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:58:01.752869] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] try gc retain ctx [2024-10-16 14:58:01.752873] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:58:01.753507] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061879345611000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890098662000, latest_log_ts:{val:1729061873558226001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:01.754229] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=55] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061879345611000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:01.754884] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=34] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061879345611000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:01.755510] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=33] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061879345611000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:01.755734] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3655) [2024-10-16 14:58:01.759212] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:01.759240] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=16] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:01.759251] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:01.762798] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2422189, replace_num=31457) [2024-10-16 14:58:01.762815] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:58:01.767958] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1001, min_version={val:1729061881568416000, v:0}, base_version={val:1729061881229371000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061881568416000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061881767944, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061881568416000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061881651065, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061881568416000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061881707514, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:01.768908] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061881767379000, start_id=1729061881767379000, end_id=1729061881767379001) [2024-10-16 14:58:01.772940] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:58:01.779188] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C64-0-0] [lt=26] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:01.779214] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C64-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061881660290002, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061881660290002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:01.780471] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C64-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061881779673000, start_id=1729061881779673000, end_id=1729061881779673001) [2024-10-16 14:58:01.780485] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.782639] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=17] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:01.783965] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C64-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061881783886000, start_id=1729061881783886000, end_id=1729061881783886001) [2024-10-16 14:58:01.788008] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=27] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:58:01.788147] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.788164] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.788416] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.788624] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.788700] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.788701] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.788788] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.788793] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.788809] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.788930] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.789573] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:01.795863] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.801989] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79DA-0-0] [lt=9] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061881793217, keepalive_ts:-1}) [2024-10-16 14:58:01.802009] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79DA-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061881793217, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:01.829479] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:01.829498] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=16] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:58:01.829514] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52626567, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:01.830737] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061881829715000, start_id=1729061881829715000, end_id=1729061881829715001) [2024-10-16 14:58:01.833303] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1}, {id:1003}]) [2024-10-16 14:58:01.837091] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C11A1-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061881836079000, start_id=1729061881836079000, end_id=1729061881836079001) [2024-10-16 14:58:01.838658] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:01.838672] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:01.838966] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:01.846435] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=18] new mysql sessid created(conn.sessid_=3221738602, support_ssl=false) [2024-10-16 14:58:01.846470] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=17] sm conn init succ(conn.sessid_=3221738602, sess.client_addr_="10.0.4.7:53452") [2024-10-16 14:58:01.846486] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=13] accept one succ(*s={this:0x2b0f27404030, session_id:3221738602, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:01.846586] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=24][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27404030, session_id:3221738602, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:01.846610] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=11] kill and revert session(conn.sessid_=3221738602, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:01.846616] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] sock ref clean, do destroy(*s={this:0x2b0f27404030, session_id:3221738602, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:01.846623] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:223) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] mark session id unused(conn.sessid_=3221738602) [2024-10-16 14:58:01.846638] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=15] connection close(sessid=3221738602, proxy_sessid=0, tenant_id=0, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=false, ret=0, trace_id=Y0-0000000000000000-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:0], disconnect_state=0) [2024-10-16 14:58:01.852936] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCB58-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061881852623000, start_id=1729061881852623000, end_id=1729061881852623001) [2024-10-16 14:58:01.853245] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=212, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:01.853262] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=57, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:01.853290] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:01.853632] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4368-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=1311, avg_log_submit_to_slide_time=1614) [2024-10-16 14:58:01.853926] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1002}, tablet_meta.tablet_id_={id:200068}) [2024-10-16 14:58:01.853942] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] success to init medium clog recorder(ret=0, ls_id={id:1002}, tablet_id={id:200068}, max_saved_version=0) [2024-10-16 14:58:01.853950] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] success to init storage recorder(ret=0, ls_id={id:1002}, tablet_id={id:200068}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.853967] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778311878001, v:0}, new_clog_checkpoint_scn_:{val:1729061778311878001, v:0}, for_replay_:true, for_inc_direct_load_:false}, ret=0, ls_id={id:1002}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e456618f0, key:{tablet_id:{id:200068}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778311878001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881853963}, ls_id_:{id:1002}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881853963, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2618e0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f2a0b4420, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e456618f0, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1002}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f2a0b4420, ref_cnt:2, is_inited:true, tablet_id:{id:200068}, freezer:0x2b0e2618e0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e456618f0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26188ed0}, ls_id:{id:1002}, tablet_id:{id:200068}}}) [2024-10-16 14:58:01.854026] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=58] succeeded to create memtable for tablet(ret=0, ls_id={id:1002}, tablet_id={id:200068}, clog_checkpoint_scn={val:1729061778311878001, v:0}, schema_version=0, for_replay=true) [2024-10-16 14:58:01.854035] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] table store update memtable success(ret=0, ls_id={id:1002}, tablet_id={id:200068}, table_store_addr={ptr:0x2b0e6999068c, addr:[7663857-6566-0-2305843511726129152-1](offset=1261568,size=468,type=2,seq=1)}, this=0x2b0e699900a0) [2024-10-16 14:58:01.854069] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1002}, tablet_meta.tablet_id_={id:1152921504606846986}) [2024-10-16 14:58:01.854075] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] success to init medium clog recorder(ret=0, ls_id={id:1002}, tablet_id={id:1152921504606846986}, max_saved_version=0) [2024-10-16 14:58:01.854079] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] success to init storage recorder(ret=0, ls_id={id:1002}, tablet_id={id:1152921504606846986}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.854085] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778311878001, v:0}, new_clog_checkpoint_scn_:{val:1729061778311878001, v:0}, for_replay_:true, for_inc_direct_load_:false}, ret=0, ls_id={id:1002}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e6b40d1a0, key:{tablet_id:{id:1152921504606846986}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778311878001, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881854084}, ls_id_:{id:1002}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881854084, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e2618e0d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f12ddcdd0, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e6b40d1a0, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1002}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f12ddcdd0, ref_cnt:2, is_inited:true, tablet_id:{id:1152921504606846986}, freezer:0x2b0e2618e0d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e6b40d1a0, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e26188ed0}, ls_id:{id:1002}, tablet_id:{id:1152921504606846986}}}) [2024-10-16 14:58:01.854110] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] succeeded to create memtable for tablet(ret=0, ls_id={id:1002}, tablet_id={id:1152921504606846986}, clog_checkpoint_scn={val:1729061778311878001, v:0}, schema_version=0, for_replay=true) [2024-10-16 14:58:01.854122] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] table store update memtable success(ret=0, ls_id={id:1002}, tablet_id={id:1152921504606846986}, table_store_addr={ptr:0x2b0e6b6b86b0, addr:[7667511-10399-0-2305843509577482240-1](offset=98304,size=466,type=2,seq=1)}, this=0x2b0e6b6b80a0) [2024-10-16 14:58:01.856530] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061881854743000, start_id=1729061881854743000, end_id=1729061881854743001) [2024-10-16 14:58:01.859671] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DC8-0-0] [lt=17] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10009, add_timestamp:1729061881859662}) [2024-10-16 14:58:01.859698] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5C-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10009, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881859685, sql_renew_time:1729061881859685, is_mark_fail:false}]) [2024-10-16 14:58:01.859719] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5C-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:10009, add_timestamp:1729061881859662}, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881859685, sql_renew_time:1729061881859685, is_mark_fail:false}]) [2024-10-16 14:58:01.860081] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DC8-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349728475, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410632, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.860259] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DC9-0-0] [lt=18] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11039, add_timestamp:1729061881860253}) [2024-10-16 14:58:01.860292] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5D-0-0] [lt=9] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11039, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:01.861856] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6135, mem_used=12478464) [2024-10-16 14:58:01.861957] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5D-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11039, locations=[{table_id:11039, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881861945, sql_renew_time:0, is_mark_fail:false}, {table_id:11039, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881861945, sql_renew_time:0, is_mark_fail:false}, {table_id:11039, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881861945, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:01.861983] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5D-0-0] [lt=25] success to process renew task(task={tenant_id:1, table_id:11039, add_timestamp:1729061881860253}, locations=[{table_id:11039, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881861945, sql_renew_time:0, is_mark_fail:false}, {table_id:11039, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881861945, sql_renew_time:0, is_mark_fail:false}, {table_id:11039, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881861945, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:01.862915] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F62-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:01.862943] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F62-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:58:01.863127] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=230, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:01.863143] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:01.863177] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:01.864629] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DCA-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349733022, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410633, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.867886] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EB0-0-0] [lt=2][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349736275, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410634, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.868167] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DCB-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12291, add_timestamp:1729061881868160}) [2024-10-16 14:58:01.868198] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5E-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12291, locations=[{table_id:12291, partition_id:12291, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881868187, sql_renew_time:1729061881868187, is_mark_fail:false}]) [2024-10-16 14:58:01.868215] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5E-0-0] [lt=17] success to process renew task(task={tenant_id:1, table_id:12291, add_timestamp:1729061881868160}, locations=[{table_id:12291, partition_id:12291, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061881868187, sql_renew_time:1729061881868187, is_mark_fail:false}]) [2024-10-16 14:58:01.870290] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DCC-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349738685, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410635, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.873487] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:01.874417] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061881873797000, start_id=1729061881873797000, end_id=1729061881873797001) [2024-10-16 14:58:01.876147] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26143][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB26CC6BB-0-0] [lt=22] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:58:01.880546] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.887973] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:01.887991] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:01.888792] INFO [STORAGE] try_shrink (ob_compaction_memory_pool.cpp:522) [22651][T1004_MergeMemP][T1004][Y0-0000000000000000-0-0] [lt=14] Compaction Mem Pool current stat: (ret=0, max_block_num=16, total_block_num=8, Chunk list size: =1) [2024-10-16 14:58:01.890374] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=47] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:01.890391] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3079, avg_group_log_size=307) [2024-10-16 14:58:01.890409] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:01.890582] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=231, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:01.890606] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:01.890627] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:01.890853] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C10F6-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=1184, avg_log_submit_to_slide_time=1469) [2024-10-16 14:58:01.895248] INFO [STORAGE] create_memtable (ob_protected_memtable_mgr_handle.h:158) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] memtable_mgr_handle_ is not exist, need create(tablet_meta.ls_id_={id:1003}, tablet_meta.tablet_id_={id:200078}) [2024-10-16 14:58:01.895266] INFO [STORAGE] init (ob_medium_compaction_mgr.cpp:156) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] success to init medium clog recorder(ret=0, ls_id={id:1003}, tablet_id={id:200078}, max_saved_version=0) [2024-10-16 14:58:01.895273] INFO [STORAGE.TRANS] init_storage_recorder (ob_tablet_memtable_mgr.cpp:144) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] success to init storage recorder(ret=0, ls_id={id:1003}, tablet_id={id:200078}, max_saved_schema_version=0, max_saved_medium_scn=0, compat_mode=0) [2024-10-16 14:58:01.895288] INFO [STORAGE] create_memtable_ (ob_tablet_memtable_mgr.cpp:344) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] succeed to create memtable(arg={schema_version_:0, clog_checkpoint_scn_:{val:1729061778310817000, v:0}, new_clog_checkpoint_scn_:{val:1729061778310817000, v:0}, for_replay_:true, for_inc_direct_load_:false}, ret=0, ls_id={id:1003}, new_tablet_memtable={ObITabletMemtable:{ObITable:{this:0x2b0e6b411a30, key:{tablet_id:{id:200078}, column_group_idx:0, table_type:"MEMTABLE", scn_range:{start_scn:{val:1729061778310817000, v:0}, end_scn:{val:4611686018427387903, v:0}}}, ref_cnt:2, upper_trans_version:9223372036854775807, timestamp:1729061881895285}, ls_id_:{id:1003}, allow_freeze_:true, is_flushed_:false, is_tablet_freeze_:false, logging_blocked_:false, resolved_active_memtable_left_boundary_:true, unset_active_memtable_logging_blocked_:false, has_backoffed_:false, offlined_:false, read_barrier_:false, freeze_clock_:13930, freeze_state_:1, unsubmitted_cnt_:0, init_timestamp_:1729061881895285, max_schema_version_:0, write_ref_cnt_:0, max_end_scn_:{val:0, v:0}, rec_scn_:{val:4611686018427387903, v:0}, freeze_scn_:{val:4611686018427387903, v:0}, migration_clog_checkpoint_scn_:{val:0, v:0}, freezer_:0x2b0e27a120d0, memtable_mgr_handle_:{memtable_mgr:0x2b0f12ddc610, pool:0x2b0e0c4f2030}, mt_stat_.frozen_time_:0, mt_stat_.ready_for_flush_time_:0, mt_stat_.create_flush_dag_time_:0, mt_stat_.release_time_:0, mt_stat_.push_table_into_gc_queue_time_:0, mt_stat_.last_print_time_:0}, this:0x2b0e6b411a30, state:0, max_data_schema_version:0, max_column_cnt:0, local_allocator:{ListHandle:{freeze_stat:0, id:-1, clock:9223372036854775807}, host:0x2b0e0c2a62b0, arena_handle:{allocated:0}}, contain_hotspot_row:false, snapshot_version:{val:4611686018427387903, v:0}, contain_hotspot_row:false, ls_id:{id:1003}, transfer_freeze_flag:false, recommend_snapshot_version:{val:18446744073709551615, v:3}}, this={ObIMemtableMgr:{Memtables:this:0x2b0f12ddc610, ref_cnt:2, is_inited:true, tablet_id:{id:200078}, freezer:0x2b0e27a120d0, memtable_head:0, memtable_tail:1, t3m:0x2b0e0bf1e030, tables:[0x2b0e6b411a30, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]}, schema_recorder:{ObIStorageClogRecorder:{max_saved_version_:0, clog_scn_:{val:18446744073709551615, v:3}, log_handler_:0x2b0e27a0ced0}, ls_id:{id:1003}, tablet_id:{id:200078}}}) [2024-10-16 14:58:01.895344] INFO [STORAGE] inner_create_memtable (ob_tablet.cpp:4633) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=55] succeeded to create memtable for tablet(ret=0, ls_id={id:1003}, tablet_id={id:200078}, clog_checkpoint_scn={val:1729061778310817000, v:0}, schema_version=0, for_replay=true) [2024-10-16 14:58:01.895353] INFO [STORAGE] create_memtable (ob_tablet.cpp:4580) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] table store update memtable success(ret=0, ls_id={id:1003}, tablet_id={id:200078}, table_store_addr={ptr:0x2b0df1d4868c, addr:[7667511-10399-0-2305843506356158488-1](offset=24,size=463,type=2,seq=1)}, this=0x2b0df1d480a0) [2024-10-16 14:58:01.895954] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.898092] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DD0-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349766486, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410636, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.901707] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21674][T1_L0_G0][T1][YB4A0A000407-00061E9979BD0322-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349770096, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410637, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.903118] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DD3-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349771512, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410638, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.905193] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DD5-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349773588, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410639, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.908334] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=25] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}) [2024-10-16 14:58:01.908743] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=2046, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:01.914105] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=44] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:01.914767] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=27] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:01.914826] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=20] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061877257554000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:58:01.916872] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DD9-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349785266, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410640, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.919194] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=11] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:01.921159] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DDB-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349789548, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410641, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.921277] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:01.921299] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=14] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=217) [2024-10-16 14:58:01.922894] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DDD-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349791289, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410642, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.923119] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DDD-0-0] [lt=14] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:58:01.925235] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DDF-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510349793631, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410643, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:01.925316] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:01.926174] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=15] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:01.929324] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061881929268000, start_id=1729061881929268000, end_id=1729061881929268001) [2024-10-16 14:58:01.937392] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C11B3-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061881936912000, start_id=1729061881936912000, end_id=1729061881936912001) [2024-10-16 14:58:01.938715] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:01.938733] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:01.939159] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD46E-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:01.939177] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD46E-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=517, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:01.945871] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=3983, mem_used=8318976) [2024-10-16 14:58:01.962885] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:58:01.969027] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061881968772000, start_id=1729061881968772000, end_id=1729061881968772001) [2024-10-16 14:58:01.980635] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.985815] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=3] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061877881374, last_first_register_time_us=1722494157071428, register_time_us=1729061877881374, curr_time_us=1729061881985813) [2024-10-16 14:58:01.985850] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061881985833, last_submit_register_req_time_us=1729061881985833, register_parent_reason=3) [2024-10-16 14:58:01.985858] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:58:01.996010] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:01.997405] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D3C-0-0] [lt=21] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:01.997425] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D3C-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:02.025726] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D81-0-0] [lt=10] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:02.025762] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D81-0-0] [lt=25] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061881660290003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061881660290003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:02.027614] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D81-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061882026895000, start_id=1729061882026895000, end_id=1729061882026895001) [2024-10-16 14:58:02.030138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22493][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D81-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061882029838000, start_id=1729061882029838000, end_id=1729061882029838001) [2024-10-16 14:58:02.030212] INFO [LIB] stat (utility.h:1165) [22493][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D81-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:02.030229] INFO [LIB] stat (utility.h:1165) [22493][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D81-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:02.030807] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061882029838000, start_id=1729061882029838000, end_id=1729061882029838001) [2024-10-16 14:58:02.035911] INFO [CLOG] do_thread_task_ (ob_remote_fetch_log_worker.cpp:249) [21544][T1_RFLWorker][T1][YB4A0A000407-00061E996BC960C1-0-0] [lt=17] ObRemoteFetchWorker is running(thread_index=0) [2024-10-16 14:58:02.038875] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061882038873, bytes: 4446822634227, bw: 0.526952 MB/s, add_ts: 1002030, add_bytes: 553671 [2024-10-16 14:58:02.039042] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=245, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:02.039041] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:02.039059] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=54, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:02.039076] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:02.039255] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD470-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:02.039260] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD470-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=623, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:02.039287] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD471-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=283, avg_log_submit_to_slide_time=508) [2024-10-16 14:58:02.044093] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=9] worker thread began to shrink(tenant_->id()=1002, group_id_=1, token=0) [2024-10-16 14:58:02.044563] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=35] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:02.048621] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107548][T1002_L0_G1][T1002][Y0-0000000000000000-0-0] [lt=10] worker thread exit(tenant_->id()=1002, workers_.get_size()=2) [2024-10-16 14:58:02.048638] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107548][T1002_L0_G1][T0][Y0-0000000000000000-0-0] [lt=14] tenant thread end_run(id_=1002, ret=0, thread_count_=190) [2024-10-16 14:58:02.056640] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061882056421000, start_id=1729061882056421000, end_id=1729061882056421001) [2024-10-16 14:58:02.063066] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F66-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=9, avg_log_batch_cnt=0, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:58:02.063091] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F66-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:02.063098] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F66-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=37660, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:02.063102] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F66-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173353, log_task={header:{begin_lsn:{lsn:14352399195}, end_lsn:{lsn:14352399317}, log_id:115173353, min_scn:{val:1729061881708855001, v:0}, max_scn:{val:1729061881708855001, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352399073}, prev_proposal_id:17, committed_end_lsn:{lsn:14352399195}, data_checksum:940009244, accum_checksum:629935528, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061881963047, freeze_ts:1729061881963047, submit_ts:1729061881963053, flushed_ts:1729061881963322, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:269}, fs_cb_begin_ts=1729061882063084, log_life_time=100037) [2024-10-16 14:58:02.063260] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:58:02.066467] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061882065936000, start_id=1729061882065936000, end_id=1729061882065936001) [2024-10-16 14:58:02.066491] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061882066460], gts:1729061882065936000, latest_srr:[mts=1729061882066460]}) [2024-10-16 14:58:02.066505] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:02.073789] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:02.074502] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061882073311000, start_id=1729061882073311000, end_id=1729061882073311001) [2024-10-16 14:58:02.075722] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE78FD-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=2316) [2024-10-16 14:58:02.075937] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:02.080642] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:02.080663] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:02.080669] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.087794] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:02.090619] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3080) [2024-10-16 14:58:02.090644] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3544) [2024-10-16 14:58:02.090835] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45CC1-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:02.090852] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45CC1-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1492, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:02.096068] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.100887] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85135-0-0] [lt=13][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497622, log_task={header:{begin_lsn:{lsn:207589573227}, end_lsn:{lsn:207589573701}, log_id:218497622, min_scn:{val:1729061881927894000, v:0}, max_scn:{val:1729061881927894000, v:0}, data_len:418, proposal_id:81, prev_lsn:{lsn:207589572753}, prev_proposal_id:81, committed_end_lsn:{lsn:207589573227}, data_checksum:2868364104, accum_checksum:1886223715, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061881929235, freeze_ts:1729061881929235, submit_ts:1729061881929240, flushed_ts:1729061881929936, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:696}, fs_cb_begin_ts=1729061882100881, log_life_time=171646) [2024-10-16 14:58:02.101475] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=46] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:02.102098] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=13] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:02.102142] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061880208141000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:58:02.128090] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:02.128108] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:02.129348] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061882129291000, start_id=1729061882129291000, end_id=1729061882129291001) [2024-10-16 14:58:02.137931] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4433, mem_used=8318976) [2024-10-16 14:58:02.139045] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=205, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:02.139094] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=43, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:02.149865] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=15] gc stale ls task succ [2024-10-16 14:58:02.159958] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=12] current monitor number(seq_=-1) [2024-10-16 14:58:02.160569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BA99-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061882159682000, start_id=1729061882159682000, end_id=1729061882159682001) [2024-10-16 14:58:02.160568] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA97-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061882159682000, start_id=1729061882159682000, end_id=1729061882159682001) [2024-10-16 14:58:02.160741] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BA9B-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061882159682000, start_id=1729061882159682000, end_id=1729061882159682001) [2024-10-16 14:58:02.160906] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BA98-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061882160740000, start_id=1729061882160740000, end_id=1729061882160740001) [2024-10-16 14:58:02.160936] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BA9A-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061882160740000, start_id=1729061882160740000, end_id=1729061882160740001) [2024-10-16 14:58:02.162960] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:58:02.163522] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92BD-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=254, avg_log_submit_to_slide_time=33639) [2024-10-16 14:58:02.166785] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15E6-0-0] [lt=14] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:02.166997] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.94, rt: 3517.75; ) [2024-10-16 14:58:02.167020] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:02.167059] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.94, rt: 140.00; ) [2024-10-16 14:58:02.167078] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:02.167092] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:02.168129] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15EA-0-0] [lt=14] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:02.168152] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:02.168209] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=20] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:02.169217] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061882167148000, start_id=1729061882167148000, end_id=1729061882167148001) [2024-10-16 14:58:02.169934] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15EC-0-0] [lt=11] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:02.180706] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.187629] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DE9-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:02.196658] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.198373] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:58:02.198495] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D89-0-0] [lt=12] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061897008855000, latest_log_ts:{val:1729061880424064001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:02.198555] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D89-0-0] [lt=56] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:02.202737] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5040-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:02.202753] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5040-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:02.214761] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=6] Garbage Collector is running(seq_=656701, gc_interval=10000000) [2024-10-16 14:58:02.214789] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=10] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:02.214804] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=10] gc_check_member_list_ cost time(ret=0, time_us=32) [2024-10-16 14:58:02.214813] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:58:02.215524] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=6] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:02.215541] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=13] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:02.215557] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=15] execute_gc cost time(ret=0, time_us=17) [2024-10-16 14:58:02.215562] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=4] ObSafeDestroyHandler start process [2024-10-16 14:58:02.219686] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061882219684, bytes: 0, bw: 0.000000 MB/s, add_ts: 1009621, add_bytes: 0 [2024-10-16 14:58:02.230853] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061882229223000, start_id=1729061882229223000, end_id=1729061882229223001) [2024-10-16 14:58:02.230924] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:02.230939] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:02.238887] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:02.238907] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:02.239236] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1914) [2024-10-16 14:58:02.239476] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE400D5-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=318, avg_log_submit_to_slide_time=561) [2024-10-16 14:58:02.253248] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=28] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:02.253268] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:02.253796] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4553-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:02.253806] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4553-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=1657, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:02.256686] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22795][T1004_TenantWea][T1003][YB4A0A000407-00061E9B3AFB9772-0-0] [lt=16] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.248012828826904297e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=312, Throttle Trigger(MB)=187, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.830766965110191677e+03) [2024-10-16 14:58:02.256843] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061882256370000, start_id=1729061882256370000, end_id=1729061882256370001) [2024-10-16 14:58:02.274087] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:02.274688] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061882274397000, start_id=1729061882274397000, end_id=1729061882274397001) [2024-10-16 14:58:02.280749] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.287730] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DEB-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178662, log_task={header:{begin_lsn:{lsn:14538513204}, end_lsn:{lsn:14538513326}, log_id:115178662, min_scn:{val:1729061882160740000, v:0}, max_scn:{val:1729061882160740000, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538513082}, prev_proposal_id:23, committed_end_lsn:{lsn:14538513204}, data_checksum:1715074492, accum_checksum:144317012, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061882187619, freeze_ts:1729061882187619, submit_ts:1729061882187624, flushed_ts:1729061882188120, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:496}, fs_cb_begin_ts=1729061882287722, log_life_time=100103) [2024-10-16 14:58:02.289317] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21664][T1_L0_G0][T1][YB4A0A000407-00061E99794D04FB-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510350157710, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410644, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:02.291235] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3545) [2024-10-16 14:58:02.296690] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.297229] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107484][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D3F-0-0] [lt=31] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:02.297258] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107484][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D3F-0-0] [lt=25] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:02.301066] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85139-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:02.301088] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85139-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=2398) [2024-10-16 14:58:02.301312] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2398) [2024-10-16 14:58:02.313872] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:02.313895] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=15] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:02.313916] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373718632, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:02.319875] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA894-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:02.319903] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA894-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061882159682001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061882159682001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:02.320910] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA894-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061882320038000, start_id=1729061882320038000, end_id=1729061882320038001) [2024-10-16 14:58:02.323405] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA894-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061882323219000, start_id=1729061882323219000, end_id=1729061882323219001) [2024-10-16 14:58:02.323486] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA894-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9874, avg_group_log_size=658) [2024-10-16 14:58:02.323504] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA894-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9874) [2024-10-16 14:58:02.323776] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=264, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:02.323790] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9874) [2024-10-16 14:58:02.323801] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=65, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:02.323825] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9874) [2024-10-16 14:58:02.323845] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:02.324080] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9637-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:02.324086] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9637-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=842, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:02.324090] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9637-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=357, avg_log_submit_to_slide_time=809) [2024-10-16 14:58:02.324155] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:02.329530] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061882328528000, start_id=1729061882328528000, end_id=1729061882328528001) [2024-10-16 14:58:02.329796] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=825, mem_used=2079744) [2024-10-16 14:58:02.335923] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:02.341835] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:58:02.341852] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=12582912, ctx.total_memstore_used_=46137344, ctx.total_memstore_hold_=46137344, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:58:02.341870] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:02.353039] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EA-0-0] [lt=32] [LS_LOCATION]dump tenant ls location caches(tenant_id=1, tenant_ls_locations=[{cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061880000357, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:02.353065] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=9] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:02.353073] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EA-0-0] [lt=33] [LS_LOCATION]dump tenant ls location caches(tenant_id=1001, tenant_ls_locations=[{cache_key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061880001143, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:02.353083] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:02.353091] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:58:02.353095] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:58:02.353098] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:58:02.353102] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17119) [2024-10-16 14:58:02.353088] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EA-0-0] [lt=10] [LS_LOCATION]dump tenant ls location caches(tenant_id=1002, tenant_ls_locations=[{cache_key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061880001877, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061880001876, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061880001878, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}]}, {cache_key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061880001874, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:02.353126] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EA-0-0] [lt=38] [LS_LOCATION]dump tenant ls location caches(tenant_id=1003, tenant_ls_locations=[{cache_key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061880002496, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:02.353139] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EA-0-0] [lt=12] [LS_LOCATION]dump tenant ls location caches(tenant_id=1004, tenant_ls_locations=[{cache_key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061880003191, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061880003194, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061880003195, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}]}, {cache_key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061880003193, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:02.353335] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1089, avg_group_log_size=121) [2024-10-16 14:58:02.353348] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1089) [2024-10-16 14:58:02.353587] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1089) [2024-10-16 14:58:02.353732] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=232, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:02.353783] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=44] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=157, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:02.353793] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=4954) [2024-10-16 14:58:02.354156] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4556-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=20, avg_log_submit_to_flush_time=1231, avg_log_submit_to_slide_time=1462) [2024-10-16 14:58:02.363042] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:58:02.369434] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061882367682000, start_id=1729061882367682000, end_id=1729061882367682001) [2024-10-16 14:58:02.370456] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=41, stat_interval=5000000, avg (count/cost)=45, this=0x2b0e07718280, extra_info="") [2024-10-16 14:58:02.370472] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=16] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=41, stat_interval=5000000, avg (count/cost)=823, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:58:02.370477] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=41, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:58:02.370480] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=41, stat_interval=5000000, avg (count/cost)=-738, this=0x2b0e07718910, extra_info="") [2024-10-16 14:58:02.370483] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=41, stat_interval=5000000, avg (count/cost)=130, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:58:02.375954] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7903-0-0] [lt=30] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=2316, avg_group_log_size=231) [2024-10-16 14:58:02.376179] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2316) [2024-10-16 14:58:02.377385] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:02.377400] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54641860, tenant_id=1003) [2024-10-16 14:58:02.377415] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=10] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:02.380809] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.387826] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DED-0-0] [lt=26] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:02.388037] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=212, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:02.388059] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=38, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:02.388103] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:02.392377] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C11BA-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061882391860000, start_id=1729061882391860000, end_id=1729061882391860001) [2024-10-16 14:58:02.395588] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:58:02.396755] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.402987] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:02.417186] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183786382, max_lsn:114618729698, max_scn:{val:1729061882367682000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45685691230}, reuse_lsn:{lsn:114618729698}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114576786658}}, last_submit_lsn:{lsn:114618729106}, last_submit_end_lsn:{lsn:114618729698}, last_submit_log_id:183786382, last_submit_log_pid:33705, max_flushed_lsn:{lsn:114618729106}, max_flushed_end_lsn:{lsn:114618729698}, max_flushed_log_pid:33705, committed_end_lsn:{lsn:114618729698}, last_slide_log_id:183786382, last_slide_scn:{val:1729061882367682000, v:0}, last_slide_lsn:{lsn:114618729106}, last_slide_end_lsn:{lsn:114618729698}, last_slide_log_pid:33705, last_slide_log_accum_checksum:1752729704, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0df99e2530}, [StateMgr]={this:0x2b0df99e6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, leader:"10.0.4.7:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183786383, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261592457, palf_role_change_cb:0x2b0df99ed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, reconfig_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, persistent_config_version:{proposal_id:33705, config_seq:14}, ms_ack_list:0, resend_config_version:{proposal_id:33705, config_seq:14}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258289572, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258289572, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0df99e3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261792660, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729029056668344001, v:0}, min_block_id:1702, min_block_min_scn:{val:1729022447183332001, v:0}, base_lsn_for_block_gc:{lsn:114547838976}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, log_config_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, log_snapshot_meta:{version:1, base_lsn:{lsn:114547838976}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:233218048}, readable_log_tail:{lsn:233218048}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114682048512}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/meta", dir_fd:146, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:35201024, block_header_serialize_buf_:0x2b0df99e89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114618729698}, readable_log_tail:{lsn:114618729698}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114614943744}, min_scn:{val:1729061506448191001, v:0}, curr_block_id:1708, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/log", dir_fd:148, min_block_id:1702, max_block_id:1709, curr_writable_block_id:1708}, logical_block_size_:67104768, curr_block_writable_size_:63318814, block_header_serialize_buf_:0x2b0df99ea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261592575, this:0x2b0df99e6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33705, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33704, majority_max_lsn:{lsn:45686897415}, saved_end_lsn:{lsn:45686897415}, last_submit_prepare_req_time_us:1722494261592686, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0df99e69b0}) [2024-10-16 14:58:02.417331] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=141] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061882369838, last_flushed_end_lsn:{lsn:114618729698}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061882370398, last_flushed_end_lsn:{lsn:114618729698}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061882370269, last_flushed_end_lsn:{lsn:114618729698}}]) [2024-10-16 14:58:02.417342] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183786383, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061882369838, last_flushed_end_lsn:{lsn:114618729698}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061882370398, last_flushed_end_lsn:{lsn:114618729698}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061882370269, last_flushed_end_lsn:{lsn:114618729698}}]) [2024-10-16 14:58:02.431112] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061882429537000, start_id=1729061882429537000, end_id=1729061882429537001) [2024-10-16 14:58:02.439204] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:02.445920] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:02.456953] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] submit log success(service_type_=0, last_id=1729061899029444000, limited_id=1729061899029444000) [2024-10-16 14:58:02.456964] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] submit log with lock success(service_type_=0, last_id=1729061899029444000, limited_id=1729061899029444000) [2024-10-16 14:58:02.456969] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061882455772000, start_id=1729061882455772000, end_id=1729061882455772001) [2024-10-16 14:58:02.458797] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=14] update id meta(ret=0, service_type=0, limited_id=1729061899029444000, latest_scn={val:1729061882455772001, v:0}, *this={tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061899029444000, latest_log_ts:{val:1729061882455772001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:58:02.458840] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=42] handle submit callback(service_type_=0, success=true, used_ts=1888, log_ts={val:1729061882455772001, v:0}, limited_id=1729061899029444000) [2024-10-16 14:58:02.458847] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:58:02.458851] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=5] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:58:02.458864] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061882458189000, start_id=1729061882458189000, end_id=1729061882458189001) [2024-10-16 14:58:02.460962] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=9] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:02.475377] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:02.475759] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061882474953000, start_id=1729061882474953000, end_id=1729061882474953001) [2024-10-16 14:58:02.476228] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7905-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:02.476975] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=181, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:02.476989] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=81, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:02.480212] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601486, offset=9648908}, end_cursor:ObLogCursor{file_id=192, log_id=47601487, offset=9649192}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd204c050, len:1268, log_data_len:284, seq:47601486, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:02.480867] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.482063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B8-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061882479680000, start_id=1729061882479680000, end_id=1729061882479680001) [2024-10-16 14:58:02.482933] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601487, offset=9649192}, end_cursor:ObLogCursor{file_id=192, log_id=47601488, offset=9649466}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db733a050, len:984, log_data_len:274, seq:47601487, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:02.484824] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601488, offset=9649466}, end_cursor:ObLogCursor{file_id=192, log_id=47601489, offset=9649740}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73ea050, len:710, log_data_len:274, seq:47601488, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:02.485096] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=25] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601489, offset=9649740}, end_cursor:ObLogCursor{file_id=192, log_id=47601490, offset=9650014}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2098050, len:436, log_data_len:274, seq:47601489, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:02.485193] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561488, miss_cnt:1, cache_read_size:152590707874, hit_ratio:"9.999999928346995359e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:02.485217] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599073, miss_cnt:9, cache_read_size:149777938630, hit_ratio:"9.999999345925868477e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:02.489863] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A94-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:02.489883] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A94-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:02.496818] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.501732] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=60, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:02.501751] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=43, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:02.525239] INFO [ARCHIVE] print_self (ob_ls_task.cpp:414) [22806][T1004_ArcSrv][T1004][YB4A0A000407-00061E9972A960C1-0-0] [lt=29] print ls archive task(id={id:1}, tenant_id=1004, station={key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, round_start_scn={val:1728967049237798000, v:0}, dest={has_encount_error:false, is_worm:false, max_no_limit_lsn:{lsn:17343057821}, max_archived_info:{offset:{lsn:17493458833}, scn:{val:1729061854598998002, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, max_seq_log_offset:{lsn:17514344448}, max_fetch_info:{offset:{lsn:17493458833}, scn:{val:1729061854598998002, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, piece_min_lsn:{lsn:17479894340}, archive_file_id:261, archive_file_offset:13564493, piece_dir_exist:true, wait_send_task_count:0, tasks:}) [2024-10-16 14:58:02.525478] INFO [ARCHIVE] print_self (ob_ls_task.cpp:414) [22806][T1004_ArcSrv][T1004][YB4A0A000407-00061E9972A960C1-0-0] [lt=49] print ls archive task(id={id:1001}, tenant_id=1004, station={key:{incarnation:1, dest_id:1004, round:3}, lease:{lease_id:17, lease_start_ts:0, lease_end_ts:0}}, round_start_scn={val:1728967049237798000, v:0}, dest={has_encount_error:false, is_worm:false, max_no_limit_lsn:{lsn:13531789325}, max_archived_info:{offset:{lsn:13645410827}, scn:{val:1729061854598998000, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, max_seq_log_offset:{lsn:13689372672}, max_fetch_info:{offset:{lsn:13645410827}, scn:{val:1729061854598998000, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1728967049237798000, v:0}, piece_id:74, base_piece_id:73}}, piece_min_lsn:{lsn:13635346983}, archive_file_id:204, archive_file_offset:10063844, piece_dir_exist:true, wait_send_task_count:0, tasks:}) [2024-10-16 14:58:02.527548] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=19] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a5feb0, tx_id:{txid:199664504}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061882526433, active_ts:1729061882526433, commit_ts:-1, finish_ts:1729061882527470, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061912526428, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061882522011, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:58:02.528400] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=94] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a540f0, tx_id:{txid:199664505}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061882527470, active_ts:1729061882527470, commit_ts:-1, finish_ts:1729061882527470, timeout_us:29999996, lock_timeout_us:-1, expire_ts:1729061912527466, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061882522011, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:58:02.530501] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061882529593000, start_id=1729061882529593000, end_id=1729061882529593001) [2024-10-16 14:58:02.531952] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DC0-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061882531842000, start_id=1729061882531842000, end_id=1729061882531842001) [2024-10-16 14:58:02.533431] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257834, offset=39053353}, end_cursor:ObLogCursor{file_id=197, log_id=49257835, offset=39053636}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72b2050, len:2007, log_data_len:283, seq:49257834, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:02.534302] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:58:02.534316] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:58:02.534329] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444646146, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:02.535492] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB383-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061882535016000, start_id=1729061882535016000, end_id=1729061882535016001) [2024-10-16 14:58:02.536428] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257835, offset=39053636}, end_cursor:ObLogCursor{file_id=197, log_id=49257836, offset=39053918}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db730e050, len:1724, log_data_len:282, seq:49257835, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:02.538380] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257836, offset=39053918}, end_cursor:ObLogCursor{file_id=197, log_id=49257837, offset=39054200}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2040050, len:1442, log_data_len:282, seq:49257836, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:02.538514] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257837, offset=39054200}, end_cursor:ObLogCursor{file_id=197, log_id=49257838, offset=39054482}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd202c050, len:1160, log_data_len:282, seq:49257837, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:02.538554] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650421, miss_cnt:32, cache_read_size:8050442777, hit_ratio:"9.999994892295511750e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:02.538567] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657159, miss_cnt:17, cache_read_size:8050190026, hit_ratio:"9.999997286823140152e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:02.539231] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:02.540012] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [21586][T1_ObXAHbWorker][T1][Y0-0000000000000000-0-0] [lt=15] XA scheduler heartbeat task statistics(avg_time=0) [2024-10-16 14:58:02.553433] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:02.553452] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:02.553474] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=5928, avg_group_log_size=592) [2024-10-16 14:58:02.553482] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=5928) [2024-10-16 14:58:02.553742] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:58:02.554034] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F437B-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:02.554044] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F437B-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=691, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:02.563125] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:58:02.566857] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061882566803000, start_id=1729061882566803000, end_id=1729061882566803001) [2024-10-16 14:58:02.566876] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] ts response task statistics(alloc_count=261365274, free_count=130682636) [2024-10-16 14:58:02.569445] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.569613] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061882568899000, start_id=1729061882568899000, end_id=1729061882568899001) [2024-10-16 14:58:02.569621] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.569636] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.569708] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.576089] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7907-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:02.576104] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7907-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10398, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:02.576131] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=2316, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:02.580927] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.584991] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:14538513748}, scn={val:1729061882160740004, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:58:02.585021] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061882584959, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538513804}, next_to_submit_scn_:{val:1729061882160740005, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538513692}, end_lsn:{lsn:14538513804}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061882160740004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061882160740004, v:0}, data_checksum:1631573933, flag:0}}, init_mode_version:0, accumulate_checksum:4195739413, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061882160740004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650422, miss_cnt:32, cache_read_size:8050442889, hit_ratio:"9.999994892295592797e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}, replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14538513748}, scn_:{val:1729061882160740004, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061882575680, first_handle_ts_:1729061882585020, replay_cost_:35, retry_cost_:4294967299, read_log_buf_:0x2b0f77c27570, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:58:02.585151] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060071843583000) [2024-10-16 14:58:02.585162] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14538513748}, scn_:{val:1729061882160740004, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061882575680, first_handle_ts_:1729061882585141, replay_cost_:19, retry_cost_:4294967299, read_log_buf_:0x2b0f77c27580, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061882584959, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538513804}, next_to_submit_scn_:{val:1729061882160740005, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538513692}, end_lsn:{lsn:14538513804}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061882160740004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061882160740004, v:0}, data_checksum:1631573933, flag:0}}, init_mode_version:0, accumulate_checksum:4195739413, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061882160740004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650422, miss_cnt:32, cache_read_size:8050442889, hit_ratio:"9.999994892295592797e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}, replay_queue_idx=0, ret=0) [2024-10-16 14:58:02.587811] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DF1-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:02.588254] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92C7-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:02.588266] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92C7-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9543, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:02.593570] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCB6E-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061882593059000, start_id=1729061882593059000, end_id=1729061882593059001) [2024-10-16 14:58:02.596368] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD01A-0-0] [lt=5] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061882587593, keepalive_ts:-1}) [2024-10-16 14:58:02.596384] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD01A-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061882587593, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:02.596884] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.597336] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:02.597360] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:02.597571] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD01B-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061882587651, keepalive_ts:-1}) [2024-10-16 14:58:02.597584] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD01B-0-0] [lt=12] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061882587651, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:02.597967] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1003}, {id:1001}]) [2024-10-16 14:58:02.600282] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=18) [2024-10-16 14:58:02.601192] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8513F-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=12, avg_log_batch_cnt=0, total_group_log_size=2520, avg_group_log_size=210) [2024-10-16 14:58:02.601405] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=233, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:02.601421] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=59, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:02.601449] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:02.601701] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5554-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:02.601710] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5554-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16176, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:02.611291] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5771, mem_used=10398720) [2024-10-16 14:58:02.618898] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:02.632705] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061882630957000, start_id=1729061882630957000, end_id=1729061882630957001) [2024-10-16 14:58:02.633456] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061882159682002, v:0}, new_max_version={val:1729061882159682002, v:0}, last_min_version={val:1729061882065936000, v:0}, last_max_version={val:1729061882065936000, v:0}, total_time_us=1667, delta_us=473772) [2024-10-16 14:58:02.633482] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=25] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061882159682002, v:0}, min_version={val:1729061882159682002, v:0}, max_version={val:1729061882159682002, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:58:02.639140] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:58:02.639164] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1342) [2024-10-16 14:58:02.642766] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1003}, tablet_id:{id:49402}}) [2024-10-16 14:58:02.643214] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1002}, tablet_id:{id:49402}}) [2024-10-16 14:58:02.644369] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1001}, tablet_id:{id:49402}}) [2024-10-16 14:58:02.645375] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1002}, tablet_id:{id:49403}}) [2024-10-16 14:58:02.645649] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1}, tablet_id:{id:49403}}) [2024-10-16 14:58:02.645974] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1001}, tablet_id:{id:49403}}) [2024-10-16 14:58:02.646800] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1003}, tablet_id:{id:49403}}) [2024-10-16 14:58:02.647125] WDIAG [STORAGE] get_meta_obj_with_external_memory (ob_tablet_pointer_map.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4050] fail to try get in memory meta obj(ret=-4050, key={ls_id:{id:1001}, tablet_id:{id:49401}}) [2024-10-16 14:58:02.647199] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=5] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=108) [2024-10-16 14:58:02.647220] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=6] successfully defragment data blocks(rewrite_cnt=0) [2024-10-16 14:58:02.647229] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [22145][T1002_SSTableDe][T1002][Y0-0000000000000000-0-0] [lt=3] tablet skip filter destructed(total_cnt=2150, skip_cnt=8, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105370475, total_skip_cnt=310305, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:58:02.655418] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=16] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:58:02.657430] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061882656099000, start_id=1729061882656099000, end_id=1729061882656099001) [2024-10-16 14:58:02.658687] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797DAC-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9386) [2024-10-16 14:58:02.658913] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=285, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:02.658928] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=155, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:02.659057] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:02.659064] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=856, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:02.659068] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:02.661301] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0D2A-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061882660978000, start_id=1729061882660978000, end_id=1729061882660978001) [2024-10-16 14:58:02.661375] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D28-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061882660978000, start_id=1729061882660978000, end_id=1729061882660978001) [2024-10-16 14:58:02.661471] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAA0-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061882660978000, start_id=1729061882660978000, end_id=1729061882660978001) [2024-10-16 14:58:02.661612] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BA9C-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061882660978000, start_id=1729061882660978000, end_id=1729061882660978001) [2024-10-16 14:58:02.672985] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:58:02.673002] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=15] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:58:02.673009] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:58:02.673013] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:02.675120] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=115, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:787547, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787548, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787549, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:787550, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:58:02.675274] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=151] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:02.675673] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:02.675726] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:02.676309] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061882675752000, start_id=1729061882675752000, end_id=1729061882675752001) [2024-10-16 14:58:02.680987] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.681139] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=9] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:58:02.681908] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061882681046000, start_id=1729061882681046000, end_id=1729061882681046001) [2024-10-16 14:58:02.688449] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92C9-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=274, avg_log_submit_to_slide_time=9535) [2024-10-16 14:58:02.696944] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.700159] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:02.711880] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DE1-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510350580274, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410645, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:02.712649] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DE5-0-0] [lt=16][errcode=-4201] hashtable not init [2024-10-16 14:58:02.712661] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DE5-0-0] [lt=11][errcode=-4016] get cursor info failed(cursor_id=1119, get_sessid()=3221565766) [2024-10-16 14:58:02.712738] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DE6-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061882712609000, start_id=1729061882712609000, end_id=1729061882712609001) [2024-10-16 14:58:02.728418] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=21] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:02.729239] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=9] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:02.730847] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061882730556000, start_id=1729061882730556000, end_id=1729061882730556001) [2024-10-16 14:58:02.731543] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D8-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:02.731571] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D8-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061882660978000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061882660978000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:02.736673] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D8-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061882736456000, start_id=1729061882736456000, end_id=1729061882736456001) [2024-10-16 14:58:02.736736] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D8-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10526, avg_group_log_size=619) [2024-10-16 14:58:02.736746] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D8-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10526) [2024-10-16 14:58:02.736996] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10526) [2024-10-16 14:58:02.737420] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD29B-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=17, avg_log_gen_to_submit_time=46, avg_log_submit_to_flush_time=502, avg_log_submit_to_slide_time=778) [2024-10-16 14:58:02.737472] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:02.753830] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=5937) [2024-10-16 14:58:02.759282] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:02.759308] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:02.759318] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:02.763208] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:58:02.769695] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061882768023000, start_id=1729061882768023000, end_id=1729061882768023001) [2024-10-16 14:58:02.770498] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15F0-0-0] [lt=11] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:02.772970] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:58:02.780268] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:02.780288] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=14] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:02.780296] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=55) [2024-10-16 14:58:02.781043] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.782724] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:02.786552] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C65-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:02.786583] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C65-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061882660978001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061882660978001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:02.787657] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C65-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061882786814000, start_id=1729061882786814000, end_id=1729061882786814001) [2024-10-16 14:58:02.787879] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.787894] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.788723] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.788797] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.788810] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.788827] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.788858] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.788874] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.788887] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.788913] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.790102] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:02.790292] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C65-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061882789979000, start_id=1729061882789979000, end_id=1729061882789979001) [2024-10-16 14:58:02.796976] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.832846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061882831080000, start_id=1729061882831080000, end_id=1729061882831080001) [2024-10-16 14:58:02.833420] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1003}, {id:1002}]) [2024-10-16 14:58:02.833762] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:207589574677}, scn={val:1729061882660978002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061882801713, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589574677}, next_to_submit_scn_:{val:1729061882660978002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589574555}, end_lsn:{lsn:207589574677}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061882660978001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061882660978001, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:703819384, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061882660978001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599077, miss_cnt:9, cache_read_size:149777939118, hit_ratio:"9.999999345925888461e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:58:02.839210] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1914, avg_group_log_size=191) [2024-10-16 14:58:02.839231] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1914) [2024-10-16 14:58:02.839571] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1914) [2024-10-16 14:58:02.847066] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=34] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:02.853934] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=234, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:02.853957] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=43, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:02.853972] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:02.854161] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4565-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=37, avg_log_submit_to_flush_time=311, avg_log_submit_to_slide_time=563) [2024-10-16 14:58:02.857566] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061882856671000, start_id=1729061882856671000, end_id=1729061882856671001) [2024-10-16 14:58:02.861906] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6188, mem_used=12478464) [2024-10-16 14:58:02.863514] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F76-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:02.863742] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1053, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:02.863746] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=42] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:02.863757] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=37, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:02.863800] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:02.864013] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92CC-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:02.864407] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:02.864456] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061874438469000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:58:02.873566] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4892-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061882840445, keepalive_ts:-1}) [2024-10-16 14:58:02.873582] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4892-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061882840445, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:02.875956] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:02.876326] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061882875364000, start_id=1729061882875364000, end_id=1729061882875364001) [2024-10-16 14:58:02.881118] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.888041] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=52] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1454, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:02.888080] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=51, stat_interval=1000000, avg (count/cost)=28, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:02.888088] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=51, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:02.891547] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:02.891565] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=4100, avg_group_log_size=372) [2024-10-16 14:58:02.891574] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:02.891820] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=337, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:02.891833] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=67, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:02.891855] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:02.892129] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1101-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=442, avg_log_submit_to_slide_time=621) [2024-10-16 14:58:02.897036] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.902683] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:02.902697] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27628320, tenant_id=1004) [2024-10-16 14:58:02.902706] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:02.904194] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=42] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:58:02.905729] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D51-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061882896957, keepalive_ts:-1}) [2024-10-16 14:58:02.905750] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D51-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061882896957, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:02.905773] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D50-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061882896908, keepalive_ts:-1}) [2024-10-16 14:58:02.905779] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D50-0-0] [lt=6] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061882896908, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:02.912425] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=49] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:02.913027] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:02.913073] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061881847501000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:58:02.913796] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] [COMPACTION DAG STATUS] (start_time=1729061872904938, end_time=1729061882913794, dag_status={}) [2024-10-16 14:58:02.913810] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:02.913817] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:02.913821] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:02.913825] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:02.913831] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:02.913835] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:02.913841] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:02.913844] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:02.913847] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:02.913853] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913863] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913867] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913873] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913877] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913884] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913888] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913894] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913898] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913902] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913908] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913912] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913919] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913923] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913929] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913933] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913940] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913943] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913950] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913954] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913958] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913965] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913969] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913975] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913979] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913986] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913990] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913993] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.913997] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914001] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914004] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914011] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914015] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914022] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914033] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914040] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914044] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914050] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914055] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914061] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914065] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914069] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914076] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914079] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914086] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914090] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914096] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914100] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914107] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914110] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914124] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=14] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914129] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914138] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914142] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914149] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914152] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914159] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914169] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:02.914174] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:58:02.914180] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:58:02.914184] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:58:02.914189] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:58:02.914191] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:58:02.914194] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:58:02.914196] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:58:02.914202] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:58:02.914204] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:58:02.914207] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:58:02.914213] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:58:02.918446] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=23] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:58:02.925114] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=15] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:02.926243] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:02.926325] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=8] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:02.930028] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1, min_version={val:1729061882730556000, v:0}, base_version={val:1729061882328528001, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061882730556000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061882930016, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061882730556000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061882838390, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061882730556000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061882886418, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:02.930777] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061882925840000, start_id=1729061882925840000, end_id=1729061882925840001) [2024-10-16 14:58:02.937341] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107484][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2019-0-0] [lt=18] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:02.937361] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107484][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE2019-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:02.939290] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:02.939308] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:02.939865] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD48C-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:02.939887] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD48C-0-0] [lt=21] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=708, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:02.942392] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBEE-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061882910113, keepalive_ts:-1}) [2024-10-16 14:58:02.942406] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBEE-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061882910113, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:02.945980] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4031, mem_used=8318976) [2024-10-16 14:58:02.958143] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:02.958162] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:02.958173] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=9] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=163) [2024-10-16 14:58:02.960344] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:58:02.960361] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:58:02.960367] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:58:02.960372] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:02.962612] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=7] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:58:02.962999] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=385] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:02.963290] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:58:02.967089] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=21] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:58:02.969753] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061882968314000, start_id=1729061882968314000, end_id=1729061882968314001) [2024-10-16 14:58:02.981180] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:02.983644] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=12] ====== tenant freeze timer task ====== [2024-10-16 14:58:02.990441] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:58:02.991155] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=10] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:02.991177] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=21] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:02.991196] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:02.991255] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061899029444000, latest_log_ts:{val:1729061882455772001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:02.991300] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=43] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:58:02.991316] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:58:02.991324] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061882455772001, v:0}) [2024-10-16 14:58:02.991328] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:02.991331] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:02.991337] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:02.991350] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:58:02.991357] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:02.991362] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:02.992674] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=13] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:58:02.997095] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.002050] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=23] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:58:03.022549] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:03.032890] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=57] get number(ret=0, service_type_=0, range=1, base_id=1729061883032736000, start_id=1729061883032736000, end_id=1729061883032736001) [2024-10-16 14:58:03.033081] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:03.033099] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151574520, tenant_id=1) [2024-10-16 14:58:03.033112] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:03.033170] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D82-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:03.033192] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D82-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061882660978004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061882660978004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:03.034287] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D82-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061883033819000, start_id=1729061883033819000, end_id=1729061883033819001) [2024-10-16 14:58:03.037050] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D82-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061883036747000, start_id=1729061883036747000, end_id=1729061883036747001) [2024-10-16 14:58:03.037124] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D82-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:03.037137] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D82-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:03.039377] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061883039376, bytes: 4446823009766, bw: 0.357962 MB/s, add_ts: 1000503, add_bytes: 375539 [2024-10-16 14:58:03.039511] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:03.039532] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=214, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:03.039540] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=28, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:03.039561] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:03.039750] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD48E-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:03.039763] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD48E-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=738, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:03.039804] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD48F-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=450, avg_log_submit_to_slide_time=690) [2024-10-16 14:58:03.042045] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:146) [22811][T1004_Compactio][T1004][YB4A0A000407-00061E99743D0D6F-0-0] [lt=11] InfoPoolResizeTask(cost_ts=6) [2024-10-16 14:58:03.043380] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:03.043396] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:03.043409] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=292010892, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:58:03.053391] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21574][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:03.053452] WDIAG read_multi_row (ob_table_access_helper.h:238) [21574][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:03.054273] INFO delete_row (ob_table_access_helper.h:312) [21574][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061883053462) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061883053462) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:58:03.057642] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061883055925000, start_id=1729061883055925000, end_id=1729061883055925001) [2024-10-16 14:58:03.063663] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F7A-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:03.063686] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F7A-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:03.063692] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F7A-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20129, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:03.063827] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:03.067301] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061883067279000, start_id=1729061883067279000, end_id=1729061883067279001) [2024-10-16 14:58:03.067322] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=649, get_gts_with_stc_cnt=28, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=28, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:03.067332] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:03.067353] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=540, get_gts_with_stc_cnt=20, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=8, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:03.067368] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061883067360], gts:1729061883067279000, latest_srr:[mts=1729061883067360]}) [2024-10-16 14:58:03.076530] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061883075778000, start_id=1729061883075778000, end_id=1729061883075778001) [2024-10-16 14:58:03.076849] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:03.079292] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE790F-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1817) [2024-10-16 14:58:03.079747] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Cache wash clean map node details(ret=0, clean_node_count=19, clean_time=0, clean_start_pos=1698678, clean_num=62914) [2024-10-16 14:58:03.079880] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:03.081244] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.086495] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=20) [2024-10-16 14:58:03.088425] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:58:03.092958] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4099) [2024-10-16 14:58:03.092985] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4099) [2024-10-16 14:58:03.093226] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1103-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:03.093240] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1103-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=705, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:03.097133] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.101471] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1694, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:03.124580] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=16] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97898369024) [2024-10-16 14:58:03.128200] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:03.128213] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:03.130152] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] no need to update min/max version(new_version={val:1729061882730556000, v:0}, min_version={val:1729061882730556000, v:0}, max_version={val:1729061882730556000, v:0}) [2024-10-16 14:58:03.138026] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4530, mem_used=8318976) [2024-10-16 14:58:03.139641] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=187, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:03.139658] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:03.149923] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:58:03.161559] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BAA2-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061883161544000, start_id=1729061883161544000, end_id=1729061883161544001) [2024-10-16 14:58:03.161672] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BAA1-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061883161544000, start_id=1729061883161544000, end_id=1729061883161544001) [2024-10-16 14:58:03.161706] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAA5-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061883161544000, start_id=1729061883161544000, end_id=1729061883161544001) [2024-10-16 14:58:03.161809] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BAA4-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061883161544000, start_id=1729061883161544000, end_id=1729061883161544001) [2024-10-16 14:58:03.161860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BAA3-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061883161544000, start_id=1729061883161544000, end_id=1729061883161544001) [2024-10-16 14:58:03.163363] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:58:03.164288] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD493-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=1361, avg_log_submit_to_slide_time=20018) [2024-10-16 14:58:03.168739] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 123.00; ) [2024-10-16 14:58:03.168766] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=26] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:03.168802] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 133.50; ) [2024-10-16 14:58:03.168808] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:03.168816] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:03.170775] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061883169908000, start_id=1729061883169908000, end_id=1729061883169908001) [2024-10-16 14:58:03.181308] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.188193] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2DFD-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:58:03.197655] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.203526] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5041-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:03.203547] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5041-0-0] [lt=20] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:03.225301] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061883225299, bytes: 0, bw: 0.000000 MB/s, add_ts: 1005615, add_bytes: 0 [2024-10-16 14:58:03.231027] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061883230587000, start_id=1729061883230587000, end_id=1729061883230587001) [2024-10-16 14:58:03.233032] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061883232702000, start_id=1729061883232702000, end_id=1729061883232702001) [2024-10-16 14:58:03.233092] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=25, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:03.233111] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:03.239434] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:03.239458] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:03.239780] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:03.253436] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:03.253864] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:03.253895] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:03.254397] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4571-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:03.254413] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4571-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=700, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:03.257718] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22795][T1004_TenantWea][T1003][YB4A0A000407-00061E9B3AFB9781-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.248164176940917969e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=312, Throttle Trigger(MB)=187, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.830590983287007475e+03) [2024-10-16 14:58:03.257876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061883257112000, start_id=1729061883257112000, end_id=1729061883257112001) [2024-10-16 14:58:03.264182] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:03.276725] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061883276181000, start_id=1729061883276181000, end_id=1729061883276181001) [2024-10-16 14:58:03.280040] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190100992, global_cache_size=1581383680, tenant_max_wash_size=7525990, tenant_min_wash_size=7525990, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:03.281371] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.288955] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=61] succ to submit log task to replay service(to_submit_lsn={lsn:14538514714}, to_submit_scn={val:1729061883161544002, v:0}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061883288926, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538514780}, next_to_submit_scn_:{val:1729061883161544003, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538514658}, end_lsn:{lsn:14538514780}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061883161544002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061883161544002, v:0}, data_checksum:3348837161, flag:1}}, init_mode_version:0, accumulate_checksum:2829553450, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061883161544002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650430, miss_cnt:32, cache_read_size:8050443865, hit_ratio:"9.999994892296245608e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:58:03.292536] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DE7-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510351160921, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410646, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:03.297711] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.301646] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8514D-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:03.301677] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8514D-0-0] [lt=23] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:03.301944] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:03.318449] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1041-0-0] [lt=12] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:58:03.326266] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA895-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:03.326298] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA895-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061883161544001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061883161544001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:03.327614] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA895-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061883327050000, start_id=1729061883327050000, end_id=1729061883327050001) [2024-10-16 14:58:03.330037] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=909, mem_used=2079744) [2024-10-16 14:58:03.330260] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA895-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061883330230000, start_id=1729061883330230000, end_id=1729061883330230001) [2024-10-16 14:58:03.330352] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA895-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9326, avg_group_log_size=666) [2024-10-16 14:58:03.330382] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA895-0-0] [lt=27] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9326) [2024-10-16 14:58:03.330684] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=230, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:03.330699] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9326) [2024-10-16 14:58:03.330710] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=64, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:03.330730] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9326) [2024-10-16 14:58:03.330740] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=10, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:03.331067] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9645-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:03.331080] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9645-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=575, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:03.331084] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9645-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=317, avg_log_submit_to_slide_time=541) [2024-10-16 14:58:03.331107] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:03.335999] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=16] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:03.339991] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD498-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=475, avg_log_submit_to_slide_time=692) [2024-10-16 14:58:03.353915] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:03.353940] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:03.354242] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:03.354241] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=215, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:03.354258] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=86, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:03.354275] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=2194) [2024-10-16 14:58:03.354758] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4573-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=21, avg_log_submit_to_flush_time=407, avg_log_submit_to_slide_time=683) [2024-10-16 14:58:03.363405] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:58:03.370809] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061883370470000, start_id=1729061883370470000, end_id=1729061883370470001) [2024-10-16 14:58:03.377510] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB8-0-0] [lt=9] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:58:03.377528] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB8-0-0] [lt=13] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=18) [2024-10-16 14:58:03.379394] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7915-0-0] [lt=21] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1817, avg_group_log_size=181) [2024-10-16 14:58:03.379616] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1817) [2024-10-16 14:58:03.381425] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.388489] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=206, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:03.388513] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:03.388549] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:03.388740] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92D7-0-0] [lt=31] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1454) [2024-10-16 14:58:03.392726] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=11] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:03.392957] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4221) [2024-10-16 14:58:03.395619] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=18] gc stale ls task succ [2024-10-16 14:58:03.396922] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=15] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:58:03.397764] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.398961] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=10] query dest_info_ succ(dest_info=[], archive_dest_changed=false) [2024-10-16 14:58:03.398981] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=18] finish to resize log external storage handler(current_ts=1729061883398979, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:58:03.408983] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C11DA-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061883408535000, start_id=1729061883408535000, end_id=1729061883408535001) [2024-10-16 14:58:03.431277] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061883430291000, start_id=1729061883430291000, end_id=1729061883430291001) [2024-10-16 14:58:03.433062] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061883432409000, start_id=1729061883432409000, end_id=1729061883432409001) [2024-10-16 14:58:03.433742] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=25] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:58:03.439796] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:03.454217] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:03.457076] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=13] no need to update min/max version(new_version={val:1729061882660978004, v:0}, min_version={val:1729061882660978004, v:0}, max_version={val:1729061882660978004, v:0}) [2024-10-16 14:58:03.475924] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] no need to update min/max version(new_version={val:1729061883075778000, v:0}, min_version={val:1729061883075778000, v:0}, max_version={val:1729061883075778000, v:0}) [2024-10-16 14:58:03.479401] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7917-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:03.479622] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=213, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:03.479645] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=49, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:03.480322] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:03.481488] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.486867] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601490, offset=9650014}, end_cursor:ObLogCursor{file_id=192, log_id=47601491, offset=9650298}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7382050, len:4258, log_data_len:284, seq:47601490, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:03.488797] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4B9-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061883486742000, start_id=1729061883486742000, end_id=1729061883486742001) [2024-10-16 14:58:03.489705] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601491, offset=9650298}, end_cursor:ObLogCursor{file_id=192, log_id=47601492, offset=9650572}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2060050, len:3974, log_data_len:274, seq:47601491, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:03.489789] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:468) [21814][T1001_SSTableDe][T1001][Y0-0000000000000000-0-0] [lt=10] skip defragment task(ret=0, need_defragment=false) [2024-10-16 14:58:03.489799] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [21814][T1001_SSTableDe][T1001][Y0-0000000000000000-0-0] [lt=8] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=10) [2024-10-16 14:58:03.489815] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [21814][T1001_SSTableDe][T1001][Y0-0000000000000000-0-0] [lt=16] successfully defragment data blocks(rewrite_cnt=0) [2024-10-16 14:58:03.489820] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [21814][T1001_SSTableDe][T1001][Y0-0000000000000000-0-0] [lt=3] tablet skip filter destructed(total_cnt=0, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105370475, total_skip_cnt=310305, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:58:03.490668] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A95-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:03.490682] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A95-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:03.491016] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:58:03.491030] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:58:03.491045] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] cannot_recycle_log_size statistics(cannot_recycle_log_size=70901782, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:03.491769] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601492, offset=9650572}, end_cursor:ObLogCursor{file_id=192, log_id=47601493, offset=9650846}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2038050, len:3700, log_data_len:274, seq:47601492, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:03.491921] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601493, offset=9650846}, end_cursor:ObLogCursor{file_id=192, log_id=47601494, offset=9651120}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2174050, len:3426, log_data_len:274, seq:47601493, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:03.491968] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561499, miss_cnt:1, cache_read_size:152590709813, hit_ratio:"9.999999928347000910e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:03.491980] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599083, miss_cnt:9, cache_read_size:149777939850, hit_ratio:"9.999999345925916217e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:03.493212] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:03.493221] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=8] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:58:03.493230] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:58:03.493235] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:58:03.493239] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:58:03.493241] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:58:03.493231] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2D-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:03.493244] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:58:03.493246] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:58:03.493249] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:58:03.493247] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2D-0-0] [lt=13] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:03.493251] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:58:03.493254] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:58:03.497797] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.502135] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=52, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:03.502157] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:03.509847] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=48] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:03.510413] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=13] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:03.510454] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061878735149000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:58:03.510493] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=39] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061878735149000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1001, freeze_info:{latest_snapshot_gc_scn:{val:1729061878735149000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}}) [2024-10-16 14:58:03.510565] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=4] Garbage Collector is running(seq_=656497, gc_interval=10000000) [2024-10-16 14:58:03.510584] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:03.510590] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=4] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:58:03.512183] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:58:03.512903] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:03.512924] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=21] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:03.512942] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:03.513011] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061895590307000, latest_log_ts:{val:1729061878965510001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:03.513055] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=41] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:03.513072] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:58:03.513079] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061878965510001, v:0}) [2024-10-16 14:58:03.513083] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:58:03.513086] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:03.513093] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:58:03.513102] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:58:03.513110] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:03.513120] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=9] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:03.513157] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:58:03.514181] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:58:03.514196] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=10] gc_check_member_list_ cost time(ret=0, time_us=3624) [2024-10-16 14:58:03.514207] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:58:03.514851] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:03.515489] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=17] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:03.516074] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=15] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:03.516664] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=14] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:03.516686] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=19] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:03.516697] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=11] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:03.516705] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=7] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:03.516710] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:03.516715] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=30) [2024-10-16 14:58:03.516721] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] ObSafeDestroyHandler start process [2024-10-16 14:58:03.539422] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257838, offset=39054482}, end_cursor:ObLogCursor{file_id=197, log_id=49257839, offset=39054765}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2244050, len:878, log_data_len:283, seq:49257838, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:03.539867] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:03.541846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB384-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061883541836000, start_id=1729061883541836000, end_id=1729061883541836001) [2024-10-16 14:58:03.542760] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=39] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257839, offset=39054765}, end_cursor:ObLogCursor{file_id=197, log_id=49257840, offset=39055047}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73be050, len:595, log_data_len:282, seq:49257839, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:03.544692] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=911101952, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=162639872, wash_ratio=6, tenant_id=1001) [2024-10-16 14:58:03.544745] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=25] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:03.544757] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=11] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:03.544793] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=11] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=971796480, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=101945344, wash_ratio=6, tenant_id=1002) [2024-10-16 14:58:03.544830] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:03.544838] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:58:03.544866] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=927879168, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=145862656, wash_ratio=6, tenant_id=1003) [2024-10-16 14:58:03.544906] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=14] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:03.544913] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011874, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:03.544945] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=958939136, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=114802688, wash_ratio=6, tenant_id=1004) [2024-10-16 14:58:03.544980] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:03.544987] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:03.545024] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257840, offset=39055047}, end_cursor:ObLogCursor{file_id=197, log_id=49257841, offset=39055329}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73fa050, len:4409, log_data_len:282, seq:49257840, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:03.545353] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257841, offset=39055329}, end_cursor:ObLogCursor{file_id=197, log_id=49257842, offset=39055611}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72ee050, len:4127, log_data_len:282, seq:49257841, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:03.545413] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657170, miss_cnt:17, cache_read_size:8050191359, hit_ratio:"9.999997286823616438e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:03.545427] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=61] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650431, miss_cnt:32, cache_read_size:8050443987, hit_ratio:"9.999994892296326654e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:03.554017] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:03.554035] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:03.554068] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:03.554077] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:03.554329] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:03.554540] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4579-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:03.554549] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4579-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=701, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:03.554586] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=30, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:58:03.554596] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=1047, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:58:03.554606] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=9] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:58:03.554609] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=-507, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:58:03.554613] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=21, stat_interval=5000000, avg (count/cost)=569, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:58:03.557938] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061883557795000, start_id=1729061883557795000, end_id=1729061883557795001) [2024-10-16 14:58:03.558623] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=15] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:58:03.558643] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:14538514902}, scn={val:1729061883161544004, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061883545397, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538514902}, next_to_submit_scn_:{val:1729061883161544004, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538514780}, end_lsn:{lsn:14538514902}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061883161544003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061883161544003, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:2668028057, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061883161544003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650431, miss_cnt:32, cache_read_size:8050443987, hit_ratio:"9.999994892296326654e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:58:03.558687] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=41] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:58:03.563489] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:58:03.567692] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061883567257000, start_id=1729061883567257000, end_id=1729061883567257001) [2024-10-16 14:58:03.567719] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061883567715], gts:1729061883567257000, latest_srr:[mts=1729061883567715]}) [2024-10-16 14:58:03.567773] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=13] submit log success(service_type_=0, last_id=1729061900098662000, limited_id=1729061900098662000) [2024-10-16 14:58:03.567779] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21292][TsMgr][T1002][Y0-0000000000000000-0-0] [lt=6] submit log with lock success(service_type_=0, last_id=1729061900098662000, limited_id=1729061900098662000) [2024-10-16 14:58:03.568347] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061900098662000, latest_scn={val:1729061883567257001, v:0}, *this={tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881384043000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900098662000, latest_log_ts:{val:1729061883567257001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:58:03.568382] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=33] handle submit callback(service_type_=0, success=true, used_ts=609, log_ts={val:1729061883567257001, v:0}, limited_id=1729061900098662000) [2024-10-16 14:58:03.568388] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:58:03.568395] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=7] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:58:03.568622] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.569462] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.569465] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.569476] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.570908] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=44] get number(ret=0, service_type_=0, range=1, base_id=1729061883570419000, start_id=1729061883570419000, end_id=1729061883570419001) [2024-10-16 14:58:03.576771] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061883575725000, start_id=1729061883575725000, end_id=1729061883575725001) [2024-10-16 14:58:03.579847] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5566-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:03.579860] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5566-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=551, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:03.579859] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1939, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:03.581528] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.582151] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26148][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB19CC6EE-0-0] [lt=57] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:58:03.588431] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E05-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:03.588456] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E05-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:03.588461] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E05-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22573, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:03.588466] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E05-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178676, log_task={header:{begin_lsn:{lsn:14538514902}, end_lsn:{lsn:14538515024}, log_id:115178676, min_scn:{val:1729061883161544004, v:0}, max_scn:{val:1729061883161544004, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538514780}, prev_proposal_id:23, committed_end_lsn:{lsn:14538514902}, data_checksum:1715074492, accum_checksum:2803975206, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061883488369, freeze_ts:1729061883488369, submit_ts:1729061883488374, flushed_ts:1729061883489146, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:772}, fs_cb_begin_ts=1729061883588451, log_life_time=100082) [2024-10-16 14:58:03.597134] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:03.597149] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:03.597854] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.598065] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}, {id:1002}, {id:1001}, {id:1003}]) [2024-10-16 14:58:03.601760] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85153-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:03.601944] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=185, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:03.601962] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=66, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:03.602001] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:03.602251] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C457C-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:03.602257] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C457C-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10372, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:03.607474] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4813-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061883575196, keepalive_ts:-1}) [2024-10-16 14:58:03.607491] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4813-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061883575196, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:03.611401] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5792, mem_used=10398720) [2024-10-16 14:58:03.631513] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061883631412000, start_id=1729061883631412000, end_id=1729061883631412001) [2024-10-16 14:58:03.632241] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=21] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:03.633237] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061883631412000, start_id=1729061883631412000, end_id=1729061883631412001) [2024-10-16 14:58:03.633823] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061883161544002, v:0}, new_max_version={val:1729061883161544002, v:0}, last_min_version={val:1729061883067279000, v:0}, last_max_version={val:1729061883067279000, v:0}, total_time_us=1392, delta_us=472278) [2024-10-16 14:58:03.633842] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061883161544002, v:0}, min_version={val:1729061883161544002, v:0}, max_version={val:1729061883161544002, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:58:03.639679] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:03.639693] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:03.642612] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=10] replay status is not follower(min_unreplayed_lsn={lsn:33216819320}, base_lsn={lsn:33149755392}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:58:03.642666] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=53] get_replay_process success(id={id:1}, replayed_log_size=67063928, unreplayed_log_size=0) [2024-10-16 14:58:03.642683] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=4] dump tenant replay process(tenant_id=1, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=63, last_replayed_log_size(MB)=63, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:58:03.655111] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=36] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:03.658519] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:468) [22707][T1004_SSTableDe][T1004][Y0-0000000000000000-0-0] [lt=10] skip defragment task(ret=0, need_defragment=false) [2024-10-16 14:58:03.658534] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:510) [22707][T1004_SSTableDe][T1004][Y0-0000000000000000-0-0] [lt=14] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=18) [2024-10-16 14:58:03.658540] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:528) [22707][T1004_SSTableDe][T1004][Y0-0000000000000000-0-0] [lt=5] successfully defragment data blocks(rewrite_cnt=0) [2024-10-16 14:58:03.658545] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:697) [22707][T1004_SSTableDe][T1004][Y0-0000000000000000-0-0] [lt=3] tablet skip filter destructed(total_cnt=0, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=105370475, total_skip_cnt=310305, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2024-10-16 14:58:03.661820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BAA9-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061883660990000, start_id=1729061883660990000, end_id=1729061883660990001) [2024-10-16 14:58:03.661966] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAAA-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061883660990000, start_id=1729061883660990000, end_id=1729061883660990001) [2024-10-16 14:58:03.662127] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BAA8-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061883662065000, start_id=1729061883662065000, end_id=1729061883662065001) [2024-10-16 14:58:03.665459] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BAA7-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061883665319000, start_id=1729061883665319000, end_id=1729061883665319001) [2024-10-16 14:58:03.665555] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26166][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BAA6-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061883665319000, start_id=1729061883665319000, end_id=1729061883665319001) [2024-10-16 14:58:03.670591] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:58:03.670611] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=18] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:58:03.670618] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:58:03.670622] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:03.672738] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=108, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}, {obj_id:616254, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616255, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616256, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:616257, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}]) [2024-10-16 14:58:03.672885] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=145] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:03.675782] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=8] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:03.680619] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:03.681591] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.688527] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E07-0-0] [lt=18] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=344, avg_log_submit_to_slide_time=33629) [2024-10-16 14:58:03.691737] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=14] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:58:03.692519] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061883692418000, start_id=1729061883692418000, end_id=1729061883692418001) [2024-10-16 14:58:03.697881] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:03.697903] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:03.697914] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.700219] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:03.725046] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:03.726769] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22859][T1004_L0_G2][T1004][YB4A0A000409-00061E9A39C4D8E8-0-0] [lt=52] get_min_unreplayed_log_info(lsn={lsn:14352401382}, scn={val:1729061883660990002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061883664564, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352401382}, next_to_submit_scn_:{val:1729061883660990002, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352401260}, end_lsn:{lsn:14352401382}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061883660990001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061883660990001, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:1667911737, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061883660990001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657172, miss_cnt:17, cache_read_size:8050191603, hit_ratio:"9.999997286823703035e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:58:03.729346] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:03.739470] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D9-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:03.739496] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9D9-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061883660990001, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061883660990001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:03.739682] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:58:03.739696] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] try gc retain ctx [2024-10-16 14:58:03.739700] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:58:03.739716] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] get_max_applied_scn(scn={val:1729061883567257000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493503050}, palf_committed_end_scn_:{val:1729061883692418000, v:0}, last_check_scn_:{val:1729061883692418000, v:0}, max_applied_cb_scn_:{val:1729061883567257000, v:0}}) [2024-10-16 14:58:03.739739] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=20] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:58:03.740388] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=44] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061890721265000, latest_log_ts:{val:1729061874059797001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:03.740514] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=48] get_max_applied_scn(scn={val:1729061883567257000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645445166}, palf_committed_end_scn_:{val:1729061883567257000, v:0}, last_check_scn_:{val:1729061883567257000, v:0}, max_applied_cb_scn_:{val:1729061883567257000, v:0}}) [2024-10-16 14:58:03.740527] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:58:03.741266] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=34] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:03.742082] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=38] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:03.742787] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=46] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:03.743830] INFO [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:262) [22249][T1002_ArcSender][T1002][YB4A0A000407-00061E996F9960C1-0-0] [lt=13] ObArchiveSender is running(thread_index=1) [2024-10-16 14:58:03.745430] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D9-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061883744406000, start_id=1729061883744406000, end_id=1729061883744406001) [2024-10-16 14:58:03.745495] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D9-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=8908, avg_group_log_size=636) [2024-10-16 14:58:03.745505] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9D9-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=8908) [2024-10-16 14:58:03.745726] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=211, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:03.745740] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=8908) [2024-10-16 14:58:03.745749] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=217, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:03.745763] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10641) [2024-10-16 14:58:03.745776] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:03.746009] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2A9-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:03.746021] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2A9-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=716, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:03.746030] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2A9-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=498, avg_log_submit_to_slide_time=712) [2024-10-16 14:58:03.746180] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:03.748356] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=18] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:58:03.754386] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:03.758167] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061883758107000, start_id=1729061883758107000, end_id=1729061883758107001) [2024-10-16 14:58:03.759336] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:03.759358] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=12] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:03.759365] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:03.763567] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:03.771071] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061883770812000, start_id=1729061883770812000, end_id=1729061883770812001) [2024-10-16 14:58:03.773028] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:03.776937] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061883776069000, start_id=1729061883776069000, end_id=1729061883776069001) [2024-10-16 14:58:03.781657] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.782799] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=22] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:03.788728] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.788797] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.788815] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.788836] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.788853] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.788856] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.788863] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.788868] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.788870] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.788877] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.789133] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:03.794534] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C66-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:03.794563] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C66-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061883662065002, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061883662065002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:03.795509] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C66-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061883794449000, start_id=1729061883794449000, end_id=1729061883794449001) [2024-10-16 14:58:03.797928] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C66-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061883797672000, start_id=1729061883797672000, end_id=1729061883797672001) [2024-10-16 14:58:03.798033] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.808746] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [21908][T1001_MediumLoo][T1001][YB4A0A000407-00061E9973AF38E6-0-0] [lt=12] MediumCheckTask(cost_ts=6) [2024-10-16 14:58:03.820123] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A13-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061883811339, keepalive_ts:-1}) [2024-10-16 14:58:03.820143] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A13-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061883811339, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:03.829588] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:58:03.829609] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=17] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:58:03.829623] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52647186, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:03.830785] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] no need to update min/max version(new_version={val:1729061883430291000, v:0}, min_version={val:1729061883430291000, v:0}, max_version={val:1729061883430291000, v:0}) [2024-10-16 14:58:03.833397] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061883832640000, start_id=1729061883832640000, end_id=1729061883832640001) [2024-10-16 14:58:03.833541] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=27] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1002}, {id:1003}]) [2024-10-16 14:58:03.839781] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=36] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1758, avg_group_log_size=159) [2024-10-16 14:58:03.839799] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1758) [2024-10-16 14:58:03.840005] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1758) [2024-10-16 14:58:03.854419] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=228, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:03.854438] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=128, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:03.854447] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:03.854726] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4582-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=462, avg_log_submit_to_slide_time=665) [2024-10-16 14:58:03.862019] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=6250, mem_used=12478464) [2024-10-16 14:58:03.864030] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F8A-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:03.864309] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=190, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:03.864328] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:03.864367] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:03.864623] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92E0-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:03.880914] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1581383680, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:03.881715] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.889089] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:03.889132] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:03.889144] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:03.892741] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:14352401682}, scn={val:1729061883660990004, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:58:03.892743] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:29, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061883892707, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352401738}, next_to_submit_scn_:{val:1729061883660990005, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352401626}, end_lsn:{lsn:14352401738}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061883660990004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061883660990004, v:0}, data_checksum:2973684320, flag:1}}, init_mode_version:0, accumulate_checksum:3809047575, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061883660990004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657175, miss_cnt:17, cache_read_size:8050191959, hit_ratio:"9.999997286823832932e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14352401682}, scn_:{val:1729061883660990004, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061883860007, first_handle_ts_:1729061883892742, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f77c2a560, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:58:03.892894] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060081847501000) [2024-10-16 14:58:03.892906] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14352401682}, scn_:{val:1729061883660990004, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061883860007, first_handle_ts_:1729061883892879, replay_cost_:26, retry_cost_:-1, read_log_buf_:0x2b0f77c2a570, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061883892707, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352401738}, next_to_submit_scn_:{val:1729061883660990005, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352401626}, end_lsn:{lsn:14352401738}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061883660990004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061883660990004, v:0}, data_checksum:2973684320, flag:1}}, init_mode_version:0, accumulate_checksum:3809047575, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061883660990004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657175, miss_cnt:17, cache_read_size:8050191959, hit_ratio:"9.999997286823832932e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}, replay_queue_idx=7, ret=0) [2024-10-16 14:58:03.893215] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:03.893247] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3079, avg_group_log_size=307) [2024-10-16 14:58:03.893256] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:03.893478] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=281, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:03.893491] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:03.893506] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:03.893638] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C110B-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=377, avg_log_submit_to_slide_time=669) [2024-10-16 14:58:03.898092] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.924411] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=20] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}) [2024-10-16 14:58:03.926317] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=10] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:03.926375] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=9] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:03.928069] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=16] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:03.931713] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061883930736000, start_id=1729061883930736000, end_id=1729061883930736001) [2024-10-16 14:58:03.939865] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:03.939881] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:03.940398] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4AB-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:03.940416] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4AB-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=489, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:03.944303] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15EA-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:03.944370] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15EC-0-0] [lt=13] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:03.946087] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4061, mem_used=8318976) [2024-10-16 14:58:03.958228] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061883957905000, start_id=1729061883957905000, end_id=1729061883957905001) [2024-10-16 14:58:03.965130] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2453646, replace_num=31457) [2024-10-16 14:58:03.965149] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:58:03.971274] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061883970555000, start_id=1729061883970555000, end_id=1729061883970555001) [2024-10-16 14:58:03.977066] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061883976906000, start_id=1729061883976906000, end_id=1729061883976906001) [2024-10-16 14:58:03.981774] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:03.993260] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:03.993284] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=216) [2024-10-16 14:58:03.998157] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.033511] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061884031950000, start_id=1729061884031950000, end_id=1729061884031950001) [2024-10-16 14:58:04.039910] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=18] PNIO [ratelimit] time: 1729061884039909, bytes: 4446823348024, bw: 0.322416 MB/s, add_ts: 1000533, add_bytes: 338258 [2024-10-16 14:58:04.039953] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] ====== checkpoint timer task ====== [2024-10-16 14:58:04.040708] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.040735] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.040759] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=20] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:04.040893] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881384043000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900098662000, latest_log_ts:{val:1729061883567257001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:04.040939] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=42] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881384043000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900098662000, latest_log_ts:{val:1729061883567257001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:58:04.040967] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=24] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:58:04.040974] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061883567257001, v:0}) [2024-10-16 14:58:04.040978] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:58:04.040985] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:04.040994] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:58:04.041002] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:04.041734] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.041748] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.041759] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:58:04.041799] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881384043000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:04.041828] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881384043000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:58:04.041851] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:58:04.041857] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:58:04.041863] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:58:04.041868] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:58:04.042560] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.042572] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.042582] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:58:04.042623] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881384043000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:04.042650] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881384043000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:58:04.042673] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=21] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:58:04.042678] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:58:04.042684] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:58:04.042689] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:58:04.043403] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.043419] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.043430] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:58:04.043463] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881384043000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:04.043493] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=29] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881384043000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:58:04.043517] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:58:04.043523] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:58:04.043530] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:58:04.043535] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:58:04.043539] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:58:04.043796] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:04.043897] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=613, avg_log_submit_to_slide_time=819) [2024-10-16 14:58:04.043931] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=291, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:04.043952] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=378, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:04.043959] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:04.043965] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1174, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:04.043969] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:04.050972] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:04.056161] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:04.064102] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F8E-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1323, avg_group_log_size=120) [2024-10-16 14:58:04.064133] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F8E-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:04.064141] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F8E-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9479, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:04.064300] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1323) [2024-10-16 14:58:04.068118] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061884067775000, start_id=1729061884067775000, end_id=1729061884067775001) [2024-10-16 14:58:04.068142] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:04.068158] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061884068155], gts:1729061884067775000, latest_srr:[mts=1729061884068155]}) [2024-10-16 14:58:04.068366] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21292][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=11] submit log success(service_type_=0, last_id=1729061900721265000, limited_id=1729061900721265000) [2024-10-16 14:58:04.068373] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21292][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=6] submit log with lock success(service_type_=0, last_id=1729061900721265000, limited_id=1729061900721265000) [2024-10-16 14:58:04.068789] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] update id meta(ret=0, service_type=0, limited_id=1729061900721265000, latest_scn={val:1729061884067775001, v:0}, *this={tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900721265000, latest_log_ts:{val:1729061884067775001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:58:04.068824] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=35] handle submit callback(service_type_=0, success=true, used_ts=458, log_ts={val:1729061884067775001, v:0}, limited_id=1729061900721265000) [2024-10-16 14:58:04.068847] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=23] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:58:04.068852] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:58:04.071749] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D83-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:04.071769] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D83-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061883662065003, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061883662065003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:04.072814] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D83-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061884072426000, start_id=1729061884072426000, end_id=1729061884072426001) [2024-10-16 14:58:04.075273] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D83-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061884075069000, start_id=1729061884075069000, end_id=1729061884075069001) [2024-10-16 14:58:04.075341] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D83-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:04.075358] INFO [LIB] stat (utility.h:1165) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D83-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:04.079614] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] Garbage Collector is running(seq_=656494, gc_interval=10000000) [2024-10-16 14:58:04.079644] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=14] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:04.079655] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:58:04.079835] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7923-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:04.080057] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:04.081276] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1188003840, global_cache_size=1581383680, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:04.081719] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash memory, (tenant_id=1002, cache_size=87392256, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:58:04.081834] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.081851] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash time detail, (compute_wash_size_time=143, refresh_score_time=90, wash_time=342) [2024-10-16 14:58:04.084637] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=8] Cache wash clean map node details(ret=0, clean_node_count=27, clean_time=0, clean_start_pos=1761592, clean_num=62914) [2024-10-16 14:58:04.084816] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=4] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:58:04.085320] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=7] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:58:04.085334] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=9] gc_check_member_list_ cost time(ret=0, time_us=5707) [2024-10-16 14:58:04.085343] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:58:04.086137] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:04.086757] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=13] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:04.087363] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=21] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:04.087969] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=14] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:04.087984] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=12] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:04.087992] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:04.087998] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:04.088006] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:04.088014] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] execute_gc cost time(ret=0, time_us=31) [2024-10-16 14:58:04.088018] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=2] ObSafeDestroyHandler start process [2024-10-16 14:58:04.088025] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:133) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-10-16 14:58:04.088037] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:140) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-10-16 14:58:04.088979] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:04.093531] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3080) [2024-10-16 14:58:04.093566] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3080) [2024-10-16 14:58:04.093712] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C110D-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:04.093726] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C110D-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1052, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:04.098211] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.101221] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061879980574, last_first_register_time_us=1722494150998907, register_time_us=1729061879980574, curr_time_us=1729061884101220) [2024-10-16 14:58:04.101257] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061884101238, last_submit_register_req_time_us=1729061884101238, register_parent_reason=3) [2024-10-16 14:58:04.101265] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:58:04.102505] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:04.128284] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:04.128305] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=21][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:04.132168] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061884132076000, start_id=1729061884132076000, end_id=1729061884132076001) [2024-10-16 14:58:04.138115] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4569, mem_used=8318976) [2024-10-16 14:58:04.140211] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=192, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:04.140229] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=390, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:04.149981] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:58:04.154589] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=24644263, this={dio_aligned_buf:{buf_write_offset:2849, buf_padding_size:1247, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:93867065, truncate_used_ts:39944974}, log_block_size:67108864, dir_fd:173, io_fd:196}, aligned_buf_len=4096, aligned_block_offset=24641536, buf_len=122, total_write_size_=19333, total_write_size_after_dio_=438272, ob_pwrite_used_ts=25420, count=102, each_pwrite_cost=249) [2024-10-16 14:58:04.158309] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061884157601000, start_id=1729061884157601000, end_id=1729061884157601001) [2024-10-16 14:58:04.160120] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=13] current monitor number(seq_=-1) [2024-10-16 14:58:04.162238] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BAAB-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061884161808000, start_id=1729061884161808000, end_id=1729061884161808001) [2024-10-16 14:58:04.162313] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BAAC-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061884161808000, start_id=1729061884161808000, end_id=1729061884161808001) [2024-10-16 14:58:04.162374] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BAAD-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061884161808000, start_id=1729061884161808000, end_id=1729061884161808001) [2024-10-16 14:58:04.162398] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAAF-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061884161808000, start_id=1729061884161808000, end_id=1729061884161808001) [2024-10-16 14:58:04.164629] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92E7-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=220, avg_log_submit_to_slide_time=9469) [2024-10-16 14:58:04.165229] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=9] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:58:04.170491] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 175.00; ) [2024-10-16 14:58:04.170516] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:04.170554] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 6144.00, iops: 3.99, rt: 260.00; ) [2024-10-16 14:58:04.170564] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:04.170580] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:04.172487] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061884172376000, start_id=1729061884172376000, end_id=1729061884172376001) [2024-10-16 14:58:04.177068] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061884176591000, start_id=1729061884176591000, end_id=1729061884176591001) [2024-10-16 14:58:04.181896] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.188730] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E11-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:04.198276] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.204422] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5042-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:04.204439] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5042-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:04.214487] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5881-0-0] [lt=9] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:58:04.214645] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5881-0-0] [lt=21][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:04.217896] INFO [SHARE.LOCATION] renew_all_ls_locations_by_rpc (ob_ls_location_service.cpp:801) [21248][AutoLSLocRpc][T0][YB4A0A000407-00061E9B390A226E-0-0] [lt=54] [LS_LOCATION] Get ls leaders by RPC(ret=0, ret="OB_SUCCESS", dests=["10.0.4.7:2890", "10.0.4.8:2890", "10.0.4.9:2890"], leaders=[{key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}}, {key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}}, {key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}}, {key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}}, {key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}}, {key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}}, {key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}}, {key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}}, {key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}]) [2024-10-16 14:58:04.228059] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [22325][T1003_COMMONLSS][T1003][YB4A0A000407-00061E9B3A19CB2A-0-0] [lt=21][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:58:04.232913] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=18] PNIO [ratelimit] time: 1729061884232911, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007612, add_bytes: 0 [2024-10-16 14:58:04.233613] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061884232624000, start_id=1729061884232624000, end_id=1729061884232624001) [2024-10-16 14:58:04.233679] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:04.233700] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:04.240022] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:04.240041] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:04.240585] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40113-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1764) [2024-10-16 14:58:04.246058] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD12-0-0] [lt=15][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:04.254403] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:04.254424] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:04.254915] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F43AE-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:04.254927] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F43AE-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=617, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:04.256308] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] ====== checkpoint timer task ====== [2024-10-16 14:58:04.257063] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.257091] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=28] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.257114] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:04.257235] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900721265000, latest_log_ts:{val:1729061884067775001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:04.257284] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=45] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:04.257304] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=15] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:58:04.257310] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061884067775001, v:0}) [2024-10-16 14:58:04.257314] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.257321] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:04.257335] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:58:04.257345] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:04.258086] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.258103] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.258118] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:58:04.258187] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:04.258225] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=37] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:58:04.258250] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=23] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:58:04.258257] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:58:04.258264] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:58:04.258273] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:58:04.258978] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.258993] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.259007] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:58:04.259079] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:04.259109] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=29] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:58:04.259132] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=23] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:58:04.259138] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:58:04.259144] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:58:04.259149] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:58:04.259876] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.259892] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:04.259901] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:58:04.259971] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:04.260003] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061881065386003, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:58:04.260034] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:58:04.260043] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:58:04.260052] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:58:04.260059] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:58:04.260063] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:58:04.261504] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E10-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.729081435203552246e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=864, Throttle Trigger(MB)=518, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.185090256740354420e+01) [2024-10-16 14:58:04.261612] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E10-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061884260922000, start_id=1729061884260922000, end_id=1729061884260922001) [2024-10-16 14:58:04.264647] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1323, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:04.282322] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.284026] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9B3769ABA0-0-0] [lt=8][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:04.285057] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:04.286076] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=11] ls blacklist refresh finish(cost_time=7567) [2024-10-16 14:58:04.295355] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DE9-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510352163748, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410647, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:04.298335] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.301289] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1E05-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061884300299000, start_id=1729061884300299000, end_id=1729061884300299001) [2024-10-16 14:58:04.302387] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85161-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:04.302645] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:04.302696] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4591-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:04.311983] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=7] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:58:04.312734] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985E1-0-0] [lt=13] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=192) [2024-10-16 14:58:04.312861] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061884311938000, start_id=1729061884311938000, end_id=1729061884311938001) [2024-10-16 14:58:04.313995] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:04.314011] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:04.314030] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373729142, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:04.327644] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=24] BlockGCTimerTask success(ret=0, cost_time_us=8, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):114, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:58:04.330127] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=968, mem_used=2079744) [2024-10-16 14:58:04.331610] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] no need to update min/max version(new_version={val:1729061883930736000, v:0}, min_version={val:1729061883930736000, v:0}, max_version={val:1729061883930736000, v:0}) [2024-10-16 14:58:04.334153] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA896-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:04.334183] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA896-0-0] [lt=20] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061884161808001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061884161808001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:04.335078] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B27-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061884334053000, start_id=1729061884334053000, end_id=1729061884334053001) [2024-10-16 14:58:04.335172] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B27-0-0] [lt=36] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9122, avg_group_log_size=651) [2024-10-16 14:58:04.335182] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B27-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9328) [2024-10-16 14:58:04.335384] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA896-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061884335299000, start_id=1729061884335299000, end_id=1729061884335299001) [2024-10-16 14:58:04.335556] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=221, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:04.335572] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9122) [2024-10-16 14:58:04.335584] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=48, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:04.335830] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9653-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9328) [2024-10-16 14:58:04.336076] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=17] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:04.336184] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=3, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:04.336195] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=534, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:04.336200] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=330, avg_log_submit_to_slide_time=504) [2024-10-16 14:58:04.336207] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:04.339323] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA896-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061884338527000, start_id=1729061884338527000, end_id=1729061884338527001) [2024-10-16 14:58:04.340111] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:04.341185] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4B7-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=732, avg_log_submit_to_slide_time=1153) [2024-10-16 14:58:04.343619] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:58:04.343641] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=17] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=12582912, ctx.total_memstore_used_=46137344, ctx.total_memstore_hold_=46137344, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:58:04.343658] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:04.351378] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21657][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE89E-0-0] [lt=8][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510352219772, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410648, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:04.354663] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=31] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=211, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:04.354683] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:04.354706] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:04.355023] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4593-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=23, avg_log_submit_to_flush_time=262, avg_log_submit_to_slide_time=585) [2024-10-16 14:58:04.358454] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061884356816000, start_id=1729061884356816000, end_id=1729061884356816001) [2024-10-16 14:58:04.365304] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:58:04.372557] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061884372387000, start_id=1729061884372387000, end_id=1729061884372387001) [2024-10-16 14:58:04.377174] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061884376641000, start_id=1729061884376641000, end_id=1729061884376641001) [2024-10-16 14:58:04.378557] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:58:04.378573] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54641860, tenant_id=1003) [2024-10-16 14:58:04.378592] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=15] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:04.379949] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7929-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:04.380131] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:04.382384] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.385750] INFO [SHARE] adaptive_update_limit (ob_share_throttle_define.cpp:112) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789E0-0-0] [lt=9] adaptive update(Tenant ID=1004, Config Specify Resource Limit(MB)=511, TxShare Current Memory Limit(MB)=157, Holding Memory(MB)=46, Tenant Remain Memory(MB)=183, Usable Remain Memory(MB)=110, Last Update Limit Timestamp=1729061884385098, Is Updated=true) [2024-10-16 14:58:04.385851] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789E0-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061884385098000, start_id=1729061884385098000, end_id=1729061884385098001) [2024-10-16 14:58:04.386152] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=46268317, this={dio_aligned_buf:{buf_write_offset:448, buf_padding_size:3648, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33202036, truncate_used_ts:11502188}, log_block_size:67108864, dir_fd:195, io_fd:131}, aligned_buf_len=8192, aligned_block_offset=46264320, buf_len=547, total_write_size_=15544, total_write_size_after_dio_=430080, ob_pwrite_used_ts=28707, count=101, each_pwrite_cost=284) [2024-10-16 14:58:04.388990] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:04.389009] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=93, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:04.389036] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:04.393414] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3504) [2024-10-16 14:58:04.395677] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:04.398394] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.401155] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=5] Garbage Collector is running(seq_=656751, gc_interval=10000000) [2024-10-16 14:58:04.401189] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=17] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:04.401204] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=9] gc_check_member_list_ cost time(ret=0, time_us=32) [2024-10-16 14:58:04.401213] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:58:04.401218] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:04.401229] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:04.401237] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=8] execute_gc cost time(ret=0, time_us=9) [2024-10-16 14:58:04.401241] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:58:04.405241] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DB-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061884396375, keepalive_ts:-1}) [2024-10-16 14:58:04.405262] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DB-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061884396375, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:04.407457] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183801901, max_lsn:114264966635, max_scn:{val:1729061884376641000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45336899169}, reuse_lsn:{lsn:114264966635}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114223023595}}, last_submit_lsn:{lsn:114264966042}, last_submit_end_lsn:{lsn:114264966635}, last_submit_log_id:183801901, last_submit_log_pid:33576, max_flushed_lsn:{lsn:114264966042}, max_flushed_end_lsn:{lsn:114264966635}, max_flushed_log_pid:33576, committed_end_lsn:{lsn:114264966635}, last_slide_log_id:183801901, last_slide_scn:{val:1729061884376641000, v:0}, last_slide_lsn:{lsn:114264966042}, last_slide_end_lsn:{lsn:114264966635}, last_slide_log_pid:33576, last_slide_log_accum_checksum:1555146974, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e2efe2530}, [StateMgr]={this:0x2b0e2efe6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, leader:"10.0.4.7:2890", leader_epoch:2, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183801902, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261840845, palf_role_change_cb:0x2b0e2efed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, reconfig_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, persistent_config_version:{proposal_id:33576, config_seq:9}, ms_ack_list:0, resend_config_version:{proposal_id:33576, config_seq:9}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494259837171, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494259837171, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e2efe3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262041271, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729031774558563002, v:0}, min_block_id:1697, min_block_min_scn:{val:1729025370959302001, v:0}, base_lsn_for_block_gc:{lsn:114212315136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, log_config_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, log_snapshot_meta:{version:1, base_lsn:{lsn:114212315136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:232079360}, readable_log_tail:{lsn:232079360}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114279419904}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/meta", dir_fd:187, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:36339712, block_header_serialize_buf_:0x2b0e2efe89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114264966635}, readable_log_tail:{lsn:114264966635}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114212315136}, min_scn:{val:1729057087482360001, v:0}, curr_block_id:1702, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/log", dir_fd:190, min_block_id:1697, max_block_id:1703, curr_writable_block_id:1702}, logical_block_size_:67104768, curr_block_writable_size_:14453269, block_header_serialize_buf_:0x2b0e2efea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261839896, this:0x2b0e2efe6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33576, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33575, majority_max_lsn:{lsn:45338084276}, saved_end_lsn:{lsn:45338084276}, last_submit_prepare_req_time_us:1722494261840968, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e2efe69b0}) [2024-10-16 14:58:04.407618] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=157] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061884377481, last_flushed_end_lsn:{lsn:114264966635}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061884378049, last_flushed_end_lsn:{lsn:114264966635}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061884377720, last_flushed_end_lsn:{lsn:114264966635}}]) [2024-10-16 14:58:04.407635] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=13] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183801902, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061884377481, last_flushed_end_lsn:{lsn:114264966635}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061884378049, last_flushed_end_lsn:{lsn:114264966635}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061884377720, last_flushed_end_lsn:{lsn:114264966635}}]) [2024-10-16 14:58:04.416264] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C11EB-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061884415972000, start_id=1729061884415972000, end_id=1729061884415972001) [2024-10-16 14:58:04.432547] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061884430733000, start_id=1729061884430733000, end_id=1729061884430733001) [2024-10-16 14:58:04.433783] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061884433769000, start_id=1729061884433769000, end_id=1729061884433769001) [2024-10-16 14:58:04.441096] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:04.450026] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=11][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:04.454504] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=2322, avg_group_log_size=193) [2024-10-16 14:58:04.454538] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2322) [2024-10-16 14:58:04.454715] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2322) [2024-10-16 14:58:04.454785] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:04.457369] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:04.480024] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE792B-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:04.480261] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=174, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:04.480275] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:04.482418] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.485352] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1202683904, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:04.488857] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E17-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:04.491539] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A96-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:04.491554] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A96-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:04.492803] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601494, offset=9651120}, end_cursor:ObLogCursor{file_id=192, log_id=47601495, offset=9651404}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73e2050, len:3152, log_data_len:284, seq:47601494, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:04.494672] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4BA-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061884492641000, start_id=1729061884492641000, end_id=1729061884492641001) [2024-10-16 14:58:04.495589] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601495, offset=9651404}, end_cursor:ObLogCursor{file_id=192, log_id=47601496, offset=9651678}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73f6050, len:2868, log_data_len:274, seq:47601495, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:04.496065] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [21604][T1_SSTableGC][T1][YB4A0A000407-00061E99776CB72A-0-0] [lt=9] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60476}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015268507128011}, old_min_reserved_snapshot=1729060074438469000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060074438469000) [2024-10-16 14:58:04.498450] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.508091] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107570][][T1001][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1001, ret=0, thread_count_=172) [2024-10-16 14:58:04.508110] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107570][][T1001][Y0-0000000000000000-0-0] [lt=18] Init thread local success [2024-10-16 14:58:04.508136] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107570][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59B4-0-0] [lt=9] init point thread id with(&point=0x55e3435cbb00, idx_=1074, point=[thread id=107570, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107570) [2024-10-16 14:58:04.508282] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] worker thread created(tenant_->id()=1001, group_id_=1, token=1) [2024-10-16 14:58:04.510645] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599094, miss_cnt:9, cache_read_size:149777941183, hit_ratio:"9.999999345925968397e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:04.510691] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=9] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=319, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:04.510699] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=8] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:04.512642] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C8A-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061884511847000, start_id=1729061884511847000, end_id=1729061884511847001) [2024-10-16 14:58:04.515708] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [21604][T1_SSTableGC][T1][YB4A0A000407-00061E99776CB72A-0-0] [lt=15] SSTableGCTask(cost_ts=19835) [2024-10-16 14:58:04.515944] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=23] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601496, offset=9651678}, end_cursor:ObLogCursor{file_id=192, log_id=47601497, offset=9651952}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2124050, len:2594, log_data_len:274, seq:47601496, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:04.516212] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601497, offset=9651952}, end_cursor:ObLogCursor{file_id=192, log_id=47601498, offset=9652226}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2088050, len:2320, log_data_len:274, seq:47601497, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:04.516277] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561509, miss_cnt:1, cache_read_size:152590711033, hit_ratio:"9.999999928347006461e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:04.534400] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:04.534418] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:58:04.534433] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444657596, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:04.541147] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:04.541515] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=24, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:58:04.541526] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=765, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:58:04.541535] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=8] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:58:04.541552] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=18] [APPLY STAT CB EXECUTE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=-595, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:58:04.541556] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=194, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:58:04.547093] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257842, offset=39055611}, end_cursor:ObLogCursor{file_id=197, log_id=49257843, offset=39055894}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db732a050, len:3845, log_data_len:283, seq:49257842, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:04.549230] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB385-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061884549192000, start_id=1729061884549192000, end_id=1729061884549192001) [2024-10-16 14:58:04.550063] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=23] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257843, offset=39055894}, end_cursor:ObLogCursor{file_id=197, log_id=49257844, offset=39056176}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2154050, len:3562, log_data_len:282, seq:49257843, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:04.552097] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257844, offset=39056176}, end_cursor:ObLogCursor{file_id=197, log_id=49257845, offset=39056458}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db734a050, len:3280, log_data_len:282, seq:49257844, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:04.552231] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257845, offset=39056458}, end_cursor:ObLogCursor{file_id=197, log_id=49257846, offset=39056740}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2090050, len:2998, log_data_len:282, seq:49257845, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:04.552291] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650442, miss_cnt:32, cache_read_size:8050445329, hit_ratio:"9.999994892297223714e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:04.552316] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657181, miss_cnt:17, cache_read_size:8050192691, hit_ratio:"9.999997286824092724e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:04.554553] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:04.554570] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:04.554605] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:04.554613] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:04.554778] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2200) [2024-10-16 14:58:04.555047] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4598-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:04.555073] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4598-0-0] [lt=25] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=604, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:04.558548] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061884557589000, start_id=1729061884557589000, end_id=1729061884557589001) [2024-10-16 14:58:04.565384] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:58:04.568558] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061884568323000, start_id=1729061884568323000, end_id=1729061884568323001) [2024-10-16 14:58:04.568584] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=501, get_gts_with_stc_cnt=74, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=74, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:04.568764] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=513, get_gts_with_stc_cnt=8, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:04.568778] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061884568756], gts:1729061884568323000, latest_srr:[mts=1729061884568756]}) [2024-10-16 14:58:04.569466] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.569468] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.569483] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.569615] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.572743] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061884572536000, start_id=1729061884572536000, end_id=1729061884572536001) [2024-10-16 14:58:04.576497] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] no need to update min/max version(new_version={val:1729061884176591000, v:0}, min_version={val:1729061884176591000, v:0}, max_version={val:1729061884176591000, v:0}) [2024-10-16 14:58:04.580450] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D557A-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:04.580462] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D557A-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=554, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:04.580462] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:04.582479] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.589070] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E19-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:04.589732] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92F0-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:04.589743] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92F0-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9552, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:04.597359] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:04.597373] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:04.598163] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1002}, {id:1003}]) [2024-10-16 14:58:04.598508] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.602297] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85167-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:04.602483] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=205, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:04.602502] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:04.602539] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:04.602813] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C459A-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:04.602821] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C459A-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=653, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:04.608444] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B31-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061884607633000, start_id=1729061884607633000, end_id=1729061884607633001) [2024-10-16 14:58:04.608767] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23602411, this={dio_aligned_buf:{buf_write_offset:1805, buf_padding_size:2291, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:47004177, truncate_used_ts:19010344}, log_block_size:67108864, dir_fd:134, io_fd:183}, aligned_buf_len=4096, aligned_block_offset=23601152, buf_len=546, total_write_size_=39711, total_write_size_after_dio_=491520, ob_pwrite_used_ts=32801, count=110, each_pwrite_cost=298) [2024-10-16 14:58:04.609015] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DE6-0-0] [lt=14] [APPLY STAT CB APPEND COST TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=47, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:58:04.609025] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DE6-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=986, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:58:04.609029] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DE6-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:58:04.609032] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DE6-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=-686, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:58:04.609035] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DE6-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=46, stat_interval=5000000, avg (count/cost)=347, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:58:04.611511] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5810, mem_used=10398720) [2024-10-16 14:58:04.612297] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1E04-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061884610835000, start_id=1729061884610835000, end_id=1729061884610835001) [2024-10-16 14:58:04.612642] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=52658081, this={dio_aligned_buf:{buf_write_offset:454, buf_padding_size:3642, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80311247, truncate_used_ts:29009589}, log_block_size:67108864, dir_fd:190, io_fd:179}, aligned_buf_len=8192, aligned_block_offset=52654080, buf_len=549, total_write_size_=102153, total_write_size_after_dio_=765952, ob_pwrite_used_ts=42320, count=162, each_pwrite_cost=261) [2024-10-16 14:58:04.634560] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061884633168000, start_id=1729061884633168000, end_id=1729061884633168001) [2024-10-16 14:58:04.634640] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061884633168000, start_id=1729061884633168000, end_id=1729061884633168001) [2024-10-16 14:58:04.635749] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061884161808002, v:0}, new_max_version={val:1729061884161808002, v:0}, last_min_version={val:1729061883662065004, v:0}, last_max_version={val:1729061883662065004, v:0}, total_time_us=1931, delta_us=473939) [2024-10-16 14:58:04.635768] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061884161808002, v:0}, min_version={val:1729061884161808002, v:0}, max_version={val:1729061884161808002, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:58:04.635786] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=246, start_service_tstamp_=1722494263598349, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:04.654860] INFO [ARCHIVE] do_thread_task_ (ob_archive_fetcher.cpp:310) [22243][T1002_ArcFetche][T1002][YB4A0A000407-00061E996F5960C1-0-0] [lt=14] ObArchiveFetcher is running(thread_index=0) [2024-10-16 14:58:04.662647] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BAB0-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061884662106000, start_id=1729061884662106000, end_id=1729061884662106001) [2024-10-16 14:58:04.662663] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BAB3-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061884662106000, start_id=1729061884662106000, end_id=1729061884662106001) [2024-10-16 14:58:04.662813] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAB4-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061884662106000, start_id=1729061884662106000, end_id=1729061884662106001) [2024-10-16 14:58:04.662960] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BAB2-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061884662106000, start_id=1729061884662106000, end_id=1729061884662106001) [2024-10-16 14:58:04.662994] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BAB1-0-0] [lt=38] get number(ret=0, service_type_=0, range=1, base_id=1729061884662106000, start_id=1729061884662106000, end_id=1729061884662106001) [2024-10-16 14:58:04.663148] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26170][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D3B-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061884662106000, start_id=1729061884662106001, end_id=1729061884662106002) [2024-10-16 14:58:04.669170] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [21736][T1001_COMMONLSS][T1001][YB4A0A000407-00061E9B3A29CB11-0-0] [lt=25][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:58:04.676512] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:04.677407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061884676340000, start_id=1729061884676340000, end_id=1729061884676340001) [2024-10-16 14:58:04.682540] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.685637] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:04.689875] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92F2-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=238, avg_log_submit_to_slide_time=527) [2024-10-16 14:58:04.695328] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22398][T1003_TntShared][T1003][Y0-0000000000000000-0-0] [lt=14] clear dtl interm result cost(us)(clear_cost=16, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:58:04.698571] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.700254] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:04.707609] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31896][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB1FCC6DD-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:58:04.707851] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:58:04.708582] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061884707067000, start_id=1729061884707067000, end_id=1729061884707067001) [2024-10-16 14:58:04.716244] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:04.716267] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=14] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=217) [2024-10-16 14:58:04.727939] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B24-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061884727461000, start_id=1729061884727461000, end_id=1729061884727461001) [2024-10-16 14:58:04.728287] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=42763952, this={dio_aligned_buf:{buf_write_offset:2260, buf_padding_size:1836, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35133358, truncate_used_ts:12254380}, log_block_size:67108864, dir_fd:169, io_fd:144}, aligned_buf_len=4096, aligned_block_offset=42762240, buf_len=548, total_write_size_=15549, total_write_size_after_dio_=430080, ob_pwrite_used_ts=27921, count=101, each_pwrite_cost=276) [2024-10-16 14:58:04.729447] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:04.732244] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=44] success to update min reserved snapshot(reserved_snapshot=1729060081847501000, duration=1800, snapshot_gc_ts=1729061881847501000) [2024-10-16 14:58:04.732798] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=14] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060081847501000) [2024-10-16 14:58:04.732862] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1201, avg_group_log_size=120) [2024-10-16 14:58:04.732877] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:58:04.733123] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23182447, this={dio_aligned_buf:{buf_write_offset:3295, buf_padding_size:801, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33385079, truncate_used_ts:14052932}, log_block_size:67108864, dir_fd:200, io_fd:174}, aligned_buf_len=4096, aligned_block_offset=23179264, buf_len=112, total_write_size_=11928, total_write_size_after_dio_=413696, ob_pwrite_used_ts=21258, count=98, each_pwrite_cost=216) [2024-10-16 14:58:04.733363] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=7] submit reserved snapshot log success(ls_id={id:1001}, new_snapshot_version=1729060081847501000) [2024-10-16 14:58:04.733374] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=7] update reserved snapshot finished(cost_ts=1134, reserved_snapshot=1729060081847501000) [2024-10-16 14:58:04.733937] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15F0-0-0] [lt=13] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:04.748198] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DA-0-0] [lt=39] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:04.748224] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DA-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061884662106000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061884662106000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:04.751414] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=33] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=226, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:04.751425] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10317) [2024-10-16 14:58:04.751442] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=51, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:04.751459] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10430) [2024-10-16 14:58:04.751472] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:04.751772] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2B9-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:04.751780] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2B9-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=547, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:04.751784] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2B9-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=298, avg_log_submit_to_slide_time=515) [2024-10-16 14:58:04.754910] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:04.755568] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DA-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061884754859000, start_id=1729061884754859000, end_id=1729061884754859001) [2024-10-16 14:58:04.756091] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:04.758565] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061884758054000, start_id=1729061884758054000, end_id=1729061884758054001) [2024-10-16 14:58:04.759400] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:04.759423] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=15] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:04.759433] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:04.765465] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:58:04.772775] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061884772750000, start_id=1729061884772750000, end_id=1729061884772750001) [2024-10-16 14:58:04.773053] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:04.780868] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=14] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:58:04.782608] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.782787] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:04.784390] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=25] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:04.788622] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.788640] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.788637] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.788649] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.788658] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.788658] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.788665] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.788671] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.788683] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.788690] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.789141] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:04.798630] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.800943] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C67-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:04.800965] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C67-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061884663195000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061884663195000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:04.801949] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C67-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061884801050000, start_id=1729061884801050000, end_id=1729061884801050001) [2024-10-16 14:58:04.804286] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C67-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061884804174000, start_id=1729061884804174000, end_id=1729061884804174001) [2024-10-16 14:58:04.828359] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=41] success to update min reserved snapshot(reserved_snapshot=1729060077257554000, duration=1800, snapshot_gc_ts=1729061877257554000) [2024-10-16 14:58:04.829159] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=17] update reserved snapshot finished(cost_ts=803, reserved_snapshot=1729060077257554000) [2024-10-16 14:58:04.833638] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1003}, {id:1001}, {id:1}]) [2024-10-16 14:58:04.833835] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061884633168000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=200662, in_cluster_service=true, cluster_version={val:1729061884430733000, v:0}, min_cluster_version={val:1729061884430733000, v:0}, max_cluster_version={val:1729061884430733000, v:0}, get_cluster_version_err=0, cluster_version_delta=403097, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65249128, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=382106545, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:04.834749] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061884834031000, start_id=1729061884834031000, end_id=1729061884834031001) [2024-10-16 14:58:04.835450] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061884835109000, start_id=1729061884835109000, end_id=1729061884835109001) [2024-10-16 14:58:04.841016] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2066, avg_group_log_size=206) [2024-10-16 14:58:04.841035] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2066) [2024-10-16 14:58:04.841302] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2066) [2024-10-16 14:58:04.854950] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=33] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=301, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:04.854976] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=49, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:04.854994] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:04.855352] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F43C1-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=378, avg_log_submit_to_slide_time=629) [2024-10-16 14:58:04.859255] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21532][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=37] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1404, free_obj_hold_cnt:42, allocator used:5564208, allocator total:5952128}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:292, free_obj_hold_cnt:108, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=32258032, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:41, free_obj_hold_cnt:72, allocator used:231424, allocator total:261632}, pending_cnt=0, wait_gc_count=0, tablet count=1696) [2024-10-16 14:58:04.862134] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=22] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6319, mem_used=12478464) [2024-10-16 14:58:04.864523] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53F9E-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:04.864765] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=59091348, this={dio_aligned_buf:{buf_write_offset:2574, buf_padding_size:1522, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:38334945, truncate_used_ts:14111050}, log_block_size:67108864, dir_fd:207, io_fd:202}, aligned_buf_len=4096, aligned_block_offset=59088896, buf_len=122, total_write_size_=12172, total_write_size_after_dio_=421888, ob_pwrite_used_ts=35503, count=100, each_pwrite_cost=355) [2024-10-16 14:58:04.864786] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=262, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:04.864796] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:04.864834] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:04.864987] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4C8-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:58:04.866250] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=10][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:04.877612] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061884876493000, start_id=1729061884876493000, end_id=1729061884876493001) [2024-10-16 14:58:04.882621] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.885905] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1200586752, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:04.889712] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:04.889767] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=51, stat_interval=1000000, avg (count/cost)=32, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:04.889778] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=51, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:04.898693] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.904834] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:04.904851] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27628320, tenant_id=1004) [2024-10-16 14:58:04.904861] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:04.917725] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=11] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:04.918338] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:04.918385] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061877257554000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:58:04.926391] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=10] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:04.926463] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:04.929873] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=12] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:04.930176] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234C-0-0] [lt=33] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:58:04.930843] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234C-0-0] [lt=13] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1004) [2024-10-16 14:58:04.930858] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234C-0-0] [lt=13] no transfer partition job [2024-10-16 14:58:04.930864] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234C-0-0] [lt=4] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:58:04.931830] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21882][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=16][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:04.931879] WDIAG read_multi_row (ob_table_access_helper.h:238) [21882][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=13][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:04.932095] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234C-0-0] [lt=6] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1004, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:58:04.932114] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234C-0-0] [lt=13] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:58:04.932592] INFO delete_row (ob_table_access_helper.h:312) [21882][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1001, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061884931885) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061884931885) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:58:04.933362] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234C-0-0] [lt=12] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1002, status:0}], ls_array_=[{tenant_id:1004, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729059563578310, last_statistic_bg_stat_time=1725269669481368, last_statistic_schema_version=1725266205820024, last_statistic_max_transfer_task_id={id:5}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:58:04.941097] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:04.941112] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:04.942818] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4C9-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:04.942829] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4C9-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1016, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:04.945749] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBEF-0-0] [lt=10] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061884911157, keepalive_ts:-1}) [2024-10-16 14:58:04.945764] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBEF-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061884911157, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:04.946160] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4100, mem_used=8318976) [2024-10-16 14:58:04.958831] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061884958365000, start_id=1729061884958365000, end_id=1729061884958365001) [2024-10-16 14:58:04.962564] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=2][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:04.962625] WDIAG read_multi_row (ob_table_access_helper.h:238) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:04.963364] INFO delete_row (ob_table_access_helper.h:312) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1004, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061884962632) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061884962632) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:58:04.963435] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=27] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:58:04.963484] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=9] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:58:04.963535] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22781][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:58:04.965540] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:58:04.972906] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061884972512000, start_id=1729061884972512000, end_id=1729061884972512001) [2024-10-16 14:58:04.980281] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7933-0-0] [lt=8][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938505, log_task={header:{begin_lsn:{lsn:213058271295}, end_lsn:{lsn:213058271417}, log_id:220938505, min_scn:{val:1729061884703908000, v:0}, max_scn:{val:1729061884703908000, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058271173}, prev_proposal_id:88, committed_end_lsn:{lsn:213058271295}, data_checksum:1715074492, accum_checksum:2925336965, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061884880213, freeze_ts:1729061884880213, submit_ts:1729061884880217, flushed_ts:1729061884880993, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:776}, fs_cb_begin_ts=1729061884980275, log_life_time=100062) [2024-10-16 14:58:04.982701] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:04.984237] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2019-0-0] [lt=17] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:04.984256] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE2019-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:04.985157] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=10] ====== tenant freeze timer task ====== [2024-10-16 14:58:04.993733] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=24, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:04.993760] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=6390, avg_group_log_size=456) [2024-10-16 14:58:04.993772] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:04.993945] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=242, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:04.993956] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=408, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:04.993970] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:04.994173] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1119-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=703, avg_log_submit_to_slide_time=901) [2024-10-16 14:58:04.998748] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.017349] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:58:05.017445] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DB0-0-0] [lt=11] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061884972512000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061895590307000, latest_log_ts:{val:1729061878965510001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:05.017506] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DB0-0-0] [lt=58] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:05.034735] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061885034206000, start_id=1729061885034206000, end_id=1729061885034206001) [2024-10-16 14:58:05.034958] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:05.035470] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061885034206000, start_id=1729061885034206000, end_id=1729061885034206001) [2024-10-16 14:58:05.035635] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:05.035648] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151155080, tenant_id=1) [2024-10-16 14:58:05.035661] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:05.041135] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061885041133, bytes: 4446823845952, bw: 0.474281 MB/s, add_ts: 1001224, add_bytes: 497928 [2024-10-16 14:58:05.043482] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] ====== check clog disk timer task ====== [2024-10-16 14:58:05.043499] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:05.043514] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=292020362, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:58:05.064823] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FA2-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:05.065060] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:05.065484] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92F8-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:05.065496] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92F8-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=553, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:05.068966] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061885068868000, start_id=1729061885068868000, end_id=1729061885068868001) [2024-10-16 14:58:05.068992] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061885068958], gts:1729061885068868000, latest_srr:[mts=1729061885068958]}) [2024-10-16 14:58:05.069003] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:05.069028] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=543, get_gts_with_stc_cnt=77, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=74, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:05.077860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061885077662000, start_id=1729061885077662000, end_id=1729061885077662001) [2024-10-16 14:58:05.077930] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:05.077945] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:05.078788] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D84-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:05.078815] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D84-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061884720826001, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061884720826001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:05.080261] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7935-0-0] [lt=25] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:58:05.080413] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D84-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061885079798000, start_id=1729061885079798000, end_id=1729061885079798001) [2024-10-16 14:58:05.080606] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:05.082746] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:05.082767] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=21] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:05.082774] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.082890] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D84-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061885081893000, start_id=1729061885081893000, end_id=1729061885081893001) [2024-10-16 14:58:05.086207] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:05.087580] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=14] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061880985057, last_first_register_time_us=1722494157071428, register_time_us=1729061880985057, curr_time_us=1729061885087578) [2024-10-16 14:58:05.087623] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061885087605, last_submit_register_req_time_us=1729061885087605, register_parent_reason=3) [2024-10-16 14:58:05.087632] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:05.091167] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:58:05.093621] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] ====== checkpoint timer task ====== [2024-10-16 14:58:05.094358] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:05.094385] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:05.094413] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=23] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:05.094523] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061897008855000, latest_log_ts:{val:1729061880424064001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:05.094568] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=43] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:05.094579] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:58:05.094587] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061880424064001, v:0}) [2024-10-16 14:58:05.094591] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:05.094594] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:05.094612] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:05.094621] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:58:05.094628] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:05.094634] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:05.098807] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.101757] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061880984971, last_first_register_time_us=1722494150998907, register_time_us=1729061880984971, curr_time_us=1729061885101756) [2024-10-16 14:58:05.101799] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061885101776, last_submit_register_req_time_us=1729061885101776, register_parent_reason=3) [2024-10-16 14:58:05.101808] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:05.105125] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=42] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:05.105750] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=12] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:05.105800] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=15] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061880208141000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:58:05.128452] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=2][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:05.128474] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:05.138205] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4657, mem_used=10398720) [2024-10-16 14:58:05.141545] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=222, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:05.141546] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:05.141566] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=95, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:05.141563] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:58:05.141576] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=106, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:05.141586] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:05.141819] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4CF-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:05.141825] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4CF-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=699, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:05.141878] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4D0-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=22, avg_log_submit_to_flush_time=327, avg_log_submit_to_slide_time=641) [2024-10-16 14:58:05.149995] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:05.158955] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061885158156000, start_id=1729061885158156000, end_id=1729061885158156001) [2024-10-16 14:58:05.163123] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BAB6-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061885162391000, start_id=1729061885162391000, end_id=1729061885162391001) [2024-10-16 14:58:05.163249] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D40-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061885162391000, start_id=1729061885162391000, end_id=1729061885162391001) [2024-10-16 14:58:05.163251] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BAB7-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061885162391000, start_id=1729061885162391000, end_id=1729061885162391001) [2024-10-16 14:58:05.163351] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAB9-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061885162391000, start_id=1729061885162391000, end_id=1729061885162391001) [2024-10-16 14:58:05.165308] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B92FA-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=312, avg_log_submit_to_slide_time=564) [2024-10-16 14:58:05.165577] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:58:05.171431] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15E8-0-0] [lt=13] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:05.172271] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15EE-0-0] [lt=6] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:05.173215] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061885172480000, start_id=1729061885172480000, end_id=1729061885172480001) [2024-10-16 14:58:05.174046] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get_max_applied_scn(scn={val:1729061885034206000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618756289}, palf_committed_end_scn_:{val:1729061885172480000, v:0}, last_check_scn_:{val:1729061885172480000, v:0}, max_applied_cb_scn_:{val:1729061885034206000, v:0}}) [2024-10-16 14:58:05.174072] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:58:05.182806] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.189477] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31906][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB20CC6DD-0-0] [lt=21] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:58:05.191275] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E23-0-0] [lt=19] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=9, avg_log_batch_cnt=0, total_group_log_size=1089, avg_group_log_size=121) [2024-10-16 14:58:05.194083] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=6390) [2024-10-16 14:58:05.194119] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=6390) [2024-10-16 14:58:05.194394] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C111B-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:05.194406] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C111B-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=714, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:05.198870] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.202645] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:05.205253] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5043-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:05.205267] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5043-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:05.234817] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061885234553000, start_id=1729061885234553000, end_id=1729061885234553001) [2024-10-16 14:58:05.235635] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061885235586000, start_id=1729061885235586000, end_id=1729061885235586001) [2024-10-16 14:58:05.235694] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=21, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:05.235708] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:05.240529] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061885240527, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007616, add_bytes: 0 [2024-10-16 14:58:05.241253] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:05.241270] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:05.241539] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2060) [2024-10-16 14:58:05.254980] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:05.254994] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:05.255498] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45AD-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:05.255511] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45AD-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=677, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:05.255547] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=10] [APPLY STAT CB APPEND COST TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=47, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:58:05.255553] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=6] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=682, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:58:05.255557] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:58:05.255560] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=-534, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:58:05.255563] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=4] [APPLY STAT CB TOTAL TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=195, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:58:05.265239] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=55] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:05.267458] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=34] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:05.272557] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=29] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.63, rt: 823.00; ) [2024-10-16 14:58:05.272574] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:05.272622] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.63, rt: 131.75; ) [2024-10-16 14:58:05.272630] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:05.272641] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:05.277028] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061885077662000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199360, in_cluster_service=true, cluster_version={val:1729061884876493000, v:0}, min_cluster_version={val:1729061884876493000, v:0}, max_cluster_version={val:1729061884876493000, v:0}, get_cluster_version_err=0, cluster_version_delta=400529, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244445, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=494346825, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:05.277824] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22455][T1003_TenantWea][T1003][YB4A0A000407-00061E9B3ABCFE14-0-0] [lt=28] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.248164176940917969e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=312, Throttle Trigger(MB)=187, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.830590983287007475e+03) [2024-10-16 14:58:05.277985] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061885276794000, start_id=1729061885276794000, end_id=1729061885276794001) [2024-10-16 14:58:05.282864] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.285517] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=13] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:58:05.285535] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=17] try gc retain ctx [2024-10-16 14:58:05.285539] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:58:05.286145] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061895590307000, latest_log_ts:{val:1729061878965510001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:05.286631] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:05.298931] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.298998] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DEB-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510353167391, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410649, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:05.302654] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85175-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:05.303093] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5588-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:05.321119] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1042-0-0] [lt=9] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:58:05.330237] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1041, mem_used=2079744) [2024-10-16 14:58:05.336185] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=8] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:05.342029] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4D6-0-0] [lt=18] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=305, avg_log_submit_to_slide_time=669) [2024-10-16 14:58:05.345716] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA897-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:05.345748] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA897-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061885162391001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061885162391001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:05.346804] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA897-0-0] [lt=40] get number(ret=0, service_type_=0, range=1, base_id=1729061885345966000, start_id=1729061885345966000, end_id=1729061885345966001) [2024-10-16 14:58:05.349640] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21922][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA897-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061885349248000, start_id=1729061885349248000, end_id=1729061885349248001) [2024-10-16 14:58:05.349710] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21922][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA897-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=10195, avg_group_log_size=637) [2024-10-16 14:58:05.349724] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21922][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA897-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9989) [2024-10-16 14:58:05.350043] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=221, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:05.350058] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10195) [2024-10-16 14:58:05.350069] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=42, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:05.350087] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9989) [2024-10-16 14:58:05.350101] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=10, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:05.350448] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59BF-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:05.350457] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59BF-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=1567, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:05.350467] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59BF-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=1307, avg_log_submit_to_slide_time=1536) [2024-10-16 14:58:05.350556] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:05.355314] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=192, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:05.355332] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=145, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:05.355347] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:05.358946] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061885358803000, start_id=1729061885358803000, end_id=1729061885358803001) [2024-10-16 14:58:05.365656] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:58:05.372382] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=48] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061885172480000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199893, in_cluster_service=true, cluster_version={val:1729061884972512000, v:0}, min_cluster_version={val:1729061884972512000, v:0}, max_cluster_version={val:1729061884972512000, v:0}, get_cluster_version_err=0, cluster_version_delta=399861, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244692, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=259768544, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:05.373211] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1729061885372501000, start_id=1729061885372501000, end_id=1729061885372501001) [2024-10-16 14:58:05.377883] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB9-0-0] [lt=17] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:58:05.377901] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21657][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1AB9-0-0] [lt=14] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=19) [2024-10-16 14:58:05.380474] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE793B-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=9, avg_log_batch_cnt=0, total_group_log_size=1089, avg_group_log_size=121) [2024-10-16 14:58:05.380667] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1089) [2024-10-16 14:58:05.381249] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:05.381271] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=219) [2024-10-16 14:58:05.382923] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.389629] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:05.389651] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:05.389682] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:05.393979] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=5844) [2024-10-16 14:58:05.395731] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:05.398807] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=18] blacklist_loop exec finished(cost_time=14, is_enabled=true, send_cnt=0) [2024-10-16 14:58:05.398993] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.402895] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1333) [2024-10-16 14:58:05.417538] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB9-0-0] [lt=15] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061885385274, keepalive_ts:-1}) [2024-10-16 14:58:05.417557] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EB9-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061885385274, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:05.417629] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EBA-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061885385327, keepalive_ts:-1}) [2024-10-16 14:58:05.417638] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EBA-0-0] [lt=8] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061885385327, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:05.432158] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C11FD-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061885431673000, start_id=1729061885431673000, end_id=1729061885431673001) [2024-10-16 14:58:05.434988] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061885433824000, start_id=1729061885433824000, end_id=1729061885433824001) [2024-10-16 14:58:05.436206] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061885435141000, start_id=1729061885435141000, end_id=1729061885435141001) [2024-10-16 14:58:05.441664] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1210) [2024-10-16 14:58:05.455088] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1646, avg_group_log_size=164) [2024-10-16 14:58:05.455103] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1646) [2024-10-16 14:58:05.455362] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1646) [2024-10-16 14:58:05.455371] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:05.455590] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F43D2-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=348, avg_log_submit_to_slide_time=620) [2024-10-16 14:58:05.456381] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=9] replay status is not follower(min_unreplayed_lsn={lsn:7668488755}, base_lsn={lsn:7582838784}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:58:05.456432] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=50] get_replay_process success(id={id:1}, replayed_log_size=85649971, unreplayed_log_size=0) [2024-10-16 14:58:05.456439] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:6198988121}, base_lsn={lsn:6106533888}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:58:05.456464] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=25] get_replay_process success(id={id:1001}, replayed_log_size=92454233, unreplayed_log_size=0) [2024-10-16 14:58:05.456470] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=8163564251, unreplayed_log_size=0) [2024-10-16 14:58:05.456474] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1003}, replayed_log_size=8111660144, unreplayed_log_size=0) [2024-10-16 14:58:05.456489] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1004, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=15691, last_replayed_log_size(MB)=15691, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:58:05.475336] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=9][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:05.478026] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061885477573000, start_id=1729061885477573000, end_id=1729061885477573001) [2024-10-16 14:58:05.480517] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE793D-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:05.480725] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=247, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:05.480747] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:05.482982] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.486909] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:05.489424] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E29-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:58:05.491120] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:58:05.491136] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:58:05.491152] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70919956, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:05.492347] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A97-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:05.492365] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A97-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:05.493561] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:05.493570] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:58:05.493580] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:58:05.493586] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:58:05.493576] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2E-0-0] [lt=7] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:05.493590] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:58:05.493592] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:58:05.493592] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2E-0-0] [lt=14] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:05.493595] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:58:05.493598] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:58:05.493607] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:58:05.493609] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:58:05.493612] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:58:05.499045] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.509307] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=6] replay status is not follower(min_unreplayed_lsn={lsn:45686897415}, base_lsn={lsn:45497032704}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:58:05.509364] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=56] get_replay_process success(id={id:1}, replayed_log_size=189864711, unreplayed_log_size=0) [2024-10-16 14:58:05.509387] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=10] dump tenant replay process(tenant_id=1001, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=181, last_replayed_log_size(MB)=181, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:58:05.513577] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] worker thread began to shrink(tenant_->id()=1001, group_id_=1, token=0) [2024-10-16 14:58:05.514546] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107570][T1001_L0_G1][T1001][Y0-0000000000000000-0-0] [lt=10] worker thread exit(tenant_->id()=1001, workers_.get_size()=2) [2024-10-16 14:58:05.514561] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107570][T1001_L0_G1][T0][Y0-0000000000000000-0-0] [lt=12] tenant thread end_run(id_=1001, ret=0, thread_count_=171) [2024-10-16 14:58:05.516360] INFO [SHARE] get_server_tenant_configs (ob_unit_getter.cpp:231) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=5] get_server_tenant_configs(unit_infos=[{unit:unit_id:1, resource_pool_id:1, unit_group_id:1, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, pool:{resource_pool_id:1, name:"sys_pool", unit_count:1, unit_config_id:1, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1, replica_type:0}}, {unit:unit_id:1001, resource_pool_id:1001, unit_group_id:1001, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1001, name:"ocp_pool", unit_count:1, unit_config_id:1001, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1002, replica_type:0}}, {unit:unit_id:1005, resource_pool_id:1003, unit_group_id:1002, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1003, name:"pool_bddb_myoceanbase1_oav", unit_count:1, unit_config_id:1003, zone_list:["myoceanbase1"], tenant_id:1004, replica_type:0}}]) [2024-10-16 14:58:05.516403] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=39] jx_debug: get tenant compatibility mode(tenant_id=1002, compat_mode=0) [2024-10-16 14:58:05.516412] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=8] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:58:05.516426] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=13] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:58:05.516441] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=14] jx_debug: get tenant compatibility mode(tenant_id=1004, compat_mode=0) [2024-10-16 14:58:05.516444] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=2] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:58:05.516450] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=6] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:58:05.517753] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601498, offset=9652226}, end_cursor:ObLogCursor{file_id=192, log_id=47601499, offset=9652510}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db739a050, len:2046, log_data_len:284, seq:47601498, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:05.517900] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:191) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=11] refresh resource plan success(tenant_id=1, plan_name=, directives=[]) [2024-10-16 14:58:05.518570] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=15] refresh resource user mapping rule(tenant_id=1, plan=, user_rules=[]) [2024-10-16 14:58:05.519248] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] refresh_resource_function_mapping_rule(tenant_id=1, plan=, rules=[]) [2024-10-16 14:58:05.519258] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] refresh resource mapping rule success(tenant_id=1, plan=) [2024-10-16 14:58:05.519473] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4BB-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061885518012000, start_id=1729061885518012000, end_id=1729061885518012001) [2024-10-16 14:58:05.519940] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=30, max_blocked_buffer_size_=9830400, max_buffer_size_=19660800) [2024-10-16 14:58:05.520005] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=18] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:05.520015] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] tenant dfc status(ret=0, get_tenant_id()=1, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=30, get_max_blocked_buffer_size()=9830400, get_max_buffer_size()=19660800, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:58:05.522068] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=7] refresh resource user mapping rule(tenant_id=1001, plan=, user_rules=[]) [2024-10-16 14:58:05.522790] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=12] refresh_resource_function_mapping_rule(tenant_id=1001, plan=, rules=[]) [2024-10-16 14:58:05.522809] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=19] refresh resource mapping rule success(tenant_id=1001, plan=) [2024-10-16 14:58:05.523476] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:58:05.523539] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=13] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:05.523549] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=8] tenant dfc status(ret=0, get_tenant_id()=1001, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:58:05.524038] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601499, offset=9652510}, end_cursor:ObLogCursor{file_id=192, log_id=47601500, offset=9652784}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2e60050, len:1762, log_data_len:274, seq:47601499, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:05.525809] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=11] refresh resource user mapping rule(tenant_id=1002, plan=, user_rules=[]) [2024-10-16 14:58:05.526522] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=11] refresh_resource_function_mapping_rule(tenant_id=1002, plan=, rules=[]) [2024-10-16 14:58:05.526531] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] refresh resource mapping rule success(tenant_id=1002, plan=) [2024-10-16 14:58:05.527191] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:58:05.527254] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=15] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:05.527262] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=7] tenant dfc status(ret=0, get_tenant_id()=1002, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=4, get_max_size_per_channel()=655360) [2024-10-16 14:58:05.530377] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=6] refresh resource user mapping rule(tenant_id=1003, plan=, user_rules=[]) [2024-10-16 14:58:05.531101] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=19] refresh_resource_function_mapping_rule(tenant_id=1003, plan=, rules=[]) [2024-10-16 14:58:05.531116] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=14] refresh resource mapping rule success(tenant_id=1003, plan=) [2024-10-16 14:58:05.531773] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=5] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:58:05.531835] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=13] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:05.531844] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=8] tenant dfc status(ret=0, get_tenant_id()=1003, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:58:05.533879] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] refresh resource user mapping rule(tenant_id=1004, plan=, user_rules=[]) [2024-10-16 14:58:05.534263] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get_max_applied_scn(scn={val:1729061885433824000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390218260}, palf_committed_end_scn_:{val:1729061885433824000, v:0}, last_check_scn_:{val:1729061885433824000, v:0}, max_applied_cb_scn_:{val:1729061885433824000, v:0}}) [2024-10-16 14:58:05.534282] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:58:05.534567] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=13] refresh_resource_function_mapping_rule(tenant_id=1004, plan=, rules=[]) [2024-10-16 14:58:05.534579] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=11] refresh resource mapping rule success(tenant_id=1004, plan=) [2024-10-16 14:58:05.535218] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:58:05.535284] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=16] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:05.535294] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=8] tenant dfc status(ret=0, get_tenant_id()=1004, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:58:05.535308] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:108) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] refresh tenant units(sys_unit_cnt=3, units=[{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, {tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, {tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}], ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:05.535339] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=28] finish set nway(tenant_id_=1, nway=3) [2024-10-16 14:58:05.535344] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=4] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1, nway=3, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}) [2024-10-16 14:58:05.535357] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=11] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=1.638399963378906250e+02, this=0x2b0ddc2dc170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=819, Throttle Trigger(MB)=491, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.948457122979588263e+01) [2024-10-16 14:58:05.535370] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=13] succ to set_memstore_threshold of memstore allocator(tenant_id=1, ret=0) [2024-10-16 14:58:05.535376] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=3] finish set nway(tenant_id_=1002, nway=1) [2024-10-16 14:58:05.535378] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=3] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1002, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:58:05.535387] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=8] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e0c298170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:58:05.535396] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] succ to set_memstore_threshold of memstore allocator(tenant_id=1002, ret=0) [2024-10-16 14:58:05.535401] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=3] finish set nway(tenant_id_=1001, nway=1) [2024-10-16 14:58:05.535407] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=5] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1001, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:58:05.535416] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=8] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0df7098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:58:05.535425] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] succ to set_memstore_threshold of memstore allocator(tenant_id=1001, ret=0) [2024-10-16 14:58:05.535438] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=11] finish set nway(tenant_id_=1004, nway=1) [2024-10-16 14:58:05.535441] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=3] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1004, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:58:05.535451] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e40098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:58:05.535459] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] succ to set_memstore_threshold of memstore allocator(tenant_id=1004, ret=0) [2024-10-16 14:58:05.535464] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=2] finish set nway(tenant_id_=1003, nway=1) [2024-10-16 14:58:05.535469] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=6] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1003, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:58:05.535479] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e2c698170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:58:05.535485] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=6] succ to set_memstore_threshold of memstore allocator(tenant_id=1003, ret=0) [2024-10-16 14:58:05.536088] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=5] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:05.536102] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=14] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1001, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:05.536108] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=5] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:05.536111] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=4] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1003, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:05.536115] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=4] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:05.536132] INFO [SERVER] cal_all_part_disk_default_percentage (ob_server_utils.cpp:301) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=3] cal_all_part_disk_default_percentage succ(data_dir="/mnt/myoceanbase/oceanbase/data//sstable", clog_dir="/mnt/myoceanbase/oceanbase/data//clog", shared_mode=true, data_disk_total_size=214429704192, data_disk_default_percentage=60, clog_disk_total_size=214429704192, clog_disk_default_percentage=30) [2024-10-16 14:58:05.536145] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:337) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=13] decide disk size finished(suggested_disk_size=32212254720, suggested_disk_percentage=0, default_disk_percentage=30, total_space=214429704192, disk_size=32212254720) [2024-10-16 14:58:05.536153] INFO [SERVER] get_log_disk_info_in_config (ob_server_utils.cpp:88) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=7] get_log_disk_info_in_config(suggested_data_disk_size=32212254720, suggested_clog_disk_size=32212254720, suggested_data_disk_percentage=0, suggested_clog_disk_percentage=0, log_disk_size=32212254720, log_disk_percentage=0, total_log_disk_size=214429704192) [2024-10-16 14:58:05.536161] INFO [CLOG] try_resize (ob_server_log_block_mgr.cpp:800) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=7] try_resize success(ret=0, log_disk_size=32212254720, total_log_disk_size=214429704192, this={dir::"/mnt/myoceanbase/oceanbase/data//clog/log_pool", dir_fd:15, meta_fd:16, log_pool_meta:{curr_total_size:32212254720, next_total_size:32212254720, status:0}, min_block_id:15037, max_block_id:15335, min_log_disk_size_for_all_tenants_:15032385536, is_inited:true}) [2024-10-16 14:58:05.536170] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:132) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0E-0-0] [lt=8] refresh tenant config(tenants=[1, 1001, 1002, 1003, 1004], ret=0) [2024-10-16 14:58:05.539382] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601500, offset=9652784}, end_cursor:ObLogCursor{file_id=192, log_id=47601501, offset=9653058}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2010050, len:1488, log_data_len:274, seq:47601500, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:05.539660] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601501, offset=9653058}, end_cursor:ObLogCursor{file_id=192, log_id=47601502, offset=9653332}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21b8050, len:1214, log_data_len:274, seq:47601501, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:05.539719] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561518, miss_cnt:1, cache_read_size:152590712122, hit_ratio:"9.999999928347010902e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:05.539746] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599104, miss_cnt:9, cache_read_size:149777942403, hit_ratio:"9.999999345926016137e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:05.541692] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:05.552980] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257846, offset=39056740}, end_cursor:ObLogCursor{file_id=197, log_id=49257847, offset=39057023}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2050050, len:2716, log_data_len:283, seq:49257846, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:05.554849] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22628][T1004_TeRec][T1004][YB4A0A000407-00061E9B221FB386-0-0] [lt=21] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=0, MAX_GAP=900000000000, new_sync_scn={val:1729061884511847000, v:0}, new_replayable_scn={val:1729061884511847000, v:0}, new_readable_scn={val:1729061884511847000, v:0}, old_tenant_info={tenant_id:1004, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061883067279004, v:0}, replayable_scn:{val:1729061883067279004, v:0}, standby_scn:{val:1729061883067279004, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"ARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:58:05.555033] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB386-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061885554020000, start_id=1729061885554020000, end_id=1729061885554020001) [2024-10-16 14:58:05.555162] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:05.555175] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:05.555196] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:05.555203] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:05.555449] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1646) [2024-10-16 14:58:05.555638] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45B6-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:05.555648] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45B6-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=649, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:05.556133] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257847, offset=39057023}, end_cursor:ObLogCursor{file_id=197, log_id=49257848, offset=39057305}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20f4050, len:2433, log_data_len:282, seq:49257847, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:05.558122] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257848, offset=39057305}, end_cursor:ObLogCursor{file_id=197, log_id=49257849, offset=39057587}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2004050, len:2151, log_data_len:282, seq:49257848, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:05.558154] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061885068868002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=489282, in_cluster_service=true, cluster_version={val:1729061884662106004, v:0}, min_cluster_version={val:1729061884662106004, v:0}, max_cluster_version={val:1729061884662106004, v:0}, get_cluster_version_err=0, cluster_version_delta=896044, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228471, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=430949671, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:05.558885] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061885557154000, start_id=1729061885557154001, end_id=1729061885557154002) [2024-10-16 14:58:05.559164] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257849, offset=39057587}, end_cursor:ObLogCursor{file_id=197, log_id=49257850, offset=39057869}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73de050, len:1869, log_data_len:282, seq:49257849, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:05.559218] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650451, miss_cnt:32, cache_read_size:8050446418, hit_ratio:"9.999994892297957572e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:05.559242] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657191, miss_cnt:17, cache_read_size:8050193911, hit_ratio:"9.999997286824525711e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:05.565738] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:58:05.569341] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061885568668000, start_id=1729061885568668000, end_id=1729061885568668001) [2024-10-16 14:58:05.569357] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] ts response task statistics(alloc_count=261365394, free_count=130682696) [2024-10-16 14:58:05.569456] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.569618] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.569659] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.569675] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.573260] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061885572893000, start_id=1729061885572893000, end_id=1729061885572893001) [2024-10-16 14:58:05.581053] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D558D-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:05.581070] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D558D-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=11594, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:05.581068] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1089, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:05.581101] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=207, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:05.581108] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:05.583019] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.589473] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E2B-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:05.589560] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC52-0-0] [lt=9] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:58:05.589616] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC52-0-0] [lt=20] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:05.589635] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC52-0-0] [lt=18] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:58:05.589964] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9303-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:05.589972] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9303-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=581, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:05.597754] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:05.597773] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:05.598222] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1}, {id:1003}]) [2024-10-16 14:58:05.599106] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.602845] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8517B-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:05.603054] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=193, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:05.603068] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:05.603097] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:05.603277] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D558E-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:05.603283] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D558E-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10064, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:05.611576] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5853, mem_used=10398720) [2024-10-16 14:58:05.635150] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1729061885634754000, start_id=1729061885634754000, end_id=1729061885634754001) [2024-10-16 14:58:05.635877] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061885234553001, v:0}, new_max_version={val:1729061885234553001, v:0}, last_min_version={val:1729061885234553000, v:0}, last_max_version={val:1729061885234553000, v:0}, total_time_us=1559, delta_us=401322) [2024-10-16 14:58:05.635894] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061885234553001, v:0}, min_version={val:1729061885234553001, v:0}, max_version={val:1729061885234553001, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:58:05.636587] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061885162391003, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=474192, in_cluster_service=true, cluster_version={val:1729061885068868000, v:0}, min_cluster_version={val:1729061885068868000, v:0}, max_cluster_version={val:1729061885068868000, v:0}, get_cluster_version_err=0, cluster_version_delta=567715, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244393, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=556966422, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:05.637400] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061885636891000, start_id=1729061885636891000, end_id=1729061885636891001) [2024-10-16 14:58:05.647355] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=41] success to update min reserved snapshot(reserved_snapshot=1729060080208141000, duration=1800, snapshot_gc_ts=1729061880208141000) [2024-10-16 14:58:05.648927] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=19] update reserved snapshot finished(cost_ts=1575, reserved_snapshot=1729060080208141000) [2024-10-16 14:58:05.663430] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BABA-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061885663243000, start_id=1729061885663243000, end_id=1729061885663243001) [2024-10-16 14:58:05.663638] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BABE-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061885663243000, start_id=1729061885663243000, end_id=1729061885663243001) [2024-10-16 14:58:05.663723] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BABB-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061885663243000, start_id=1729061885663243000, end_id=1729061885663243001) [2024-10-16 14:58:05.663843] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BABD-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061885663243000, start_id=1729061885663243000, end_id=1729061885663243001) [2024-10-16 14:58:05.663919] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BABC-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061885663243000, start_id=1729061885663243000, end_id=1729061885663243001) [2024-10-16 14:58:05.668980] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:05.676612] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:05.678026] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061885677128000, start_id=1729061885677128000, end_id=1729061885677128001) [2024-10-16 14:58:05.683076] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.684128] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=8] query dest_info_ succ(dest_info=[[0, "file:///mnt/oceanbasedata/bddb/archive"]], archive_dest_changed=false) [2024-10-16 14:58:05.684154] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=24] finish to resize log external storage handler(current_ts=1729061885684152, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:58:05.687199] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:05.694043] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=16] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:58:05.694056] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] try gc retain ctx [2024-10-16 14:58:05.694059] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] try to retry start_working [2024-10-16 14:58:05.694654] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061897008855000, latest_log_ts:{val:1729061880424064001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:05.699165] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.700617] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=21] gc stale ls task succ [2024-10-16 14:58:05.717587] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DC-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061885685323, keepalive_ts:-1}) [2024-10-16 14:58:05.717613] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DC-0-0] [lt=24] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061885685323, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:05.718842] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=10][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:05.729565] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:05.736618] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=246, start_service_tstamp_=1722494263598349, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:05.736814] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:05.736831] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=10] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=219) [2024-10-16 14:58:05.741569] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:58:05.741585] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:58:05.757631] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DB-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:05.757663] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DB-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061885663243000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061885663243000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:05.759217] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061885759025000, start_id=1729061885759025000, end_id=1729061885759025001) [2024-10-16 14:58:05.759277] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=10521, avg_group_log_size=526) [2024-10-16 14:58:05.759291] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10570) [2024-10-16 14:58:05.759446] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:05.759467] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=11] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:05.759474] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:05.759744] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=204, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:05.759754] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10521) [2024-10-16 14:58:05.759764] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:05.760508] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10570) [2024-10-16 14:58:05.760524] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:05.760629] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797DE1-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:05.760643] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797DE1-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=602, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:05.760647] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797DE1-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=317, avg_log_submit_to_slide_time=570) [2024-10-16 14:58:05.761790] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DB-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061885761143000, start_id=1729061885761143000, end_id=1729061885761143001) [2024-10-16 14:58:05.762274] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=11] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:05.765817] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:58:05.773112] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=20] gc stale ls task succ [2024-10-16 14:58:05.773496] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061885772777000, start_id=1729061885772777000, end_id=1729061885772777001) [2024-10-16 14:58:05.775019] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15F2-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:05.777348] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get_max_applied_scn(scn={val:1729061885677128000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114264982730}, palf_committed_end_scn_:{val:1729061885761143000, v:0}, last_check_scn_:{val:1729061885761143000, v:0}, max_applied_cb_scn_:{val:1729061885677128000, v:0}}) [2024-10-16 14:58:05.777370] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:58:05.780704] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7943-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938513, log_task={header:{begin_lsn:{lsn:213058272262}, end_lsn:{lsn:213058272384}, log_id:220938513, min_scn:{val:1729061885663243001, v:0}, max_scn:{val:1729061885663243001, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058272140}, prev_proposal_id:88, committed_end_lsn:{lsn:213058272262}, data_checksum:1715074492, accum_checksum:2508691019, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061885680632, freeze_ts:1729061885680632, submit_ts:1729061885680636, flushed_ts:1729061885681981, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1345}, fs_cb_begin_ts=1729061885780698, log_life_time=100066) [2024-10-16 14:58:05.782832] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=11] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:05.783140] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.788228] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:111799619, max_lsn:17493506449, max_scn:{val:1729061885568668001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7667642653}, reuse_lsn:{lsn:17493506449}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17451563409}}, last_submit_lsn:{lsn:17493506327}, last_submit_end_lsn:{lsn:17493506449}, last_submit_log_id:111799619, last_submit_log_pid:17, max_flushed_lsn:{lsn:17493506327}, max_flushed_end_lsn:{lsn:17493506449}, max_flushed_log_pid:17, committed_end_lsn:{lsn:17493506449}, last_slide_log_id:111799619, last_slide_scn:{val:1729061885568668001, v:0}, last_slide_lsn:{lsn:17493506327}, last_slide_end_lsn:{lsn:17493506449}, last_slide_log_pid:17, last_slide_log_accum_checksum:1115472320, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431d4530}, [StateMgr]={this:0x2b0e431d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:111799620, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261853177, palf_role_change_cb:0x2b0e431df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:18}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:18}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494261753074, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494261753074, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262053331, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728249494772393002, v:0}, min_block_id:241, min_block_min_scn:{val:1728207118648006001, v:0}, base_lsn_for_block_gc:{lsn:17380134912}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:17380134912}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:757760}, readable_log_tail:{lsn:757760}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17514344448}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/meta", dir_fd:193, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66347008, block_header_serialize_buf_:0x2b0e431da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17493506449}, readable_log_tail:{lsn:17493506449}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17447239680}, min_scn:{val:1729032503563174001, v:0}, curr_block_id:260, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/log", dir_fd:195, min_block_id:241, max_block_id:261, curr_writable_block_id:260}, logical_block_size_:67104768, curr_block_writable_size_:20837999, block_header_serialize_buf_:0x2b0e431dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261852794, this:0x2b0e431d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:7668488755}, saved_end_lsn:{lsn:7668488755}, last_submit_prepare_req_time_us:1722494261853360, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431d89b0}) [2024-10-16 14:58:05.788371] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=139] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061885741886, last_flushed_end_lsn:{lsn:17493506449}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061885742098, last_flushed_end_lsn:{lsn:17493506449}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061885742147, last_flushed_end_lsn:{lsn:17493506449}}]) [2024-10-16 14:58:05.788383] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=111799620, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061885741886, last_flushed_end_lsn:{lsn:17493506449}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061885742098, last_flushed_end_lsn:{lsn:17493506449}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061885742147, last_flushed_end_lsn:{lsn:17493506449}}]) [2024-10-16 14:58:05.788512] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.788616] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.788625] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.788633] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.788641] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.788675] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.788679] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.789061] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.789578] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.789593] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.789679] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:05.789976] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9307-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=286, avg_log_submit_to_slide_time=509) [2024-10-16 14:58:05.799225] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.804376] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79DB-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061885795613, keepalive_ts:-1}) [2024-10-16 14:58:05.804394] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79DB-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061885795613, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:05.806607] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C68-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:05.806630] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C68-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061885663243002, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061885663243002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:05.807677] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C68-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1729061885807570000, start_id=1729061885807570000, end_id=1729061885807570001) [2024-10-16 14:58:05.810016] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C68-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061885809649000, start_id=1729061885809649000, end_id=1729061885809649001) [2024-10-16 14:58:05.829678] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:05.829696] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:58:05.829708] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=52667594, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:05.833739] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1002}, {id:1001}]) [2024-10-16 14:58:05.835240] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061885834979000, start_id=1729061885834979000, end_id=1729061885834979001) [2024-10-16 14:58:05.837448] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061885837062000, start_id=1729061885837062000, end_id=1729061885837062001) [2024-10-16 14:58:05.841596] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:05.841615] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:05.841847] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:05.855648] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1454) [2024-10-16 14:58:05.855674] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=230, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:05.855682] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=43, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:05.855914] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:05.856230] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45C1-0-0] [lt=25] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=316, avg_log_submit_to_slide_time=608) [2024-10-16 14:58:05.862238] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6389, mem_used=12478464) [2024-10-16 14:58:05.865094] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FB2-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:05.865112] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FB2-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:58:05.865310] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=189, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:05.865324] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:05.865358] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:05.867358] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=41] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:05.867980] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=12] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:05.868024] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061884443293000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:58:05.878179] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=42] get number(ret=0, service_type_=0, range=1, base_id=1729061885877672000, start_id=1729061885877672000, end_id=1729061885877672001) [2024-10-16 14:58:05.883186] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.887458] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:05.890091] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:05.890129] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:05.890141] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:05.891444] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=4] ====== traversal_flush timer task ====== [2024-10-16 14:58:05.891633] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4C-0-0] [lt=11] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061885568668001, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061883408535000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900098662000, latest_log_ts:{val:1729061883567257001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:05.891773] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4C-0-0] [lt=96] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061885568668001, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061883408535000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:05.891842] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4C-0-0] [lt=34] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061885663243003, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061883408535000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:05.891911] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4C-0-0] [lt=34] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061885663243001, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061883408535000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:05.891945] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4C-0-0] [lt=32] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:58:05.899286] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.915625] INFO [COMMON] loop_running_dag_net_list (ob_tenant_dag_scheduler.cpp:3738) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] loop running dag net list(ret=0, running_dag_net_list_size=0, slow_dag_net_cnt=0, finished_dag_net_list_size=0) [2024-10-16 14:58:05.915641] INFO [COMMON] loop_finished_dag_net_list (ob_tenant_dag_scheduler.cpp:3761) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=15] loop finsihed dag net list(ret=0, finished_dag_net_list_size=0) [2024-10-16 14:58:05.916036] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=3] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:05.916649] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:05.916696] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061881847501000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:58:05.925359] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=28] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):114, sequence:1}) [2024-10-16 14:58:05.926472] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=14] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:05.926565] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:05.931593] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=12] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:05.941680] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:05.941696] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:05.942159] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=45, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:58:05.942182] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=23] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=649, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:58:05.942187] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=5] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:58:05.942191] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=-564, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:58:05.942194] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=130, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:58:05.949354] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4151, mem_used=8318976) [2024-10-16 14:58:05.949997] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=9] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:05.950017] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:05.950026] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=8] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=154) [2024-10-16 14:58:05.959246] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061885957731000, start_id=1729061885957731000, end_id=1729061885957731001) [2024-10-16 14:58:05.965897] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:05.973742] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061885972302000, start_id=1729061885972302000, end_id=1729061885972302001) [2024-10-16 14:58:05.980963] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=638324, this={dio_aligned_buf:{buf_write_offset:3566, buf_padding_size:530, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111160373, truncate_used_ts:41411005}, log_block_size:67108864, dir_fd:177, io_fd:142}, aligned_buf_len=4096, aligned_block_offset=634880, buf_len=122, total_write_size_=15508, total_write_size_after_dio_=430080, ob_pwrite_used_ts=21513, count=103, each_pwrite_cost=208) [2024-10-16 14:58:05.983237] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:05.994312] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=46] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:05.994328] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3658, avg_group_log_size=332) [2024-10-16 14:58:05.994339] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:05.994523] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=226, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:05.994534] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=164, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:05.994553] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:05.994778] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1124-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=482, avg_log_submit_to_slide_time=780) [2024-10-16 14:58:05.999337] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.008258] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=22] LogLoopThread round_cost_time(us)(round_cost_time=9) [2024-10-16 14:58:06.025431] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061886024524000, start_id=1729061886024524000, end_id=1729061886024524001) [2024-10-16 14:58:06.035369] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061886035129000, start_id=1729061886035129000, end_id=1729061886035129001) [2024-10-16 14:58:06.037526] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061886037245000, start_id=1729061886037245000, end_id=1729061886037245001) [2024-10-16 14:58:06.041887] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=20] PNIO [ratelimit] time: 1729061886041885, bytes: 4446824225774, bw: 0.361954 MB/s, add_ts: 1000752, add_bytes: 379822 [2024-10-16 14:58:06.042403] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4EB-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:06.042439] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4EB-0-0] [lt=36] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=559, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:06.065470] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:06.065800] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B930C-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:06.065812] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B930C-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=509, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:06.069713] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061886069616000, start_id=1729061886069616000, end_id=1729061886069616001) [2024-10-16 14:58:06.069736] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:06.069755] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061886069752], gts:1729061886069616000, latest_srr:[mts=1729061886069752]}) [2024-10-16 14:58:06.070585] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [21811][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:855, free_obj_hold_cnt:46, allocator used:3467048, allocator total:3728256}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:78, free_obj_hold_cnt:120, allocator used:12987018, allocator total:12987018}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=16784778, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:16, free_obj_hold_cnt:31, allocator used:96256, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:06.078289] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061886076734000, start_id=1729061886076734000, end_id=1729061886076734001) [2024-10-16 14:58:06.078357] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:06.078371] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:06.081041] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7949-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:06.081309] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:06.083249] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.085275] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D85-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:06.085308] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D85-0-0] [lt=19] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061885663243004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061885663243004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:06.087876] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:06.088419] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061881985833, last_first_register_time_us=1722494157071428, register_time_us=1729061881985833, curr_time_us=1729061886088417) [2024-10-16 14:58:06.088450] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061886088435, last_submit_register_req_time_us=1729061886088435, register_parent_reason=3) [2024-10-16 14:58:06.088458] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:58:06.091241] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Cache wash clean map node details(ret=0, clean_node_count=23, clean_time=0, clean_start_pos=1824506, clean_num=62914) [2024-10-16 14:58:06.092715] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D85-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061886092533000, start_id=1729061886092533000, end_id=1729061886092533001) [2024-10-16 14:58:06.095527] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:58:06.095719] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4A-0-0] [lt=10] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061885568668004, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061883067279004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900721265000, latest_log_ts:{val:1729061884067775001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:06.095909] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4A-0-0] [lt=99] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061885568668004, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061883067279004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:06.096063] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4A-0-0] [lt=69] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061885663243004, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061883067279004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:06.096212] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4A-0-0] [lt=70] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061885663243004, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061883067279004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:06.096344] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4A-0-0] [lt=129] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:58:06.099400] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.128506] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:06.128526] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:06.138304] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=4722, mem_used=10398720) [2024-10-16 14:58:06.139796] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [21543][T1_CdcSrv][T1][Y0-0000000000000000-0-0] [lt=13] finish to resize log external storage handler(current_ts=1729061886139794, tenant_max_cpu=3, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:58:06.142092] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=216, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:06.142091] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=233, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:06.142108] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:06.142109] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:06.142131] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:06.142528] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4ED-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=13, avg_log_submit_to_flush_time=279, avg_log_submit_to_slide_time=564) [2024-10-16 14:58:06.142590] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4EE-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:06.142605] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD4EE-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=590, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:06.150058] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=18] gc stale ls task succ [2024-10-16 14:58:06.159373] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061886159188000, start_id=1729061886159188000, end_id=1729061886159188001) [2024-10-16 14:58:06.160293] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=11] current monitor number(seq_=-1) [2024-10-16 14:58:06.164002] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D4A-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061886163476000, start_id=1729061886163476000, end_id=1729061886163476001) [2024-10-16 14:58:06.164312] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D4B-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061886163476000, start_id=1729061886163476000, end_id=1729061886163476001) [2024-10-16 14:58:06.164328] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BAC1-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061886163476000, start_id=1729061886163476000, end_id=1729061886163476001) [2024-10-16 14:58:06.164363] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BAC2-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061886163476000, start_id=1729061886163476000, end_id=1729061886163476001) [2024-10-16 14:58:06.164394] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAC3-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061886163476000, start_id=1729061886163476000, end_id=1729061886163476001) [2024-10-16 14:58:06.165172] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FB8-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:58:06.165636] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B930E-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=223, avg_log_submit_to_slide_time=492) [2024-10-16 14:58:06.167384] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=2485103, replace_num=31457) [2024-10-16 14:58:06.167398] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:58:06.170533] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:06.170551] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:06.170559] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=7] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:58:06.170566] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:58:06.170569] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:58:06.170572] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16730) [2024-10-16 14:58:06.173706] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061886172703000, start_id=1729061886172703000, end_id=1729061886172703001) [2024-10-16 14:58:06.183309] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.189878] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E37-0-0] [lt=11][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178701, log_task={header:{begin_lsn:{lsn:14538517943}, end_lsn:{lsn:14538518065}, log_id:115178701, min_scn:{val:1729061885663243005, v:0}, max_scn:{val:1729061885663243005, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538517821}, prev_proposal_id:23, committed_end_lsn:{lsn:14538517943}, data_checksum:1715074492, accum_checksum:568997477, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061886089747, freeze_ts:1729061886089747, submit_ts:1729061886089751, flushed_ts:1729061886089996, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:245}, fs_cb_begin_ts=1729061886189871, log_life_time=100124) [2024-10-16 14:58:06.190080] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1333) [2024-10-16 14:58:06.193022] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC90-0-0] [lt=9] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:58:06.193050] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC90-0-0] [lt=18] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:06.193067] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC90-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:58:06.194810] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3658) [2024-10-16 14:58:06.194838] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3658) [2024-10-16 14:58:06.195427] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1126-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:06.195445] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1126-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=783, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:06.199462] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.203696] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:06.206050] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5044-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:06.206070] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5044-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:06.235515] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061886234454000, start_id=1729061886234454000, end_id=1729061886234454001) [2024-10-16 14:58:06.236915] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1002, min_version={val:1729061885568668003, v:0}, base_version={val:1729061885568668002, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061885568668003, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061886236906, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061885568668004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061886184601, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061885568668004, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061886216320, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:06.237667] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061886236820000, start_id=1729061886236820000, end_id=1729061886236820001) [2024-10-16 14:58:06.237722] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:06.237734] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:06.241826] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:06.241847] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:06.242036] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:58:06.242060] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:06.242922] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:58:06.242939] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:58:06.242946] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:58:06.242951] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:06.245168] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=86, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}, {obj_id:78893917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:78893918, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893919, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893920, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}]) [2024-10-16 14:58:06.245287] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=115] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:06.248147] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061886248145, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007618, add_bytes: 0 [2024-10-16 14:58:06.254097] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=18] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:58:06.254971] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061886254627000, start_id=1729061886254627000, end_id=1729061886254627001) [2024-10-16 14:58:06.255660] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:06.255679] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:06.256417] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45CE-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:06.256444] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45CE-0-0] [lt=27] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=640, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:06.265878] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=43] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:06.274544] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 861.75; ) [2024-10-16 14:58:06.274568] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:06.274615] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 152.25; ) [2024-10-16 14:58:06.274632] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:06.274641] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:06.278325] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [22455][T1003_TenantWea][T1003][YB4A0A000407-00061E9B3ABCFE23-0-0] [lt=26] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.224467515945434570e+01, this=0x2b0e2c69f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=311, Throttle Trigger(MB)=186, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.858404044080484709e+03) [2024-10-16 14:58:06.278481] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061886277576000, start_id=1729061886277576000, end_id=1729061886277576001) [2024-10-16 14:58:06.283353] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.289859] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E39-0-0] [lt=28] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:58:06.292173] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:06.299523] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.302189] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DED-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510354170582, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410650, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:06.303320] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85189-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:06.303833] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45CF-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:06.313050] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=8] ====== traversal_flush timer task ====== [2024-10-16 14:58:06.313131] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAF-0-0] [lt=12] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061899029444000, latest_log_ts:{val:1729061882455772001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:06.313193] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DAF-0-0] [lt=60] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:06.314153] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:06.314174] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=15] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:06.314192] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373739601, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:06.330325] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1134, mem_used=4159488) [2024-10-16 14:58:06.336217] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=16] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:06.342455] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55D2-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10006, add_timestamp:1729061886342448}) [2024-10-16 14:58:06.342510] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5F-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10006, locations=[{table_id:10006, partition_id:10006, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886342495, sql_renew_time:1729061886342495, is_mark_fail:false}]) [2024-10-16 14:58:06.342528] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E5F-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:10006, add_timestamp:1729061886342448}, locations=[{table_id:10006, partition_id:10006, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886342495, sql_renew_time:1729061886342495, is_mark_fail:false}]) [2024-10-16 14:58:06.342644] WDIAG [SHARE] inner_to_show_str (ob_system_variable.cpp:516) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55D2-0-0] [lt=14][errcode=0] sys var casted obj is not ObVarcharType(ret=0, value={"NULL":"NULL"}, *res_obj={"NULL":"NULL"}, get_name()=secure_file_priv) [2024-10-16 14:58:06.343234] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40150-0-0] [lt=20] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=291, avg_log_submit_to_slide_time=639) [2024-10-16 14:58:06.346495] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:06.346513] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=12582912, ctx.total_memstore_used_=46137344, ctx.total_memstore_hold_=46137344, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:58:06.346529] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:06.352342] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA898-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:06.352370] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA898-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061886163476001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061886163476001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:06.353479] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA898-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061886352644000, start_id=1729061886352644000, end_id=1729061886352644001) [2024-10-16 14:58:06.355967] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=208, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:06.355983] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:06.356024] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:58:06.356289] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA898-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061886355843000, start_id=1729061886355843000, end_id=1729061886355843001) [2024-10-16 14:58:06.356348] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA898-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9874, avg_group_log_size=658) [2024-10-16 14:58:06.356357] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA898-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9874) [2024-10-16 14:58:06.356556] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=210, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:06.356565] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9874) [2024-10-16 14:58:06.356573] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=108, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:06.356585] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9874) [2024-10-16 14:58:06.356592] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:06.356911] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9672-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=3, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:06.356917] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9672-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=1167, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:06.356920] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F9672-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=403, avg_log_submit_to_slide_time=1137) [2024-10-16 14:58:06.356978] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:06.359380] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061886359117000, start_id=1729061886359117000, end_id=1729061886359117001) [2024-10-16 14:58:06.367461] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:58:06.367734] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21656][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3E05-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510354236122, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410651, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:06.373908] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=15] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:06.375027] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E0A-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061886368629000, start_id=1729061886368629000, end_id=1729061886368629001) [2024-10-16 14:58:06.375893] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061886368629000, start_id=1729061886368629001, end_id=1729061886368629002) [2024-10-16 14:58:06.377177] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [WRS] update tenant weak read server version(tenant_id=1001, server_version={version:{val:1729061886254627000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061886377156}, version_delta=122550) [2024-10-16 14:58:06.379852] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:58:06.379869] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54432140, tenant_id=1003) [2024-10-16 14:58:06.379888] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=15] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:06.381168] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE794F-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:06.381359] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:06.383414] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.390091] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=181, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:06.390108] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:06.390141] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:06.394562] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3658) [2024-10-16 14:58:06.395795] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:06.396997] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=8] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:58:06.397943] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DF0-0-0] [lt=24] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12276, add_timestamp:1729061886397936}) [2024-10-16 14:58:06.398143] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=23] new mysql sessid created(conn.sessid_=3221740115, support_ssl=false) [2024-10-16 14:58:06.398173] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=8] sm conn init succ(conn.sessid_=3221740115, sess.client_addr_="127.0.0.1:33608") [2024-10-16 14:58:06.398180] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0e2917e030, session_id:3221740115, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398231] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=18] new mysql sessid created(conn.sessid_=3221740903, support_ssl=false) [2024-10-16 14:58:06.398245] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] sm conn init succ(conn.sessid_=3221740903, sess.client_addr_="127.0.0.1:33610") [2024-10-16 14:58:06.398249] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f27406030, session_id:3221740903, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398259] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] new mysql sessid created(conn.sessid_=3221741981, support_ssl=false) [2024-10-16 14:58:06.398269] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221741981, sess.client_addr_="127.0.0.1:33614") [2024-10-16 14:58:06.398273] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] accept one succ(*s={this:0x2b0f27404030, session_id:3221741981, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:214}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398282] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221738167, support_ssl=false) [2024-10-16 14:58:06.398299] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221738167, sess.client_addr_="127.0.0.1:33618") [2024-10-16 14:58:06.398303] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f27414df0, session_id:3221738167, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:215}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398457] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E60-0-0] [lt=9] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12276, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:06.398487] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=10] new mysql sessid created(conn.sessid_=3221739422, support_ssl=false) [2024-10-16 14:58:06.398507] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=5] sm conn init succ(conn.sessid_=3221739422, sess.client_addr_="127.0.0.1:33622") [2024-10-16 14:58:06.398512] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdcc030, session_id:3221739422, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:218}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398538] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=10] new mysql sessid created(conn.sessid_=3221740353, support_ssl=false) [2024-10-16 14:58:06.398553] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] sm conn init succ(conn.sessid_=3221740353, sess.client_addr_="127.0.0.1:33612") [2024-10-16 14:58:06.398557] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] accept one succ(*s={this:0x2b0f2742c030, session_id:3221740353, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:221}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398565] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221700775, support_ssl=false) [2024-10-16 14:58:06.398576] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221700775, sess.client_addr_="127.0.0.1:33620") [2024-10-16 14:58:06.398579] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0f2742cdf0, session_id:3221700775, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:223}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398588] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221741914, support_ssl=false) [2024-10-16 14:58:06.398598] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221741914, sess.client_addr_="127.0.0.1:33624") [2024-10-16 14:58:06.398613] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=14] accept one succ(*s={this:0x2b0f2749c030, session_id:3221741914, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:224}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398630] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.398643] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=5] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.398651] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.398656] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.398663] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.398668] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21318][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.398670] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=60] new mysql sessid created(conn.sessid_=3221741916, support_ssl=false) [2024-10-16 14:58:06.398694] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=11] sm conn init succ(conn.sessid_=3221741916, sess.client_addr_="127.0.0.1:33606") [2024-10-16 14:58:06.398701] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] accept one succ(*s={this:0x2b0e5cdb8df0, session_id:3221741916, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:225}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398715] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=10] new mysql sessid created(conn.sessid_=3221739083, support_ssl=false) [2024-10-16 14:58:06.398726] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221739083, sess.client_addr_="127.0.0.1:33616") [2024-10-16 14:58:06.398731] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC2-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.398730] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdf8df0, session_id:3221739083, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:227}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398740] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3221740082, support_ssl=false) [2024-10-16 14:58:06.398750] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] sm conn init succ(conn.sessid_=3221740082, sess.client_addr_="127.0.0.1:33626") [2024-10-16 14:58:06.398753] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] accept one succ(*s={this:0x2b0e5cdfcdf0, session_id:3221740082, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:229}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398762] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221688030, support_ssl=false) [2024-10-16 14:58:06.398772] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] sm conn init succ(conn.sessid_=3221688030, sess.client_addr_="127.0.0.1:33630") [2024-10-16 14:58:06.398775] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] accept one succ(*s={this:0x2b0e5cdb6df0, session_id:3221688030, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:230}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.398953] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.398965] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=6] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.398972] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.398976] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.398981] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.398985] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.398990] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.398994] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.399214] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC2-0-0] [lt=5] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221740353, proxy_sessid=0, client_sessid=3221740353, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.399266] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC3-0-0] [lt=15] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.399384] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55D4-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12142, add_timestamp:1729061886399380}) [2024-10-16 14:58:06.399399] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55D4-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12217, add_timestamp:1729061886399399}) [2024-10-16 14:58:06.399412] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55D4-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12068, add_timestamp:1729061886399411}) [2024-10-16 14:58:06.399417] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55D4-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11035, add_timestamp:1729061886399417}) [2024-10-16 14:58:06.399423] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21655][T1_L0_G0][T1][YB4A0A000407-00061E9B3A0A55D4-0-0] [lt=3] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12072, add_timestamp:1729061886399422}) [2024-10-16 14:58:06.399641] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC3-0-0] [lt=9] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221700775, proxy_sessid=0, client_sessid=3221700775, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.399686] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC4-0-0] [lt=16] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.400040] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC4-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221741914, proxy_sessid=0, client_sessid=3221741914, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.400076] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC5-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.400416] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC5-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221688030, proxy_sessid=0, client_sessid=3221688030, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.400455] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC6-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.400797] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC6-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221740082, proxy_sessid=0, client_sessid=3221740082, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.400832] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC7-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.401164] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC7-0-0] [lt=3] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221739083, proxy_sessid=0, client_sessid=3221739083, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.401200] INFO load_privilege_info (obmp_connect.cpp:561) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC8-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.401520] INFO [SERVER] process (obmp_connect.cpp:502) [21579][T1_MysqlQueueTh][T1][Y0-00061E9B318A1DC8-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221741916, proxy_sessid=0, client_sessid=3221741916, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.401555] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.401761] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.401773] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=6] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.401781] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=4] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.401786] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21315][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.401826] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA4-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.402199] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA4-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221740115, proxy_sessid=0, client_sessid=3221740115, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.402245] INFO load_privilege_info (obmp_connect.cpp:561) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA5-0-0] [lt=15] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.402583] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA5-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221739422, proxy_sessid=0, client_sessid=3221739422, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.402657] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221738866, support_ssl=false) [2024-10-16 14:58:06.402684] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] sm conn init succ(conn.sessid_=3221738866, sess.client_addr_="127.0.0.1:33628") [2024-10-16 14:58:06.402690] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] accept one succ(*s={this:0x2b0f2749cdf0, session_id:3221738866, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:232}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.402710] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.402718] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=3] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.402726] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.402730] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.402742] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.402745] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=2] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.402817] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B8-0-0] [lt=17] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.403167] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B8-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221738167, proxy_sessid=0, client_sessid=3221738167, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.403211] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B9-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.403541] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37B9-0-0] [lt=3] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221741981, proxy_sessid=0, client_sessid=3221741981, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.403577] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37BA-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.403937] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37BA-0-0] [lt=4] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221740903, proxy_sessid=0, client_sessid=3221740903, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.404289] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E60-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12276, locations=[{table_id:12276, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404275, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404275, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404275, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.404309] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E60-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:12276, add_timestamp:1729061886397936}, locations=[{table_id:12276, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404275, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404275, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404275, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.404332] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E61-0-0] [lt=13] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12142, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404329, sql_renew_time:1729061886404329, is_mark_fail:false}]) [2024-10-16 14:58:06.404338] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E61-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12142, add_timestamp:1729061886399380}, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404329, sql_renew_time:1729061886404329, is_mark_fail:false}]) [2024-10-16 14:58:06.404348] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E62-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12217, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404346, sql_renew_time:1729061886404346, is_mark_fail:false}]) [2024-10-16 14:58:06.404354] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E62-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12217, add_timestamp:1729061886399399}, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404346, sql_renew_time:1729061886404346, is_mark_fail:false}]) [2024-10-16 14:58:06.404363] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E63-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12068, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404360, sql_renew_time:1729061886404360, is_mark_fail:false}]) [2024-10-16 14:58:06.404368] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E63-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12068, add_timestamp:1729061886399411}, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404360, sql_renew_time:1729061886404360, is_mark_fail:false}]) [2024-10-16 14:58:06.404377] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E64-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11035, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404375, sql_renew_time:1729061886404375, is_mark_fail:false}]) [2024-10-16 14:58:06.404383] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E64-0-0] [lt=6] success to process renew task(task={tenant_id:1, table_id:11035, add_timestamp:1729061886399417}, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404375, sql_renew_time:1729061886404375, is_mark_fail:false}]) [2024-10-16 14:58:06.404393] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E65-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12072, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404391, sql_renew_time:1729061886404391, is_mark_fail:false}]) [2024-10-16 14:58:06.404398] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E65-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12072, add_timestamp:1729061886399422}, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886404391, sql_renew_time:1729061886404391, is_mark_fail:false}]) [2024-10-16 14:58:06.405369] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC921-0-0] [lt=13] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1729061886405364}) [2024-10-16 14:58:06.409946] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21674][T1_L0_G0][T1][YB4A0A000407-00061E9979BD0323-0-0] [lt=17] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11021, add_timestamp:1729061886409944}) [2024-10-16 14:58:06.413367] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [21679][T1_L0_G0][T1][YB4A0A000407-00061E9B196AC921-0-0] [lt=10][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:06.414227] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E66-0-0] [lt=5] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:06.414279] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:80) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] username and tenantname(user_name=ocp_monitor, tenant_name=) [2024-10-16 14:58:06.414292] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21316][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=6] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:06.414564] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=13] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1729061886414562}) [2024-10-16 14:58:06.414583] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12358, add_timestamp:1729061886414582}) [2024-10-16 14:58:06.414590] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=4] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12396, add_timestamp:1729061886414590}) [2024-10-16 14:58:06.415553] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.415564] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.416254] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.416281] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=11] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.416290] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.416297] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.416303] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.416349] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:58:06.416353] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.416362] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.416366] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.416374] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:58:06.416377] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.416381] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.416384] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79D-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.416589] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED79E-0-0] [lt=3] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11003, add_timestamp:1729061886416586}) [2024-10-16 14:58:06.417075] INFO load_privilege_info (obmp_connect.cpp:561) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37BB-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys) [2024-10-16 14:58:06.417213] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:290) [21674][T1_L0_G0][T1][YB4A0A000407-00061E9979BD0323-0-0] [lt=11][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:06.417486] INFO [SERVER] process (obmp_connect.cpp:502) [21580][T1_MysqlQueueTh][T1][Y0-00061E9B308A37BB-0-0] [lt=9] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3221738866, proxy_sessid=0, client_sessid=3221738866, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:06.418539] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e2917e030, session_id:3221740115, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:793, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.418563] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=19] kill and revert session(conn.sessid_=3221740115, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.418569] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdcc030, session_id:3221739422, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:218}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:654, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.418577] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=7] kill and revert session(conn.sessid_=3221739422, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.418581] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0e2917e030, session_id:3221740115, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:203}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:793, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.418620] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=12] unlock session of tenant(conn.sessid_=3221740115, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.418625] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221740115, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9979BD0323-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:688, rec_:688, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:370, seq_:120}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:121}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.418648] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=22] sock ref clean, do destroy(*s={this:0x2b0e5cdcc030, session_id:3221739422, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:218}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:654, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.418657] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221739422, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.418660] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221739422, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B196AC921-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:549, rec_:549, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:25, seq_:32}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:33}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.418721] INFO [SERVER] run (obmp_disconnect.cpp:82) [21680][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] free session successfully(sessid=3221740115) [2024-10-16 14:58:06.418727] INFO [SERVER] run (obmp_disconnect.cpp:89) [21680][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=6] mark session id unused(sessid=3221740115) [2024-10-16 14:58:06.418752] INFO [SERVER] run (obmp_disconnect.cpp:82) [21680][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221739422) [2024-10-16 14:58:06.418755] INFO [SERVER] run (obmp_disconnect.cpp:89) [21680][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221739422) [2024-10-16 14:58:06.418891] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E66-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11017, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886418880, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886418880, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886418880, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.418918] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E66-0-0] [lt=26] success to process renew task(task={tenant_id:1, table_id:11017, add_timestamp:1729061886405364}, locations=[{table_id:11017, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886418880, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886418880, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886418880, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.418937] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E67-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11021, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:06.418994] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27414df0, session_id:3221738167, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:215}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:203, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.419009] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] kill and revert session(conn.sessid_=3221738167, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.419013] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27404030, session_id:3221741981, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:214}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.419020] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221741981, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.419023] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f27406030, session_id:3221740903, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:239, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.419029] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] kill and revert session(conn.sessid_=3221740903, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.419032] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0f27414df0, session_id:3221738167, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:215}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:203, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.419046] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221738167, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.419050] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221738167, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED79E-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:98, rec_:98, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.419070] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=18] sock ref clean, do destroy(*s={this:0x2b0f27404030, session_id:3221741981, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:214}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.419080] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221741981, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.419082] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=2] connection close(sessid=3221741981, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED79D-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:452, rec_:452, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.419086] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=13] free session successfully(sessid=3221738167) [2024-10-16 14:58:06.419096] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=9] mark session id unused(sessid=3221738167) [2024-10-16 14:58:06.419095] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0f27406030, session_id:3221740903, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:1729061886405127, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:239, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.419104] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221740903, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.419107] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221740903, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B274ED79C-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:134, rec_:134, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:7}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:8}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.419122] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221741981) [2024-10-16 14:58:06.419125] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221741981) [2024-10-16 14:58:06.419126] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:06.419150] INFO [SERVER] run (obmp_disconnect.cpp:82) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221740903) [2024-10-16 14:58:06.419152] INFO [SERVER] run (obmp_disconnect.cpp:89) [21682][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221740903) [2024-10-16 14:58:06.419242] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9B-0-0] [lt=7] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11008, add_timestamp:1729061886419241}) [2024-10-16 14:58:06.419245] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=2][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061886419242}) [2024-10-16 14:58:06.419259] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=14][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061886419242}) [2024-10-16 14:58:06.419263] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=4][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12358, vtable_locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403632, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.419280] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=10][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061886419280}) [2024-10-16 14:58:06.419284] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=3][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061886419280}) [2024-10-16 14:58:06.419287] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=3][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12396, vtable_locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403648, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.420027] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.420036] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.420716] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=12] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.420737] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.420744] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.420749] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.420753] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.420925] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=2] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:58:06.420929] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.420939] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.420942] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.420953] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:58:06.420956] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.420960] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.420963] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF854-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.421171] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:10008, add_timestamp:1729061886421169}) [2024-10-16 14:58:06.421179] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=8][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:10008, add_timestamp:1729061886421169}) [2024-10-16 14:58:06.421183] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=4][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=10008, vtable_locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434957, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434957, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871434957, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.421208] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=16][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061886421207}) [2024-10-16 14:58:06.421211] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061886421207}) [2024-10-16 14:58:06.421215] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12358, vtable_locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403632, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.421223] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=6][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061886421223}) [2024-10-16 14:58:06.421226] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061886421223}) [2024-10-16 14:58:06.421229] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12396, vtable_locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403648, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.421739] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=6][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061886421737}) [2024-10-16 14:58:06.421748] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=9][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061886421737}) [2024-10-16 14:58:06.421752] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=4][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12358, vtable_locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403632, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.421766] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=10][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061886421766}) [2024-10-16 14:58:06.421770] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=4][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061886421766}) [2024-10-16 14:58:06.421773] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=3][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12396, vtable_locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403648, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.422113] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.422122] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.422485] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.422494] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.422769] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.422789] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.422795] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.422800] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.422804] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.422839] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:58:06.422843] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.422851] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.422854] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.422861] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:58:06.422865] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.422869] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.422872] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF855-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.423079] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061886423077}) [2024-10-16 14:58:06.423086] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=7][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12358, add_timestamp:1729061886423077}) [2024-10-16 14:58:06.423090] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=4][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12358, vtable_locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403632, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.423102] WDIAG [SHARE.LOCATION] add_update_task (ob_vtable_location_service.cpp:388) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=10][errcode=-4023] fail to add task(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061886423102}) [2024-10-16 14:58:06.423106] WDIAG [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:444) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=4][errcode=-4023] add location update task failed(ret=-4023, ret="OB_EAGAIN", task={tenant_id:1, table_id:12396, add_timestamp:1729061886423102}) [2024-10-16 14:58:06.423109] WDIAG [SHARE.LOCATION] vtable_get (ob_vtable_location_service.cpp:156) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3][errcode=0] add location update task failed(temp_ret=-4023, temp_ret="OB_EAGAIN", tenant_id=1, table_id=12396, vtable_locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061871403648, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.423114] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.423139] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.423147] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.423154] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.423160] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.423183] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:58:06.423186] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.423194] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.423197] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.423205] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:58:06.423208] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.423212] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.423215] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9C-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.423246] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=6] get_stable_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:06.423264] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21682][T1_L0_G0][T1001][YB4A0A000407-00061E9AECBAF856-0-0] [lt=5] get_stable_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:58:06.423274] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21682][T1_L0_G0][T1002][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] get_stable_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:58:06.423281] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21682][T1_L0_G0][T1003][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] get_stable_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:58:06.423291] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:908) [21682][T1_L0_G0][T1004][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] get_stable_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:06.424046] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=2] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.424054] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"sys", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.424678] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:06.424698] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.424705] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.424709] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.424713] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=2] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.424747] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:58:06.424750] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.424758] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:06.424761] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.424771] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:58:06.424774] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.424778] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:06.424781] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21682][T1_L0_G0][T1][YB4A0A000407-00061E9AECBAF856-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:06.424979] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=18] LogLoopThread round_cost_time(us)(round_cost_time=11) [2024-10-16 14:58:06.426525] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E67-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11021, locations=[{table_id:11021, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426512, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426512, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426512, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.426546] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E67-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:11021, add_timestamp:1729061886409944}, locations=[{table_id:11021, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426512, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426512, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426512, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.426565] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E68-0-0] [lt=12] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:06.426676] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E68-0-0] [lt=6] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10008, locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426672, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426672, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426672, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.426689] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E68-0-0] [lt=13] success to process renew task(task={tenant_id:1, table_id:10008, add_timestamp:1729061886414562}, locations=[{table_id:10008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426672, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426672, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426672, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.426705] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E69-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12358, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426703, sql_renew_time:1729061886426703, is_mark_fail:false}]) [2024-10-16 14:58:06.426711] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E69-0-0] [lt=6] success to process renew task(task={tenant_id:1, table_id:12358, add_timestamp:1729061886414582}, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426703, sql_renew_time:1729061886426703, is_mark_fail:false}]) [2024-10-16 14:58:06.426720] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6A-0-0] [lt=5] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12396, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426718, sql_renew_time:1729061886426718, is_mark_fail:false}]) [2024-10-16 14:58:06.426726] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6A-0-0] [lt=5] success to process renew task(task={tenant_id:1, table_id:12396, add_timestamp:1729061886414590}, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426718, sql_renew_time:1729061886426718, is_mark_fail:false}]) [2024-10-16 14:58:06.426733] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6B-0-0] [lt=5] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11003, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:06.426807] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6B-0-0] [lt=4] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11003, locations=[{table_id:11003, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426804, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426804, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426804, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.426818] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6B-0-0] [lt=11] success to process renew task(task={tenant_id:1, table_id:11003, add_timestamp:1729061886416586}, locations=[{table_id:11003, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426804, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426804, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426804, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.426831] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6C-0-0] [lt=10] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11008, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:06.426919] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6C-0-0] [lt=3] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11008, locations=[{table_id:11008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426912, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426912, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426912, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.426931] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6C-0-0] [lt=11] success to process renew task(task={tenant_id:1, table_id:11008, add_timestamp:1729061886419241}, locations=[{table_id:11008, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426912, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426912, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061886426912, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:06.427342] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdb6df0, session_id:3221688030, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:230}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:667, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427364] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=17] kill and revert session(conn.sessid_=3221688030, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.427370] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdfcdf0, session_id:3221740082, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:229}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:738, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427377] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221740082, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.427380] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdf8df0, session_id:3221739083, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:227}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:534, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427387] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] kill and revert session(conn.sessid_=3221739083, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.427389] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdb8df0, session_id:3221741916, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:225}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:231, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427398] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=7] kill and revert session(conn.sessid_=3221741916, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.427403] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] sock ref clean, do destroy(*s={this:0x2b0e5cdb6df0, session_id:3221688030, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:230}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:667, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427422] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=13] unlock session of tenant(conn.sessid_=3221688030, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.427427] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221688030, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AECBAF856-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:562, rec_:562, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:10}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:11}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.427450] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=21] sock ref clean, do destroy(*s={this:0x2b0e5cdfcdf0, session_id:3221740082, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:229}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:738, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427458] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221740082, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.427461] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221740082, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B2D49CF9C-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:633, rec_:633, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.427474] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0e5cdf8df0, session_id:3221739083, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:227}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:534, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427482] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=4] unlock session of tenant(conn.sessid_=3221739083, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.427485] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221739083, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AECBAF855-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:429, rec_:429, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.427498] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=12] sock ref clean, do destroy(*s={this:0x2b0e5cdb8df0, session_id:3221741916, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:225}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:231, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427506] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221741916, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.427509] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=2] connection close(sessid=3221741916, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B2D49CF9B-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:126, rec_:126, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:64, seq_:70}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:71}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.427568] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=11] free session successfully(sessid=3221688030) [2024-10-16 14:58:06.427574] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221688030) [2024-10-16 14:58:06.427636] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221740082) [2024-10-16 14:58:06.427640] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221740082) [2024-10-16 14:58:06.427662] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221739083) [2024-10-16 14:58:06.427664] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] mark session id unused(sessid=3221739083) [2024-10-16 14:58:06.427683] INFO [SERVER] run (obmp_disconnect.cpp:82) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221741916) [2024-10-16 14:58:06.427685] INFO [SERVER] run (obmp_disconnect.cpp:89) [21681][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] mark session id unused(sessid=3221741916) [2024-10-16 14:58:06.427697] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2749cdf0, session_id:3221738866, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:232}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427710] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9] kill and revert session(conn.sessid_=3221738866, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.427714] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=3] sock ref clean, do destroy(*s={this:0x2b0f2749cdf0, session_id:3221738866, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:232}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:557, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.427723] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221738866, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.427726] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21316][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221738866, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AECBAF854-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:452, rec_:452, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.428493] INFO [SERVER] run (obmp_disconnect.cpp:82) [21678][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] free session successfully(sessid=3221738866) [2024-10-16 14:58:06.428501] INFO [SERVER] run (obmp_disconnect.cpp:89) [21678][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=9] mark session id unused(sessid=3221738866) [2024-10-16 14:58:06.429169] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107580][][T1002][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1002, ret=0, thread_count_=191) [2024-10-16 14:58:06.429186] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107580][][T1002][Y0-0000000000000000-0-0] [lt=16] Init thread local success [2024-10-16 14:58:06.432771] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [25332][T1_L0_G0][T1][YB4A0A000407-00061E9AA18D5EB2-0-0] [lt=15][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:06.435715] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061886435217000, start_id=1729061886435217000, end_id=1729061886435217001) [2024-10-16 14:58:06.436411] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:866) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9D-0-0] [lt=3][errcode=0] server close connection(sessid=3221741914, proxy_sessid=0, stack="0x1a5796b1 0xc7f3c2c 0xc7baa9b 0xc95cc71 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:06.436422] INFO [SERVER] process (obmp_quit.h:56) [21680][T1_L0_G0][T1][YB4A0A000407-00061E9B2D49CF9D-0-0] [lt=10] quit [2024-10-16 14:58:06.436444] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2749c030, session_id:3221741914, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:224}, err:5, last_decode_time:1729061886436277, pending_write_task:{buf:0x2b0ec00ca110, sz:0}, need_epoll_trigger_write:false, consume_size:471, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.436460] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=13] kill and revert session(conn.sessid_=3221741914, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.436465] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2742cdf0, session_id:3221700775, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:223}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:346, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.436472] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] kill and revert session(conn.sessid_=3221700775, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.436476] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:978) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] can close safely, do destroy(*s={this:0x2b0f2749c030, session_id:3221741914, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:224}, err:5, last_decode_time:1729061886436277, pending_write_task:{buf:0x2b0ec00ca110, sz:0}, need_epoll_trigger_write:false, consume_size:471, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2024-10-16 14:58:06.436488] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221741914, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.436492] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221741914, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA18D5EB1-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:8, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:361, rec_:361, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:7, seq_:9}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:1, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{receive:obp_mysql_header_{len_:1, rec_:1, seq_:0}, pkt_name:"COM_QUIT", obp_mysql_header_.type_:1, obp_mysql_header_.is_send_:0}], disconnect_state=3) [2024-10-16 14:58:06.436515] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=22] sock ref clean, do destroy(*s={this:0x2b0f2742cdf0, session_id:3221700775, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:223}, err:5, last_decode_time:1729061886418767, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:346, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.436524] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=5] unlock session of tenant(conn.sessid_=3221700775, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.436527] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221700775, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9AA18D5EB2-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:241, rec_:241, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.436583] INFO [SERVER] run (obmp_disconnect.cpp:82) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=10] free session successfully(sessid=3221741914) [2024-10-16 14:58:06.436588] INFO [SERVER] run (obmp_disconnect.cpp:89) [21679][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=4] mark session id unused(sessid=3221741914) [2024-10-16 14:58:06.436667] INFO [SERVER] run (obmp_disconnect.cpp:82) [21678][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221700775) [2024-10-16 14:58:06.436676] INFO [SERVER] run (obmp_disconnect.cpp:89) [21678][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=9] mark session id unused(sessid=3221700775) [2024-10-16 14:58:06.437745] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061886436277000, start_id=1729061886436277000, end_id=1729061886436277001) [2024-10-16 14:58:06.438706] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] get_max_applied_scn(scn={val:1729061886069616001, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959735524}, palf_committed_end_scn_:{val:1729061886069616002, v:0}, last_check_scn_:{val:1729061886069616002, v:0}, max_applied_cb_scn_:{val:1729061886069616001, v:0}}) [2024-10-16 14:58:06.438726] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:58:06.438778] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=43] get_max_applied_scn(scn={val:1729061886069616001, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023137805}, palf_committed_end_scn_:{val:1729061886069616002, v:0}, last_check_scn_:{val:1729061886069616002, v:0}, max_applied_cb_scn_:{val:1729061886069616001, v:0}}) [2024-10-16 14:58:06.438786] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:58:06.438810] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:58:06.438817] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=5] get_min_unreplayed_log_info(lsn={lsn:213058273238}, scn={val:1729061886163476004, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061886381643, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058273238}, next_to_submit_scn_:{val:1729061886163476004, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058273116}, end_lsn:{lsn:213058273238}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061886163476003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061886163476003, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:629224682, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061886163476003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561527, miss_cnt:1, cache_read_size:152590713220, hit_ratio:"9.999999928347015343e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:58:06.442990] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40154-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:06.444553] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=11] worker thread created(tenant_->id()=1002, group_id_=1, token=1) [2024-10-16 14:58:06.447297] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C120F-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061886446268000, start_id=1729061886446268000, end_id=1729061886446268001) [2024-10-16 14:58:06.458665] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:58:06.458680] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:58:06.459036] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:06.459066] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:58:06.459356] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F43F2-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=294, avg_log_submit_to_slide_time=551) [2024-10-16 14:58:06.468980] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=42] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:58:06.468992] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:58:06.468996] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:06.469583] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061899029444000, latest_log_ts:{val:1729061882455772001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:06.471650] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0f2742c030, session_id:3221740353, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:221}, err:5, last_decode_time:1729061886397501, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:224, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.471674] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=18] kill and revert session(conn.sessid_=3221740353, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:06.471680] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] sock ref clean, do destroy(*s={this:0x2b0f2742c030, session_id:3221740353, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:221}, err:5, last_decode_time:1729061886397501, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:224, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:06.471692] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:159) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] unlock session of tenant(conn.sessid_=3221740353, proxy_sessid=0, conn.tenant_id_=1) [2024-10-16 14:58:06.471698] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21318][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=4] connection close(sessid=3221740353, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB4A0A000407-00061E9B3A0A55D4-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:7, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:119, rec_:119, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:2, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:5, seq_:10}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:11}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2024-10-16 14:58:06.471781] INFO [SERVER] run (obmp_disconnect.cpp:82) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=3] free session successfully(sessid=3221740353) [2024-10-16 14:58:06.471787] INFO [SERVER] run (obmp_disconnect.cpp:89) [21655][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] mark session id unused(sessid=3221740353) [2024-10-16 14:58:06.477721] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1003, min_version={val:1729061886159188000, v:0}, base_version={val:1729061886076734000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061886277576000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061886477709, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061886159188000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061886277988, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061886277576000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061886430617, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:06.478680] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061886478394000, start_id=1729061886478394000, end_id=1729061886478394001) [2024-10-16 14:58:06.481233] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7951-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:06.481459] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=180, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:06.481477] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=127, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:06.481883] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=13] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:133533538, max_lsn:66390221918, max_scn:{val:1729061886435217000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:33216076378}, reuse_lsn:{lsn:66390221918}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:66348278878}}, last_submit_lsn:{lsn:66390221331}, last_submit_end_lsn:{lsn:66390221918}, last_submit_log_id:133533538, last_submit_log_pid:48, max_flushed_lsn:{lsn:66390221331}, max_flushed_end_lsn:{lsn:66390221918}, max_flushed_log_pid:48, committed_end_lsn:{lsn:66390221918}, last_slide_log_id:133533538, last_slide_scn:{val:1729061886435217000, v:0}, last_slide_lsn:{lsn:66390221331}, last_slide_end_lsn:{lsn:66390221918}, last_slide_log_pid:48, last_slide_log_accum_checksum:1698457918, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0de0acc530}, [StateMgr]={this:0x2b0de0ad0830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:133533539, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261740374, palf_role_change_cb:0x2b0de0ad7230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, reconfig_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, persistent_config_version:{proposal_id:48, config_seq:11}, ms_ack_list:0, resend_config_version:{proposal_id:48, config_seq:11}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494260838312, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494260838312, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0de0acdc70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261941355, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728741813401825001, v:0}, min_block_id:965, min_block_min_scn:{val:1728727703672931001, v:0}, base_lsn_for_block_gc:{lsn:66098196480}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, log_config_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, log_snapshot_meta:{version:1, base_lsn:{lsn:66098196480}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1634304}, readable_log_tail:{lsn:1634304}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66433720320}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/meta", dir_fd:128, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65470464, block_header_serialize_buf_:0x2b0de0ad29f8, flashback_version:0}, log_storage:{log_tail:{lsn:66390221918}, readable_log_tail:{lsn:66390221918}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66366615552}, min_scn:{val:1729057321384354001, v:0}, curr_block_id:989, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/log", dir_fd:134, min_block_id:965, max_block_id:990, curr_writable_block_id:989}, logical_block_size_:67104768, curr_block_writable_size_:43498402, block_header_serialize_buf_:0x2b0de0ad46b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261739251, this:0x2b0de0ad0f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:48, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:47, majority_max_lsn:{lsn:33216819320}, saved_end_lsn:{lsn:33216819320}, last_submit_prepare_req_time_us:1722494261740552, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0de0ad09b0}) [2024-10-16 14:58:06.482019] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=134] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061886436449, last_flushed_end_lsn:{lsn:66390221918}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061886438120, last_flushed_end_lsn:{lsn:66390221918}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061886436491, last_flushed_end_lsn:{lsn:66390221918}}]) [2024-10-16 14:58:06.482030] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=133533539, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061886436449, last_flushed_end_lsn:{lsn:66390221918}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061886438120, last_flushed_end_lsn:{lsn:66390221918}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061886436491, last_flushed_end_lsn:{lsn:66390221918}}]) [2024-10-16 14:58:06.483475] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.484077] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:06.486339] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:58:06.486353] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:58:06.486363] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=9] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:58:06.486368] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:06.488473] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=6] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:58:06.488622] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=148] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:06.490393] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9315-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:06.492469] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:06.493149] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A98-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:06.493169] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A98-0-0] [lt=18] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:06.501621] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.503734] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=14] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:06.513147] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=61] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:06.513755] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:06.513800] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061878735149000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:58:06.534506] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:58:06.534523] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:58:06.534540] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444667875, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:06.542266] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:06.542438] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601502, offset=9653332}, end_cursor:ObLogCursor{file_id=192, log_id=47601503, offset=9653616}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd203c050, len:940, log_data_len:284, seq:47601502, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:06.544543] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4BC-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061886541693000, start_id=1729061886541693000, end_id=1729061886541693001) [2024-10-16 14:58:06.545093] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=9] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=915296256, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=158445568, wash_ratio=6, tenant_id=1001) [2024-10-16 14:58:06.545150] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=21] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:06.545164] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=13] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:06.545220] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=963407872, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=110333952, wash_ratio=6, tenant_id=1002) [2024-10-16 14:58:06.545259] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=11] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:06.545267] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:58:06.545304] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=929976320, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=143765504, wash_ratio=6, tenant_id=1003) [2024-10-16 14:58:06.545343] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:06.545351] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011875, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:06.545387] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=956841984, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=116899840, wash_ratio=6, tenant_id=1004) [2024-10-16 14:58:06.545423] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:06.545430] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:06.545789] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601503, offset=9653616}, end_cursor:ObLogCursor{file_id=192, log_id=47601504, offset=9653890}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2080050, len:656, log_data_len:274, seq:47601503, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:06.547728] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601504, offset=9653890}, end_cursor:ObLogCursor{file_id=192, log_id=47601505, offset=9654164}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2de8050, len:382, log_data_len:274, seq:47601504, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:06.548007] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601505, offset=9654164}, end_cursor:ObLogCursor{file_id=192, log_id=47601506, offset=9654438}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2140050, len:4204, log_data_len:274, seq:47601505, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:06.548066] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561528, miss_cnt:1, cache_read_size:152590713342, hit_ratio:"9.999999928347016454e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:06.548074] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599114, miss_cnt:9, cache_read_size:149777943623, hit_ratio:"9.999999345926063876e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:06.558720] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:06.558738] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:06.558761] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:58:06.558774] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:58:06.559160] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45D6-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:58:06.559682] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061886559392000, start_id=1729061886559392000, end_id=1729061886559392001) [2024-10-16 14:58:06.559975] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45D7-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:06.559986] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45D7-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=636, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:06.560384] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257850, offset=39057869}, end_cursor:ObLogCursor{file_id=197, log_id=49257851, offset=39058152}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20cc050, len:1587, log_data_len:283, seq:49257850, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:06.562266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB387-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061886561499000, start_id=1729061886561499000, end_id=1729061886561499001) [2024-10-16 14:58:06.563653] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=24] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257851, offset=39058152}, end_cursor:ObLogCursor{file_id=197, log_id=49257852, offset=39058434}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2290050, len:1304, log_data_len:282, seq:49257851, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:06.565881] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657202, miss_cnt:17, cache_read_size:8050195253, hit_ratio:"9.999997286825003107e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:06.566189] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257852, offset=39058434}, end_cursor:ObLogCursor{file_id=197, log_id=49257853, offset=39058716}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2148050, len:1022, log_data_len:282, seq:49257852, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:06.566332] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257853, offset=39058716}, end_cursor:ObLogCursor{file_id=197, log_id=49257854, offset=39058998}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db732e050, len:740, log_data_len:282, seq:49257853, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:06.566374] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650461, miss_cnt:32, cache_read_size:8050447638, hit_ratio:"9.999994892298772475e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:06.567538] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:58:06.569053] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.569053] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.569377] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.569513] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.570091] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061886569156000, start_id=1729061886569156000, end_id=1729061886569156001) [2024-10-16 14:58:06.570115] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061886570111], gts:1729061886569156000, latest_srr:[mts=1729061886570111]}) [2024-10-16 14:58:06.571862] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22697][T1004_TntTransf][T1004][YB4A0A000407-00061E9ABD5C8939-0-0] [lt=15] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:58:06.576051] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061886575477000, start_id=1729061886575477000, end_id=1729061886575477001) [2024-10-16 14:58:06.581690] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55A1-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:06.581702] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55A1-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10409, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:06.581702] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:06.581741] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:06.581751] INFO [LIB] stat (utility.h:1165) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:06.583533] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.590644] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E3F-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:06.597887] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:06.597905] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:06.598330] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1}, {id:1003}]) [2024-10-16 14:58:06.601686] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.603367] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8518F-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:06.603538] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=212, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:06.603562] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:06.603589] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:06.603816] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55A2-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:06.603823] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55A2-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8871, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:06.611633] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5879, mem_used=10398720) [2024-10-16 14:58:06.635970] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061886635959000, start_id=1729061886635959000, end_id=1729061886635959001) [2024-10-16 14:58:06.636784] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=28] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061886234454001, v:0}, new_max_version={val:1729061886234454001, v:0}, last_min_version={val:1729061886069616000, v:0}, last_max_version={val:1729061886069616000, v:0}, total_time_us=1721, delta_us=402328) [2024-10-16 14:58:06.636801] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061886234454001, v:0}, min_version={val:1729061886234454001, v:0}, max_version={val:1729061886234454001, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:58:06.637989] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061886637020000, start_id=1729061886637020000, end_id=1729061886637020001) [2024-10-16 14:58:06.658523] INFO [COMMON] generate_mod_stat_task (memory_dump.cpp:218) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] task info(*task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:58:06.658547] INFO [COMMON] handle (memory_dump.cpp:550) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=23] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:58:06.659494] INFO [COMMON] update_check_range (ob_sql_mem_leak_checker.cpp:62) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] update_check_range(min_check_version=652901, max_check_version=652902, global_version=652903) [2024-10-16 14:58:06.664284] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BAC4-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061886658158000, start_id=1729061886658158000, end_id=1729061886658158001) [2024-10-16 14:58:06.664472] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAC8-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061886658158000, start_id=1729061886658158000, end_id=1729061886658158001) [2024-10-16 14:58:06.664672] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BAC5-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061886658158000, start_id=1729061886658158000, end_id=1729061886658158001) [2024-10-16 14:58:06.665678] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D51-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061886658158000, start_id=1729061886658158000, end_id=1729061886658158001) [2024-10-16 14:58:06.665733] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BAC7-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061886658158000, start_id=1729061886658158000, end_id=1729061886658158001) [2024-10-16 14:58:06.669280] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45D9-0-0] [lt=4] init point thread id with(&point=0x55e3435cbd80, idx_=1084, point=[thread id=107580, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107580) [2024-10-16 14:58:06.676703] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:06.679003] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061886677651000, start_id=1729061886677651000, end_id=1729061886677651001) [2024-10-16 14:58:06.680874] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] [APPLY STAT CB APPEND COST TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=41, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:58:06.680886] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=838, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:58:06.680890] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:58:06.680894] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=-474, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:58:06.680897] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=33, stat_interval=5000000, avg (count/cost)=406, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:58:06.683604] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.690010] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E41-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:06.690022] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E41-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20311, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:06.692721] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:06.695335] INFO handle (memory_dump.cpp:677) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=13] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 206 258 0 1 2 117 1 0 1 4 1 1 0 1 5 38 4 0 1 7 1 2 0 1 8 65 1 0 1 12 18 3 0 1 13 6 1 0 1 14 1 2 0 1 15 1 1 0 1 18 26 3 0 1 20 2 2 0 1 22 1 1 0 500 0 139 248 0 500 7 7 4 0 500 8 50 1 0 500 9 2 1 0 500 10 10 2 0 500 16 3 1 0 500 17 10 7 0 500 22 111 63 0 500 23 61 11 0 508 0 4 13 0 508 8 5 1 0 509 0 3 8 0 509 8 5 1 0 1001 0 131 232 0 1001 2 22 1 0 1001 4 1 1 0 1001 5 23 4 0 1001 8 44 1 0 1001 12 10 3 0 1001 13 3 1 0 1001 14 1 2 0 1001 18 14 3 0 1001 20 1 2 0 1002 0 174 258 0 1002 2 22 1 0 1002 4 2 1 0 1002 5 24 4 0 1002 8 50 1 0 1002 12 19 3 0 1002 13 7 1 0 1002 14 1 2 0 1002 18 14 3 0 1002 20 7 2 0 1003 0 137 235 0 1003 2 29 1 0 1003 4 1 1 0 1003 5 23 4 0 1003 8 44 1 0 1003 12 10 3 0 1003 13 3 1 0 1003 14 1 2 0 1003 18 13 3 0 1003 20 1 2 0 1004 0 291 256 0 1004 2 19 1 0 1004 4 2 1 0 1004 5 34 4 0 1004 8 51 1 0 1004 12 19 3 0 1004 13 10 1 0 1004 14 2 2 0 1004 15 2 1 0 1004 18 25 3 0 1004 20 11 2 0 cost_time: 35870 [2024-10-16 14:58:06.695366] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=14] [MEMORY] tenant: 1, limit: 2,147,483,648 hold: 1,986,498,560 rpc_hold: 0 cache_hold: 861,929,472 cache_used: 861,929,472 cache_item_count: 411 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 543,637,504 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MEMSTORE_CTX_ID hold_bytes= 245,366,784 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TRANS_CTX_MGR_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PLAN_CACHE_CTX_ID hold_bytes= 79,691,776 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 136,314,880 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= META_OBJ_CTX_ID hold_bytes= 37,748,736 limit= 429,496,720 [MEMORY] ctx_id= TX_CALLBACK_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOB_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PS_CACHE_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TX_DATA_TABLE hold_bytes= 54,546,432 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MDS_DATA_ID hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [2024-10-16 14:58:06.695583] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=9] [MEMORY] tenant_id= 1 ctx_id= DEFAULT_CTX_ID hold= 543,637,504 used= 430,885,424 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 32 [MEMORY] hold= 56,180,736 used= 56,153,088 count= 27 avg_used= 2,079,744 block_cnt= 27 chunk_cnt= 27 mod=CACHE_MAP_NODE [MEMORY] hold= 41,963,520 used= 41,943,040 count= 1 avg_used= 41,943,040 block_cnt= 1 chunk_cnt= 1 mod=LogGroupBuffer [MEMORY] hold= 30,965,760 used= 27,525,120 count= 420 avg_used= 65,536 block_cnt= 420 chunk_cnt= 50 mod=MinorMergeMgr [MEMORY] hold= 23,511,040 used= 23,433,792 count= 8 avg_used= 2,929,224 block_cnt= 8 chunk_cnt= 7 mod=IoControl [MEMORY] hold= 22,880,256 used= 21,568,040 count= 161 avg_used= 133,962 block_cnt= 161 chunk_cnt= 38 mod=SqlPlanMon [MEMORY] hold= 18,874,368 used= 16,967,424 count= 256 avg_used= 66,279 block_cnt= 256 chunk_cnt= 63 mod=SqlDtlBuf [MEMORY] hold= 17,694,720 used= 16,942,080 count= 240 avg_used= 70,592 block_cnt= 240 chunk_cnt= 30 mod=[T]ObSessionDIB [MEMORY] hold= 16,797,696 used= 16,777,336 count= 1 avg_used= 16,777,336 block_cnt= 1 chunk_cnt= 1 mod=MrgMemPoolChk [MEMORY] hold= 15,034,848 used= 14,886,881 count= 98 avg_used= 151,906 block_cnt= 31 chunk_cnt= 7 mod=OmtTenant [MEMORY] hold= 11,771,904 used= 9,838,592 count= 236 avg_used= 41,688 block_cnt= 236 chunk_cnt= 61 mod=[T]char [MEMORY] hold= 11,223,040 used= 11,209,664 count= 7 avg_used= 1,601,380 block_cnt= 7 chunk_cnt= 7 mod=MysqlRequesReco [MEMORY] hold= 10,911,744 used= 9,699,328 count= 148 avg_used= 65,536 block_cnt= 148 chunk_cnt= 38 mod=MajorMergeMgr [MEMORY] hold= 8,777,728 used= 8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv [MEMORY] hold= 8,409,088 used= 8,388,640 count= 1 avg_used= 8,388,640 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 6,094,848 used= 6,033,408 count= 8 avg_used= 754,176 block_cnt= 8 chunk_cnt= 7 mod=ResourceGroup [MEMORY] hold= 4,964,352 used= 4,959,488 count= 3 avg_used= 1,653,162 block_cnt= 3 chunk_cnt= 3 mod=SqlFltSpanRec [MEMORY] hold= 4,943,872 used= 4,915,456 count= 2 avg_used= 2,457,728 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe [MEMORY] hold= 4,235,264 used= 4,194,304 count= 2 avg_used= 2,097,152 block_cnt= 2 chunk_cnt= 2 mod=SHARE_BLOCK [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMem [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 12 mod=CkptDgnMemCU [MEMORY] hold= 3,772,416 used= 3,751,936 count= 1 avg_used= 3,751,936 block_cnt= 1 chunk_cnt= 1 mod=DiskIteratorSto [MEMORY] hold= 3,723,264 used= 3,698,688 count= 2 avg_used= 1,849,344 block_cnt= 2 chunk_cnt= 2 mod=LogDIOAligned [MEMORY] hold= 3,162,112 used= 3,145,808 count= 2 avg_used= 1,572,904 block_cnt= 2 chunk_cnt= 2 mod=TenantDLMgr [MEMORY] hold= 3,145,728 used= 2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 5 mod=SqlDtlQueue [MEMORY] hold= 2,828,928 used= 2,708,352 count= 604 avg_used= 4,484 block_cnt= 576 chunk_cnt= 72 mod=WorkerMap [MEMORY] hold= 2,777,088 used= 2,760,192 count= 1 avg_used= 2,760,192 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo [MEMORY] hold= 2,415,648 used= 2,387,720 count= 104 avg_used= 22,958 block_cnt= 104 chunk_cnt= 4 mod=TabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=FastFrezCkr [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap [MEMORY] hold= 2,375,680 used= 2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta [MEMORY] hold= 2,248,704 used= 2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb [MEMORY] hold= 2,198,864 used= 416,700 count= 25,277 avg_used= 16 block_cnt= 270 chunk_cnt= 2 mod=Number [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=LogReplayTask [MEMORY] hold= 1,843,200 used= 1,770,696 count= 9 avg_used= 196,744 block_cnt= 9 chunk_cnt= 2 mod=DagMap [MEMORY] hold= 1,777,664 used= 1,770,696 count= 1 avg_used= 1,770,696 block_cnt= 1 chunk_cnt= 1 mod=TableProc [MEMORY] hold= 1,548,288 used= 1,507,118 count= 5 avg_used= 301,423 block_cnt= 5 chunk_cnt= 1 mod=LDIOSetup [MEMORY] hold= 1,376,256 used= 1,322,752 count= 8 avg_used= 165,344 block_cnt= 8 chunk_cnt= 7 mod=SQLSessionInfo [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=XATimeWheel [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=TransTimeWheel [MEMORY] hold= 1,261,568 used= 681,912 count= 77 avg_used= 8,856 block_cnt= 77 chunk_cnt= 31 mod=[T]ObDSActionAr [MEMORY] hold= 1,196,032 used= 1,182,720 count= 2 avg_used= 591,360 block_cnt= 2 chunk_cnt= 2 mod=ServerObjecPool [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsCache [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsInfo [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPlanCac [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=RewriteRuleMap [MEMORY] hold= 1,024,000 used= 1,015,808 count= 1 avg_used= 1,015,808 block_cnt= 1 chunk_cnt= 1 mod=CommitVersions [MEMORY] hold= 983,488 used= 960,256 count= 121 avg_used= 7,936 block_cnt= 121 chunk_cnt= 9 mod=TltMemtablMgr [MEMORY] hold= 958,464 used= 858,728 count= 13 avg_used= 66,056 block_cnt= 13 chunk_cnt= 13 mod=CommSysVarFac [MEMORY] hold= 958,464 used= 950,272 count= 1 avg_used= 950,272 block_cnt= 1 chunk_cnt= 1 mod=IOWorkerLQ [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=ArcSenderQueue [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=TransService [MEMORY] hold= 802,816 used= 394,352 count= 49 avg_used= 8,048 block_cnt= 49 chunk_cnt= 22 mod=[T]ObPerfEventR [MEMORY] hold= 798,400 used= 779,200 count= 100 avg_used= 7,792 block_cnt= 100 chunk_cnt= 20 mod=CkptDgnMemNode [MEMORY] hold= 798,112 used= 789,624 count= 3 avg_used= 263,208 block_cnt= 2 chunk_cnt= 2 mod=PartitTableTask [MEMORY] hold= 786,688 used= 524,352 count= 33 avg_used= 15,889 block_cnt= 33 chunk_cnt= 4 mod=LogAlloc [MEMORY] hold= 765,440 used= 730,112 count= 184 avg_used= 3,968 block_cnt= 184 chunk_cnt= 44 mod=CkpDgn [MEMORY] hold= 663,552 used= 659,200 count= 1 avg_used= 659,200 block_cnt= 1 chunk_cnt= 1 mod=MulLevelQueue [MEMORY] hold= 663,552 used= 655,360 count= 1 avg_used= 655,360 block_cnt= 1 chunk_cnt= 1 mod=FetchLog [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=ElectTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=FrzTrigger [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTF [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DetectorTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MultiVersionGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=OBJLockGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DupTbLease [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTR [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MdsT [MEMORY] hold= 598,016 used= 590,232 count= 1 avg_used= 590,232 block_cnt= 1 chunk_cnt= 1 mod=DagNetIdMap [MEMORY] hold= 458,752 used= 440,928 count= 3 avg_used= 146,976 block_cnt= 3 chunk_cnt= 3 mod=bf_queue [MEMORY] hold= 421,504 used= 404,416 count= 89 avg_used= 4,544 block_cnt= 89 chunk_cnt= 31 mod=[T]ObTraceEvent [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ApplySrv [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ReplaySrv [MEMORY] hold= 406,400 used= 396,800 count= 50 avg_used= 7,936 block_cnt= 50 chunk_cnt= 29 mod=LinearHashMapNo [MEMORY] hold= 399,200 used= 389,600 count= 50 avg_used= 7,792 block_cnt= 50 chunk_cnt= 32 mod=CkptDgnCUNode [MEMORY] hold= 385,024 used= 377,120 count= 1 avg_used= 377,120 block_cnt= 1 chunk_cnt= 1 mod=ClogGe [MEMORY] hold= 338,112 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 44 chunk_cnt= 2 mod=LSMap [MEMORY] hold= 311,296 used= 262,144 count= 256 avg_used= 1,024 block_cnt= 44 chunk_cnt= 1 mod=TabletToLS [MEMORY] hold= 311,104 used= 303,904 count= 6 avg_used= 50,650 block_cnt= 6 chunk_cnt= 2 mod=PoolFreeList [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompactMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=DmlStatHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompStMap [MEMORY] hold= 294,912 used= 261,632 count= 4 avg_used= 65,408 block_cnt= 4 chunk_cnt= 3 mod=MemTblObj [MEMORY] hold= 268,224 used= 261,888 count= 33 avg_used= 7,936 block_cnt= 33 chunk_cnt= 18 mod=CtxBlockPl [MEMORY] hold= 260,224 used= 253,952 count= 31 avg_used= 8,192 block_cnt= 31 chunk_cnt= 22 mod=ParamCharsType [MEMORY] hold= 212,992 used= 212,160 count= 1 avg_used= 212,160 block_cnt= 1 chunk_cnt= 1 mod=MetaMemMgr [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 19 mod=QueryAllocator [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 20 mod=SqlSession [MEMORY] hold= 204,800 used= 196,744 count= 1 avg_used= 196,744 block_cnt= 1 chunk_cnt= 1 mod=DagNetMap [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=TenDLBucket [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=T3MBucket [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=ResourMapLock [MEMORY] hold= 163,776 used= 156,741 count= 2 avg_used= 78,370 block_cnt= 2 chunk_cnt= 2 mod=CsSstableReader [MEMORY] hold= 147,456 used= 138,240 count= 2 avg_used= 69,120 block_cnt= 2 chunk_cnt= 1 mod=LDBlockBitMap [MEMORY] hold= 147,456 used= 139,264 count= 1 avg_used= 139,264 block_cnt= 1 chunk_cnt= 1 mod=RFLTaskQueue [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 11 mod=RawPl [MEMORY] hold= 128,576 used= 33,168 count= 471 avg_used= 70 block_cnt= 175 chunk_cnt= 33 mod=Coro [MEMORY] hold= 128,000 used= 124,928 count= 16 avg_used= 7,808 block_cnt= 16 chunk_cnt= 2 mod=TabletIDSetNode [MEMORY] hold= 122,880 used= 110,608 count= 2 avg_used= 55,304 block_cnt= 2 chunk_cnt= 1 mod=TabletIDSetBkt [MEMORY] hold= 114,688 used= 106,496 count= 1 avg_used= 106,496 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlMgr [MEMORY] hold= 114,688 used= 111,096 count= 1 avg_used= 111,096 block_cnt= 1 chunk_cnt= 1 mod=AsyncFreezeTabl [MEMORY] hold= 106,496 used= 98,432 count= 1 avg_used= 98,432 block_cnt= 1 chunk_cnt= 1 mod=tenant_mf_mgr [MEMORY] hold= 106,496 used= 98,528 count= 1 avg_used= 98,528 block_cnt= 1 chunk_cnt= 1 mod=LSSvr [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=DictPl [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 10 mod=SqlSessiVarMap [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=PlanVaIdx [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=RlePl [MEMORY] hold= 81,920 used= 73,896 count= 1 avg_used= 73,896 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairCache [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 10 mod=ConstPl [MEMORY] hold= 77,824 used= 65,536 count= 64 avg_used= 1,024 block_cnt= 12 chunk_cnt= 3 mod=TxCtxMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=StorMetaCacheIO [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushLog [MEMORY] hold= 73,728 used= 66,176 count= 1 avg_used= 66,176 block_cnt= 1 chunk_cnt= 1 mod=Rpc [MEMORY] hold= 73,728 used= 65,536 count= 1 avg_used= 65,536 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LockMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxCtxMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxDataMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushMeta [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=DEVICE_MANAGER [MEMORY] hold= 73,728 used= 72,736 count= 1 avg_used= 72,736 block_cnt= 1 chunk_cnt= 1 mod=LogSharedQueueT [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 9 mod=IntDiffPl [MEMORY] hold= 72,240 used= 1,024 count= 896 avg_used= 1 block_cnt= 10 chunk_cnt= 1 mod=CharsetUtil [MEMORY] hold= 65,216 used= 60,712 count= 4 avg_used= 15,178 block_cnt= 4 chunk_cnt= 4 mod=HB_SERVICE [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 8 mod=MdsTableMgr [MEMORY] hold= 64,000 used= 62,464 count= 8 avg_used= 7,808 block_cnt= 8 chunk_cnt= 8 mod=HashNodeLCSta [MEMORY] hold= 58,400 used= 55,296 count= 16 avg_used= 3,456 block_cnt= 11 chunk_cnt= 8 mod=CACHE_INST [MEMORY] hold= 57,344 used= 50,496 count= 1 avg_used= 50,496 block_cnt= 1 chunk_cnt= 1 mod=palf_env [MEMORY] hold= 54,000 used= 5,760 count= 240 avg_used= 24 block_cnt= 146 chunk_cnt= 27 mod=[T]MemoryContex [MEMORY] hold= 48,960 used= 44,840 count= 2 avg_used= 22,420 block_cnt= 2 chunk_cnt= 2 mod=DynamicFactor [MEMORY] hold= 48,880 used= 44,760 count= 2 avg_used= 22,380 block_cnt= 2 chunk_cnt= 2 mod=Autoincrement [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 5 mod=ColEqualPl [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 4 mod=HexStrPl [MEMORY] hold= 45,792 used= 35,616 count= 53 avg_used= 672 block_cnt= 51 chunk_cnt= 23 mod=[T]ObWarningBuf [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DagWarnHisBkt [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 1 mod=ObDMMDL [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=StrDiffPl [MEMORY] hold= 32,912 used= 22,464 count= 54 avg_used= 416 block_cnt= 23 chunk_cnt= 7 mod=OMT_Worker [MEMORY] hold= 32,768 used= 17,408 count= 2 avg_used= 8,704 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTTP [MEMORY] hold= 32,768 used= 25,664 count= 1 avg_used= 25,664 block_cnt= 1 chunk_cnt= 1 mod=TSQLSessionMgr [MEMORY] hold= 32,768 used= 24,688 count= 2 avg_used= 12,344 block_cnt= 2 chunk_cnt= 2 mod=TLD_AssignedMgr [MEMORY] hold= 32,704 used= 26,079 count= 2 avg_used= 13,039 block_cnt= 2 chunk_cnt= 2 mod=CGReadInfoMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 3 mod=StrPrefixPl [MEMORY] hold= 32,000 used= 31,232 count= 4 avg_used= 7,808 block_cnt= 4 chunk_cnt= 4 mod=HashNodePlanCac [MEMORY] hold= 24,576 used= 22,592 count= 1 avg_used= 22,592 block_cnt= 1 chunk_cnt= 1 mod=IO_MGR [MEMORY] hold= 24,576 used= 16,384 count= 1 avg_used= 16,384 block_cnt= 1 chunk_cnt= 1 mod=SlogWriteBuffer [MEMORY] hold= 24,368 used= 20,136 count= 2 avg_used= 10,068 block_cnt= 2 chunk_cnt= 2 mod=TabletStats [MEMORY] hold= 24,336 used= 20,104 count= 2 avg_used= 10,052 block_cnt= 2 chunk_cnt= 2 mod=TLD_ResourceMgr [MEMORY] hold= 21,152 used= 16,896 count= 22 avg_used= 768 block_cnt= 15 chunk_cnt= 12 mod=TGTimer [MEMORY] hold= 17,056 used= 8,448 count= 3 avg_used= 2,816 block_cnt= 2 chunk_cnt= 2 mod=BaseLogWriter [MEMORY] hold= 16,384 used= 8,448 count= 1 avg_used= 8,448 block_cnt= 1 chunk_cnt= 1 mod=PalfEnv [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=SlogNopLog [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TabletSvrBucket [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=DiaTabletMap [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_TableCtxMgr [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MedCheckSet [MEMORY] hold= 16,384 used= 13,824 count= 1 avg_used= 13,824 block_cnt= 1 chunk_cnt= 1 mod=LogReplayStatus [MEMORY] hold= 16,384 used= 11,584 count= 1 avg_used= 11,584 block_cnt= 1 chunk_cnt= 1 mod=LogApplyStatus [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_ClientTask [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=ColSubStrPl [MEMORY] hold= 15,648 used= 15,264 count= 2 avg_used= 7,632 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoNode [MEMORY] hold= 11,984 used= 11,592 count= 2 avg_used= 5,796 block_cnt= 2 chunk_cnt= 1 mod=StopWordSet [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=Scheduler [MEMORY] hold= 9,376 used= 8,984 count= 2 avg_used= 4,492 block_cnt= 2 chunk_cnt= 1 mod=FTPluginMap [MEMORY] hold= 9,248 used= 6,732 count= 13 avg_used= 517 block_cnt= 13 chunk_cnt= 12 mod=SqlSessiQuerSql [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapCn [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapDi [MEMORY] hold= 8,192 used= 7,760 count= 2 avg_used= 3,880 block_cnt= 1 chunk_cnt= 1 mod=HashBucPsSessIn [MEMORY] hold= 8,160 used= 5,512 count= 13 avg_used= 424 block_cnt= 12 chunk_cnt= 7 mod=PackStateMap [MEMORY] hold= 8,144 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 9 mod=SequenceIdMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompCkmItems [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LockWaitMgr [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairs [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=APPLY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MatchLsnMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=PurgeThrottle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FetchLogTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=REPLAY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSIter [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=IORunners [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ls_id [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=HandleSubmit [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=DASIDCache [MEMORY] hold= 8,128 used= 5,512 count= 13 avg_used= 424 block_cnt= 12 chunk_cnt= 8 mod=SequenceMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ShareBlksMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MdsDebugMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FTPluginHandle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LCLSender [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LsElectRefInfo [MEMORY] hold= 8,112 used= 5,512 count= 13 avg_used= 424 block_cnt= 13 chunk_cnt= 8 mod=ContextsMap [MEMORY] hold= 8,096 used= 7,904 count= 1 avg_used= 7,904 block_cnt= 1 chunk_cnt= 1 mod=PsSessionInfo [MEMORY] hold= 8,000 used= 7,808 count= 1 avg_used= 7,808 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsInfo [MEMORY] hold= 7,984 used= 7,792 count= 1 avg_used= 7,792 block_cnt= 1 chunk_cnt= 1 mod=ColUsagNode [MEMORY] hold= 7,920 used= 7,728 count= 1 avg_used= 7,728 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsCache [MEMORY] hold= 7,904 used= 7,712 count= 1 avg_used= 7,712 block_cnt= 1 chunk_cnt= 1 mod=DmlStatNode [MEMORY] hold= 7,824 used= 7,632 count= 1 avg_used= 7,632 block_cnt= 1 chunk_cnt= 1 mod=CheckMapNode [MEMORY] hold= 7,136 used= 66 count= 33 avg_used= 2 block_cnt= 31 chunk_cnt= 11 mod=[T]TSIUseWeak [MEMORY] hold= 6,800 used= 6,216 count= 3 avg_used= 2,072 block_cnt= 2 chunk_cnt= 2 mod=DetectManager [MEMORY] hold= 5,056 used= 4,656 count= 2 avg_used= 2,328 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResRuleIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=PxPoolBkt [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResGrpIdMap [MEMORY] hold= 4,288 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 4,160 used= 3,968 count= 1 avg_used= 3,968 block_cnt= 1 chunk_cnt= 1 mod=TinyAllocator [MEMORY] hold= 3,328 used= 256 count= 16 avg_used= 16 block_cnt= 16 chunk_cnt= 14 mod=[T]TSILastOper [MEMORY] hold= 3,328 used= 1,792 count= 8 avg_used= 224 block_cnt= 5 chunk_cnt= 2 mod=LogIOTask [MEMORY] hold= 3,200 used= 3,008 count= 1 avg_used= 3,008 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlDfc [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=IRMMemHashBuck [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=MedCheckMap [MEMORY] hold= 2,368 used= 1,536 count= 4 avg_used= 384 block_cnt= 4 chunk_cnt= 3 mod=TableLockNode [MEMORY] hold= 2,048 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=LogIOWS [MEMORY] hold= 2,000 used= 1,800 count= 1 avg_used= 1,800 block_cnt= 1 chunk_cnt= 1 mod=PxResMgr [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=MSTXCTX [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TabStatMgrLock [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TXBFRetryId [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GCMemtableMap [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GROUP_INDEX_MAP [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=DUP_LS_SET [MEMORY] hold= 1,344 used= 960 count= 2 avg_used= 480 block_cnt= 1 chunk_cnt= 1 mod=PoolArenaArray [MEMORY] hold= 1,280 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=ModuleInitCtx [MEMORY] hold= 1,248 used= 1,056 count= 1 avg_used= 1,056 block_cnt= 1 chunk_cnt= 1 mod=LOG_HASH_MAP [MEMORY] hold= 1,216 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=LockMemtableMgr [MEMORY] hold= 1,152 used= 960 count= 1 avg_used= 960 block_cnt= 1 chunk_cnt= 1 mod=FTParserMgr [MEMORY] hold= 1,120 used= 120 count= 5 avg_used= 24 block_cnt= 2 chunk_cnt= 1 mod=FreezeTask [MEMORY] hold= 976 used= 776 count= 1 avg_used= 776 block_cnt= 1 chunk_cnt= 1 mod=DepTabletSet [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=tenant_rm_mgr [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=CheckPointTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=FlushTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=DiskUsageTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TabShellTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TLD_TIMER [MEMORY] hold= 848 used= 648 count= 1 avg_used= 648 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfo [MEMORY] hold= 752 used= 552 count= 1 avg_used= 552 block_cnt= 1 chunk_cnt= 1 mod=LSFreeze [MEMORY] hold= 672 used= 472 count= 1 avg_used= 472 block_cnt= 1 chunk_cnt= 1 mod=Election [MEMORY] hold= 672 used= 288 count= 2 avg_used= 144 block_cnt= 2 chunk_cnt= 2 mod=ObTxLSLogCb [MEMORY] hold= 576 used= 384 count= 1 avg_used= 384 block_cnt= 1 chunk_cnt= 1 mod=HAScheduler [MEMORY] hold= 544 used= 144 count= 2 avg_used= 72 block_cnt= 1 chunk_cnt= 1 mod=TntSrvObjPool [MEMORY] hold= 512 used= 120 count= 2 avg_used= 60 block_cnt= 2 chunk_cnt= 2 mod=UserResourceMgr [MEMORY] hold= 448 used= 64 count= 2 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfFSCbNode [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTHandler [MEMORY] hold= 400 used= 208 count= 1 avg_used= 208 block_cnt= 1 chunk_cnt= 1 mod=[T]ObMacroBlock [MEMORY] hold= 320 used= 102 count= 1 avg_used= 102 block_cnt= 1 chunk_cnt= 1 mod=TxLSLogBuf [MEMORY] hold= 304 used= 104 count= 1 avg_used= 104 block_cnt= 1 chunk_cnt= 1 mod=RedoLogBuckLock [MEMORY] hold= 304 used= 112 count= 1 avg_used= 112 block_cnt= 1 chunk_cnt= 1 mod=Coordinator [MEMORY] hold= 256 used= 48 count= 1 avg_used= 48 block_cnt= 1 chunk_cnt= 1 mod=[T]TSITabletTab [MEMORY] hold= 256 used= 64 count= 1 avg_used= 64 block_cnt= 1 chunk_cnt= 1 mod=NegParamIdex [MEMORY] hold= 256 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfoMap [MEMORY] hold= 240 used= 41 count= 1 avg_used= 41 block_cnt= 1 chunk_cnt= 1 mod=KeepAliveBuf [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfRCCbNode [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=RebuildCbNode [MEMORY] hold= 208 used= 16 count= 1 avg_used= 16 block_cnt= 1 chunk_cnt= 1 mod=logservice [MEMORY] hold= 430,885,424 used= 413,697,828 count= 33,079 avg_used= 12,506 mod=SUMMARY [2024-10-16 14:58:06.695667] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=74] [MEMORY] tenant_id= 1 ctx_id= MEMSTORE_CTX_ID hold= 245,366,784 used= 243,449,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 block_cnt= 117 chunk_cnt= 117 mod=Memstore [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 mod=SUMMARY [2024-10-16 14:58:06.695678] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TRANS_CTX_MGR_ID hold= 2,097,152 used= 729,088 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 block_cnt= 1 chunk_cnt= 1 mod=PartTranCtxMgr [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 mod=SUMMARY [2024-10-16 14:58:06.695689] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= PLAN_CACHE_CTX_ID hold= 79,691,776 used= 73,331,136 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 68,663,616 used= 62,143,518 count= 2,832 avg_used= 21,943 block_cnt= 2,832 chunk_cnt= 37 mod=SqlPhyPlan [MEMORY] hold= 3,414,336 used= 3,146,720 count= 326 avg_used= 9,652 block_cnt= 326 chunk_cnt= 37 mod=SqlPlanCache [MEMORY] hold= 1,251,200 used= 1,140,800 count= 575 avg_used= 1,984 block_cnt= 192 chunk_cnt= 37 mod=CreateContext [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 73,331,136 used= 66,432,638 count= 3,735 avg_used= 17,786 mod=SUMMARY [2024-10-16 14:58:06.695700] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= GLIBC hold= 2,097,152 used= 81,040 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 80,944 used= 59,018 count= 263 avg_used= 224 block_cnt= 20 chunk_cnt= 1 mod=PlJit [MEMORY] hold= 96 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PlCodeGen [MEMORY] hold= 81,040 used= 59,050 count= 264 avg_used= 223 mod=SUMMARY [2024-10-16 14:58:06.695711] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= CO_STACK hold= 136,314,880 used= 133,668,864 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 block_cnt= 259 chunk_cnt= 65 mod=CoStack [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 mod=SUMMARY [2024-10-16 14:58:06.695724] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= KVSTORE_CACHE_ID hold= 861,929,472 used= 861,929,472 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 mod=SUMMARY [2024-10-16 14:58:06.695736] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= META_OBJ_CTX_ID hold= 37,748,736 used= 36,290,496 limit= 429,496,720 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 29,491,200 used= 26,236,400 count= 400 avg_used= 65,591 block_cnt= 400 chunk_cnt= 18 mod=L_TabletPool [MEMORY] hold= 6,709,248 used= 5,952,128 count= 91 avg_used= 65,408 block_cnt= 91 chunk_cnt= 15 mod=N_TabletPool [MEMORY] hold= 90,048 used= 84,768 count= 2 avg_used= 42,384 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 36,290,496 used= 32,273,296 count= 493 avg_used= 65,463 mod=SUMMARY [2024-10-16 14:58:06.695746] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= TX_CALLBACK_CTX_ID hold= 12,582,912 used= 6,234,176 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 block_cnt= 767 chunk_cnt= 6 mod=MemtableCallbac [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:58:06.695757] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= LOB_CTX_ID hold= 2,097,152 used= 81,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LobAllocator [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 81,856 used= 73,344 count= 2 avg_used= 36,672 mod=SUMMARY [2024-10-16 14:58:06.695770] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant_id= 1 ctx_id= PS_CACHE_CTX_ID hold= 2,097,152 used= 16,288 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 block_cnt= 4 chunk_cnt= 1 mod=SqlPsCache [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 mod=SUMMARY [2024-10-16 14:58:06.695792] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=8] [MEMORY] tenant_id= 1 ctx_id= TX_DATA_TABLE hold= 54,546,432 used= 52,795,840 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 49,605,184 used= 48,433,408 count= 6,103 avg_used= 7,936 block_cnt= 6,103 chunk_cnt= 25 mod=TX_DATA_SLICE [MEMORY] hold= 3,182,528 used= 3,153,728 count= 3 avg_used= 1,051,242 block_cnt= 3 chunk_cnt= 3 mod=MEMTABLE_ARENA [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 52,795,840 used= 51,595,072 count= 6,107 avg_used= 8,448 mod=SUMMARY [2024-10-16 14:58:06.695807] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= MDS_DATA_ID hold= 4,194,304 used= 398,272 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 260,096 used= 253,952 count= 32 avg_used= 7,936 block_cnt= 32 chunk_cnt= 2 mod=MdsTable [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 2 mod=TX_OP [MEMORY] hold= 398,272 used= 388,864 count= 49 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:58:06.695820] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= SCHEMA_SERVICE hold= 2,097,152 used= 8,128 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=NonPartTenCac [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:58:06.695839] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,082,318,848 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 371,380,224 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 14,680,064 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 104,857,600 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 20,971,520 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= RPC_CTX_ID hold_bytes= 6,291,456 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 26,357,760 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 237,785,088 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= UNEXPECTED_IN_500 hold_bytes= 295,800,832 limit= 9,223,372,036,854,775,807 [2024-10-16 14:58:06.696065] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 371,380,224 used= 311,601,520 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 66,584,784 used= 66,551,824 count= 33 avg_used= 2,016,721 block_cnt= 33 chunk_cnt= 33 mod=Compressor [MEMORY] hold= 31,477,760 used= 31,458,304 count= 1 avg_used= 31,458,304 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 23,584,768 used= 23,499,200 count= 9 avg_used= 2,611,022 block_cnt= 9 chunk_cnt= 8 mod=IoControl [MEMORY] hold= 15,925,248 used= 15,247,872 count= 216 avg_used= 70,592 block_cnt= 216 chunk_cnt= 42 mod=[T]ObSessionDIB [MEMORY] hold= 12,873,728 used= 12,856,352 count= 1 avg_used= 12,856,352 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 9,457,664 used= 9,437,784 count= 1 avg_used= 9,437,784 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 5,505,024 used= 5,238,784 count= 32 avg_used= 163,712 block_cnt= 32 chunk_cnt= 22 mod=Iteratorget_logical_del_time()=9510354584529, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410652, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:06.716870] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DF5-0-0] [lt=17][errcode=-4201] hashtable not init [2024-10-16 14:58:06.716882] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DF5-0-0] [lt=12][errcode=-4016] get cursor info failed(cursor_id=1120, get_sessid()=3221565766) [2024-10-16 14:58:06.716961] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061886716608000, start_id=1729061886716608000, end_id=1729061886716608001) [2024-10-16 14:58:06.729652] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:06.742141] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:06.742156] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:06.759721] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061886758608000, start_id=1729061886758608000, end_id=1729061886758608001) [2024-10-16 14:58:06.760027] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=219, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:06.760042] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=46, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:06.761470] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:06.761470] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:06.761492] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=10] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:06.761497] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=4] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:06.761615] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2DF-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:06.761623] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2DF-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=678, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:06.763634] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2E0-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11046) [2024-10-16 14:58:06.763851] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DC-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:06.763872] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DC-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061886658158000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061886658158000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:06.764936] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9DC-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061886764868000, start_id=1729061886764868000, end_id=1729061886764868001) [2024-10-16 14:58:06.767317] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DC-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061886766985000, start_id=1729061886766985000, end_id=1729061886766985001) [2024-10-16 14:58:06.767378] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DC-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=11208, avg_group_log_size=560) [2024-10-16 14:58:06.767394] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DC-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11046) [2024-10-16 14:58:06.767619] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:58:06.767640] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11208) [2024-10-16 14:58:06.767805] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797DF5-0-0] [lt=15] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=281, avg_log_submit_to_slide_time=625) [2024-10-16 14:58:06.767884] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:06.768705] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:58:06.768716] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:58:06.768719] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:06.769338] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061884415972000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900098662000, latest_log_ts:{val:1729061883567257001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:06.770029] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=51] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061884415972000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:06.770665] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=37] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061884415972000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:06.771256] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=29] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061884415972000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:06.773170] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:58:06.776135] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061886775342000, start_id=1729061886775342000, end_id=1729061886775342001) [2024-10-16 14:58:06.782859] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=14] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:06.783614] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.788289] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.788293] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.788307] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.788315] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.788616] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.788628] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.788934] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.789281] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.789551] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.789558] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.789609] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:06.790057] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E43-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178707, log_task={header:{begin_lsn:{lsn:14538518675}, end_lsn:{lsn:14538518797}, log_id:115178707, min_scn:{val:1729061886658158001, v:0}, max_scn:{val:1729061886658158001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538518553}, prev_proposal_id:23, committed_end_lsn:{lsn:14538518675}, data_checksum:1715074492, accum_checksum:3807365299, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061886689991, freeze_ts:1729061886689991, submit_ts:1729061886689999, flushed_ts:1729061886694632, gen_to_freeze cost time:0, gen_to_submit cost time:8, submit_to_flush cost time:4633}, fs_cb_begin_ts=1729061886790047, log_life_time=100056) [2024-10-16 14:58:06.790477] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B931B-0-0] [lt=25] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=662, avg_log_submit_to_slide_time=30273) [2024-10-16 14:58:06.801822] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.812320] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C69-0-0] [lt=10] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:06.812346] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C69-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061886658158001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061886658158001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:06.813334] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C69-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061886812390000, start_id=1729061886812390000, end_id=1729061886812390001) [2024-10-16 14:58:06.816041] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C69-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061886815612000, start_id=1729061886815612000, end_id=1729061886815612001) [2024-10-16 14:58:06.819289] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:58:06.819340] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21538][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1 and flag like "%DUPLICATE%", tenant_id=1, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:58:06.823798] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=10][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:58:06.823853] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [21817][T1001_DupTbLeas][T1001][Y0-0000000000000000-0-0] [lt=14][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1001 and flag like "%DUPLICATE%", tenant_id=1001, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:58:06.828262] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC588-0-0] [lt=34] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:58:06.828882] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC588-0-0] [lt=12] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1002) [2024-10-16 14:58:06.828898] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC588-0-0] [lt=14] no transfer partition job [2024-10-16 14:58:06.828904] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC588-0-0] [lt=4] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:58:06.830178] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC588-0-0] [lt=9] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1002, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:58:06.830207] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC588-0-0] [lt=21] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:58:06.831377] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:500) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC588-0-0] [lt=15][errcode=-4152] get obj error(ret=-4152) [2024-10-16 14:58:06.831418] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC588-0-0] [lt=13] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1001, status:0}], ls_array_=[{tenant_id:1002, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729057174083520, last_statistic_bg_stat_time=1729008957751717, last_statistic_schema_version=1729008953019688, last_statistic_max_transfer_task_id={id:-1}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:58:06.833829] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1001}, {id:1003}, {id:1}]) [2024-10-16 14:58:06.835846] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061886835538000, start_id=1729061886835538000, end_id=1729061886835538001) [2024-10-16 14:58:06.838006] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061886837664000, start_id=1729061886837664000, end_id=1729061886837664001) [2024-10-16 14:58:06.838607] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=246, start_service_tstamp_=1722494263598349, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:06.842120] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:06.842134] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:06.842301] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:06.862347] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6459, mem_used=12478464) [2024-10-16 14:58:06.864099] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22703][T1004_TntShared][T1004][Y0-0000000000000000-0-0] [lt=10] clear dtl interm result cost(us)(clear_cost=916, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:58:06.865707] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FC6-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:06.865728] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FC6-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:06.865931] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=198, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:06.865947] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:06.865984] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:06.869038] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:06.869060] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=227, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:06.869068] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=146, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:06.869082] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:06.869327] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F43FF-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=396, avg_log_submit_to_slide_time=648) [2024-10-16 14:58:06.878792] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061886878746000, start_id=1729061886878746000, end_id=1729061886878746001) [2024-10-16 14:58:06.883681] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.886291] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=22] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:06.891316] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:06.891356] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=60, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:06.891369] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:06.893004] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1198489600, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:06.901861] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.904826] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:58:06.904884] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22710][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1004 and flag like "%DUPLICATE%", tenant_id=1004, exec_tenant_id=1003, need_member_list=false) [2024-10-16 14:58:06.910149] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:58:06.910166] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27628320, tenant_id=1004) [2024-10-16 14:58:06.910175] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:06.921355] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:06.921374] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=14] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=218) [2024-10-16 14:58:06.923201] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DDE-0-0] [lt=11] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:58:06.926505] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:06.926653] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:06.936178] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=15] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:06.942174] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:06.942190] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:06.944766] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF1-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061886912334, keepalive_ts:-1}) [2024-10-16 14:58:06.944785] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF1-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061886912334, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:06.949472] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4190, mem_used=8318976) [2024-10-16 14:58:06.954468] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15EE-0-0] [lt=7] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:06.959927] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061886958890000, start_id=1729061886958890000, end_id=1729061886958890001) [2024-10-16 14:58:06.960613] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBE-0-0] [lt=13] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:06.960632] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBE-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:06.966589] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061876964845, end_time=1729061886966588, dag_status={}) [2024-10-16 14:58:06.966611] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=20] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:06.966619] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:06.966623] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:06.966630] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:06.966633] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:06.966637] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:06.966640] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:06.966646] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:06.966649] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:06.966655] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966664] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966668] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966675] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966679] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966686] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966690] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966697] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966701] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966708] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966711] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966718] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966723] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966729] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966733] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966740] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966744] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966750] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966754] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966761] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966765] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966771] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966776] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966782] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966786] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966792] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966796] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966803] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966807] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966810] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966814] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966818] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966822] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966826] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966833] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966836] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966843] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966850] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966854] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966861] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966865] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966870] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966874] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966878] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966886] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966889] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966896] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966900] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966906] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966910] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966917] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966920] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966927] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966931] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966938] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966942] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966949] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966952] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:06.966959] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:58:06.966962] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:58:06.966966] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:58:06.966968] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:58:06.966971] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:58:06.966973] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:58:06.966976] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:58:06.966978] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:58:06.966981] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:58:06.966986] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:58:06.966989] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:58:06.967698] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:58:06.975015] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4893-0-0] [lt=15] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061886942633, keepalive_ts:-1}) [2024-10-16 14:58:06.975030] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4893-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061886942633, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:06.976286] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061886975783000, start_id=1729061886975783000, end_id=1729061886975783001) [2024-10-16 14:58:06.979804] INFO [STORAGE] normal_running_env_check_ (ob_tenant_snapshot_service.cpp:279) [22262][T1002_TSnapSvc][T1002][Y0-0000000000000000-0-0] [lt=20] normal_running_env_check_ succ(this={is_inited_:true, is_running_:true, running_mode_:3, meta_loaded_:true, tg_id_:566}) [2024-10-16 14:58:06.979820] INFO [STORAGE] try_gc_tenant_snapshot_ (ob_tenant_snapshot_service.cpp:650) [22262][T1002_TSnapSvc][T1002][Y0-0000000000000000-0-0] [lt=15] try_gc_tenant_snapshot finished(ret=0, ret="OB_SUCCESS", this={is_inited_:true, is_running_:true, running_mode_:3, meta_loaded_:true, tg_id_:566}) [2024-10-16 14:58:06.980531] INFO [STORAGE] calculate_idle_time_ (ob_tenant_snapshot_service.cpp:829) [22262][T1002_TSnapSvc][T1002][Y0-0000000000000000-0-0] [lt=5] ObTenantSnapshotService thread idle time(tg_id_=566, idle_time=60000, running_mode_=3) [2024-10-16 14:58:06.983747] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:06.985984] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=9] ====== tenant freeze timer task ====== [2024-10-16 14:58:06.994884] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:06.994901] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3979, avg_group_log_size=397) [2024-10-16 14:58:06.994920] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:06.995145] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=228, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:06.995173] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=24] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=70, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:06.995189] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:06.995334] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C112E-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=342, avg_log_submit_to_slide_time=628) [2024-10-16 14:58:07.001974] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.007932] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D53-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061886999141, keepalive_ts:-1}) [2024-10-16 14:58:07.007946] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D53-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061886999141, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:07.011994] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D54-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061886999192, keepalive_ts:-1}) [2024-10-16 14:58:07.012005] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D54-0-0] [lt=10] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061886999192, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:07.014323] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] Garbage Collector is running(seq_=656701, gc_interval=10000000) [2024-10-16 14:58:07.014349] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=10] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:07.014363] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=8] gc_check_member_list_ cost time(ret=0, time_us=29) [2024-10-16 14:58:07.014374] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=10] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:58:07.015061] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:07.015078] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=14] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:07.015087] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=9] execute_gc cost time(ret=0, time_us=9) [2024-10-16 14:58:07.015091] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:58:07.036045] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061887035678000, start_id=1729061887035678000, end_id=1729061887035678001) [2024-10-16 14:58:07.038173] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061887037790000, start_id=1729061887037790000, end_id=1729061887037790001) [2024-10-16 14:58:07.039298] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:07.039313] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151364800, tenant_id=1) [2024-10-16 14:58:07.039327] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:07.042255] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=18] PNIO [ratelimit] time: 1729061887042254, bytes: 4446824626512, bw: 0.382033 MB/s, add_ts: 1000369, add_bytes: 400738 [2024-10-16 14:58:07.042709] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40166-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:07.042728] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40166-0-0] [lt=19] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=636, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:07.043538] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:07.043559] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=17] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:07.043575] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=8] cannot_recycle_log_size statistics(cannot_recycle_log_size=292027999, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:58:07.066039] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:07.066554] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD50A-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:07.066567] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD50A-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=498, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:07.070466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061887069830000, start_id=1729061887069830000, end_id=1729061887069830001) [2024-10-16 14:58:07.070494] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=19] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:07.070525] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=19] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061887070520], gts:1729061887069830000, latest_srr:[mts=1729061887070520]}) [2024-10-16 14:58:07.079538] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061887079421000, start_id=1729061887079421000, end_id=1729061887079421001) [2024-10-16 14:58:07.079610] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:07.079627] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:07.081659] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:07.081861] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55AB-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:07.083811] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.093309] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:07.101996] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.102088] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D86-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:07.102121] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D86-0-0] [lt=20] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061886658158005, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061886658158005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:07.103586] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D86-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061887102841000, start_id=1729061887102841000, end_id=1729061887102841001) [2024-10-16 14:58:07.106510] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D86-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061887106076000, start_id=1729061887106076000, end_id=1729061887106076001) [2024-10-16 14:58:07.128497] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:07.128520] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:07.129231] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061877128128, end_time=1729061887129230, dag_status={}) [2024-10-16 14:58:07.129244] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:07.129251] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:07.129254] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:07.129258] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:07.129261] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:07.129264] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:07.129268] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:07.129271] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:07.129277] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:07.129281] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129296] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=15] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129306] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129312] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129320] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129330] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129335] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129340] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129347] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129352] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129357] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129363] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129368] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129372] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129375] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129379] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129383] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129395] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129399] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129402] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129406] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129411] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129417] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129421] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129428] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129438] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129443] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129449] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129453] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129457] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129464] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129468] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129471] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129483] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129490] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129495] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129501] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129505] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129509] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129516] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129520] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129526] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129530] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129537] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129541] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129547] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129552] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129558] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129562] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129565] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129572] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129576] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129583] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129587] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129597] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129606] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129610] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129616] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:07.129620] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:58:07.129624] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:58:07.129629] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:58:07.129631] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:58:07.129634] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:58:07.129636] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:58:07.129639] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:58:07.129641] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:58:07.129644] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:58:07.129646] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:58:07.129649] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:58:07.138407] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4810, mem_used=10398720) [2024-10-16 14:58:07.142567] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=195, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:07.142588] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=37, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:07.142615] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=229, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:07.142622] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=32, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:07.142639] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:07.142970] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40168-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:07.142985] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40168-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=750, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:07.150107] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:58:07.160131] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061887158889000, start_id=1729061887158889000, end_id=1729061887158889001) [2024-10-16 14:58:07.165017] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D54-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061887164747000, start_id=1729061887164747000, end_id=1729061887164747001) [2024-10-16 14:58:07.165084] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D58-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061887164747000, start_id=1729061887164747000, end_id=1729061887164747001) [2024-10-16 14:58:07.165160] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D55-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061887164747000, start_id=1729061887164747000, end_id=1729061887164747001) [2024-10-16 14:58:07.165192] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D56-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061887164747000, start_id=1729061887164747000, end_id=1729061887164747001) [2024-10-16 14:58:07.166120] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FCC-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:07.166617] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD50D-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=265, avg_log_submit_to_slide_time=493) [2024-10-16 14:58:07.167772] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=9] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:58:07.176471] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061887176309000, start_id=1729061887176309000, end_id=1729061887176309001) [2024-10-16 14:58:07.183850] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.190503] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:07.193743] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=10] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:07.195295] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3979) [2024-10-16 14:58:07.195325] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3979) [2024-10-16 14:58:07.195522] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1130-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:07.195531] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1130-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=578, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:07.198616] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=6] ====== traversal_flush timer task ====== [2024-10-16 14:58:07.198732] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D8A-0-0] [lt=13] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061897008855000, latest_log_ts:{val:1729061880424064001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:07.198790] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D8A-0-0] [lt=55] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:07.200692] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=38] success to update min reserved snapshot(reserved_snapshot=1729060084443293000, duration=1800, snapshot_gc_ts=1729061884443293000) [2024-10-16 14:58:07.201489] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=14] update reserved snapshot finished(cost_ts=799, reserved_snapshot=1729060084443293000) [2024-10-16 14:58:07.202056] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.204556] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:07.206752] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5045-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:07.206771] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5045-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:07.213016] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5882-0-0] [lt=19] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:58:07.213121] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5882-0-0] [lt=16][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:07.236253] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061887235207000, start_id=1729061887235207000, end_id=1729061887235207001) [2024-10-16 14:58:07.238398] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061887238391000, start_id=1729061887238391000, end_id=1729061887238391001) [2024-10-16 14:58:07.238464] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:07.238485] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:07.242294] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:07.242311] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:07.242891] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE4016B-0-0] [lt=15] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=13, avg_log_submit_to_flush_time=251, avg_log_submit_to_slide_time=604) [2024-10-16 14:58:07.244161] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD50E-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:07.248500] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD13-0-0] [lt=10][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:07.255760] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061887255758, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007613, add_bytes: 0 [2024-10-16 14:58:07.267220] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21531][T1_TntSharedTim][T1][Y0-0000000000000000-0-0] [lt=15] clear dtl interm result cost(us)(clear_cost=2040, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:58:07.269024] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=27] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:07.269042] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:07.269513] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F440A-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:07.269525] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F440A-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=629, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:07.276451] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 647.25; ) [2024-10-16 14:58:07.276473] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:07.276521] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 154.50; ) [2024-10-16 14:58:07.276528] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:07.276536] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:07.279253] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769ABA1-0-0] [lt=14] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12239, add_timestamp:1729061887279245}) [2024-10-16 14:58:07.279670] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061887278030000, start_id=1729061887278030000, end_id=1729061887278030001) [2024-10-16 14:58:07.283310] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6D-0-0] [lt=10] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12239, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061887283292, sql_renew_time:1729061887283292, is_mark_fail:false}]) [2024-10-16 14:58:07.283331] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6D-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:12239, add_timestamp:1729061887279245}, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061887283292, sql_renew_time:1729061887283292, is_mark_fail:false}]) [2024-10-16 14:58:07.284228] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21668][T1_L0_G0][T1][YB4A0A000407-00061E9B3769ABA1-0-0] [lt=7][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:07.284368] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.286307] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=12] start to print blacklist info [2024-10-16 14:58:07.286463] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=11] ls blacklist refresh finish(cost_time=7876) [2024-10-16 14:58:07.287525] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:07.293597] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:07.302372] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.303864] INFO [LIB] stat (utility.h:1165) [106969][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8519D-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:07.305468] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DF7-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510355173861, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410653, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:07.306197] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [106969][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55B0-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:07.323805] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1043-0-0] [lt=13] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:58:07.330430] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1216, mem_used=4159488) [2024-10-16 14:58:07.336294] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=16] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:07.340420] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=50] success to update min reserved snapshot(reserved_snapshot=1729060078735149000, duration=1800, snapshot_gc_ts=1729061878735149000) [2024-10-16 14:58:07.341273] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=15] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060078735149000) [2024-10-16 14:58:07.341290] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=14] update reserved snapshot finished(cost_ts=874, reserved_snapshot=1729060078735149000) [2024-10-16 14:58:07.342584] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:58:07.350876] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E99841D34BB-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061887349513000, start_id=1729061887349513000, end_id=1729061887349513001) [2024-10-16 14:58:07.353177] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:07.353196] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:07.353204] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:58:07.353211] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:58:07.353214] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:58:07.353217] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17120) [2024-10-16 14:58:07.359175] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA899-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:07.359202] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA899-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061887164747002, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061887164747002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:07.360086] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061887359582000, start_id=1729061887359582000, end_id=1729061887359582001) [2024-10-16 14:58:07.360676] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA899-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061887360610000, start_id=1729061887360610000, end_id=1729061887360610001) [2024-10-16 14:58:07.361563] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A27AFA899-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.060951662063598633e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=303, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.065452636664601869e+03) [2024-10-16 14:58:07.364006] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA899-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061887363796000, start_id=1729061887363796000, end_id=1729061887363796001) [2024-10-16 14:58:07.364069] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA899-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=9987, avg_group_log_size=624) [2024-10-16 14:58:07.364082] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA899-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9987) [2024-10-16 14:58:07.364099] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:142) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA899-0-0] [lt=11] transaction rpc statistics(total_rt=2065, total_process=19, avg_rt=103) [2024-10-16 14:58:07.364410] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=207, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:07.364422] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9987) [2024-10-16 14:58:07.364432] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=121, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:07.364454] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9987) [2024-10-16 14:58:07.364467] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=9, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:07.364924] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59DE-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:07.364938] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59DE-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=987, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:07.364947] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59DE-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=44, avg_log_gen_to_submit_time=71, avg_log_submit_to_flush_time=360, avg_log_submit_to_slide_time=915) [2024-10-16 14:58:07.365102] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:07.366245] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:07.367846] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:58:07.369285] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=175, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:07.369328] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:07.369350] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=32] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=42, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:07.376419] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061887376371000, start_id=1729061887376371000, end_id=1729061887376371001) [2024-10-16 14:58:07.376918] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=41, this=0x2b0e07718280, extra_info="") [2024-10-16 14:58:07.376938] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=20] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=1120, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:58:07.376942] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:58:07.376946] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=-471, this=0x2b0e07718910, extra_info="") [2024-10-16 14:58:07.376949] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=691, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:58:07.378183] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABA-0-0] [lt=7] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:58:07.378199] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21659][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABA-0-0] [lt=11] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=17) [2024-10-16 14:58:07.381809] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:07.384395] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.390432] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E4F-0-0] [lt=23] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:58:07.390637] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=272, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:07.390655] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=371, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:07.390691] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:07.395807] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:58:07.402436] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.423941] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=12] worker thread began to shrink(tenant_->id()=1002, group_id_=1, token=0) [2024-10-16 14:58:07.424313] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106969][T1002_L0_G1][T1002][Y0-0000000000000000-0-0] [lt=19] worker thread exit(tenant_->id()=1002, workers_.get_size()=2) [2024-10-16 14:58:07.424325] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106969][T1002_L0_G1][T0][Y0-0000000000000000-0-0] [lt=9] tenant thread end_run(id_=1002, ret=0, thread_count_=190) [2024-10-16 14:58:07.436339] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061887434403000, start_id=1729061887434403000, end_id=1729061887434403001) [2024-10-16 14:58:07.436425] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4213) [2024-10-16 14:58:07.438460] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061887436704000, start_id=1729061887436704000, end_id=1729061887436704001) [2024-10-16 14:58:07.442987] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40172-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=276, avg_log_submit_to_slide_time=711) [2024-10-16 14:58:07.457427] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C122A-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061887456861000, start_id=1729061887456861000, end_id=1729061887456861001) [2024-10-16 14:58:07.469139] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:07.469154] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:58:07.469348] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:07.469388] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:07.469738] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45F2-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=224, avg_log_submit_to_slide_time=679) [2024-10-16 14:58:07.479898] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061887479214000, start_id=1729061887479214000, end_id=1729061887479214001) [2024-10-16 14:58:07.481631] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7965-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:07.481651] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7965-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:58:07.481833] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=176, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:07.481847] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:07.484415] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.490737] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=43893859, this={dio_aligned_buf:{buf_write_offset:1245, buf_padding_size:2851, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35369255, truncate_used_ts:14641147}, log_block_size:67108864, dir_fd:201, io_fd:197}, aligned_buf_len=4096, aligned_block_offset=43892736, buf_len=122, total_write_size_=12172, total_write_size_after_dio_=421888, ob_pwrite_used_ts=20855, count=100, each_pwrite_cost=208) [2024-10-16 14:58:07.491068] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9329-0-0] [lt=26] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:07.491184] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:58:07.491199] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:58:07.491216] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70939818, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:07.493993] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A99-0-0] [lt=12] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:07.494009] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A99-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:07.494013] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:07.494250] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:07.494260] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:58:07.494266] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2F-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:07.494271] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:58:07.494276] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:58:07.494275] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A2F-0-0] [lt=8] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:07.494280] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:58:07.494283] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:58:07.494285] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:58:07.494288] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:58:07.494290] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:58:07.494293] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:58:07.494295] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:58:07.502499] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.504029] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:58:07.517786] INFO [COMMON] loop_running_dag_net_list (ob_tenant_dag_scheduler.cpp:3738) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] loop running dag net list(ret=0, running_dag_net_list_size=0, slow_dag_net_cnt=0, finished_dag_net_list_size=0) [2024-10-16 14:58:07.517794] INFO [COMMON] loop_finished_dag_net_list (ob_tenant_dag_scheduler.cpp:3761) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] loop finsihed dag net list(ret=0, finished_dag_net_list_size=0) [2024-10-16 14:58:07.527411] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=40] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a67d30, tx_id:{txid:199664578}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061887523613, active_ts:1729061887523613, commit_ts:-1, finish_ts:1729061887523613, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061917523608, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061887523612, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:58:07.528283] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=45] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a81850, tx_id:{txid:199664579}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061887523613, active_ts:1729061887523613, commit_ts:-1, finish_ts:1729061887523613, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061917523608, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061887523612, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:58:07.539979] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21171, task_cnt=299, avg_time=602) [2024-10-16 14:58:07.539993] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=14] TIMER THREAD STAT: (thread_id=21172, task_cnt=299, avg_time=374) [2024-10-16 14:58:07.539997] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=21446, task_cnt=1984, avg_time=9) [2024-10-16 14:58:07.540000] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21532, task_cnt=2971, avg_time=1) [2024-10-16 14:58:07.540002] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21074, task_cnt=170, avg_time=134) [2024-10-16 14:58:07.540005] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21268, task_cnt=60, avg_time=0) [2024-10-16 14:58:07.540007] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21257, task_cnt=60, avg_time=15) [2024-10-16 14:58:07.540010] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21449, task_cnt=120, avg_time=4) [2024-10-16 14:58:07.540012] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21468, task_cnt=60, avg_time=0) [2024-10-16 14:58:07.540018] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21531, task_cnt=74, avg_time=647) [2024-10-16 14:58:07.540020] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21726, task_cnt=1983, avg_time=9) [2024-10-16 14:58:07.540025] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=21811, task_cnt=2971, avg_time=1) [2024-10-16 14:58:07.540027] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21076, task_cnt=20, avg_time=364) [2024-10-16 14:58:07.540030] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21320, task_cnt=20, avg_time=14) [2024-10-16 14:58:07.540040] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=9] TIMER THREAD STAT: (thread_id=21577, task_cnt=60, avg_time=844) [2024-10-16 14:58:07.540045] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21610, task_cnt=248, avg_time=2) [2024-10-16 14:58:07.540047] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21729, task_cnt=120, avg_time=4) [2024-10-16 14:58:07.540050] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21748, task_cnt=60, avg_time=0) [2024-10-16 14:58:07.540055] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21810, task_cnt=72, avg_time=657) [2024-10-16 14:58:07.540058] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21593, task_cnt=30, avg_time=39) [2024-10-16 14:58:07.540060] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21077, task_cnt=12, avg_time=20) [2024-10-16 14:58:07.540063] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21885, task_cnt=60, avg_time=807) [2024-10-16 14:58:07.540065] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21915, task_cnt=248, avg_time=2) [2024-10-16 14:58:07.540067] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21523, task_cnt=26, avg_time=2723) [2024-10-16 14:58:07.540072] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=21452, task_cnt=12, avg_time=14) [2024-10-16 14:58:07.540074] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21467, task_cnt=12, avg_time=2649) [2024-10-16 14:58:07.540077] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21525, task_cnt=12, avg_time=1945) [2024-10-16 14:58:07.540081] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21898, task_cnt=30, avg_time=39) [2024-10-16 14:58:07.540084] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22056, task_cnt=1983, avg_time=10) [2024-10-16 14:58:07.540086] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22142, task_cnt=2970, avg_time=2) [2024-10-16 14:58:07.540091] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=22059, task_cnt=119, avg_time=213) [2024-10-16 14:58:07.540093] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21804, task_cnt=26, avg_time=3149) [2024-10-16 14:58:07.540096] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21595, task_cnt=12, avg_time=32) [2024-10-16 14:58:07.540101] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21591, task_cnt=12, avg_time=1036) [2024-10-16 14:58:07.540109] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=7] TIMER THREAD STAT: (thread_id=21592, task_cnt=12, avg_time=193) [2024-10-16 14:58:07.540115] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=21594, task_cnt=12, avg_time=36) [2024-10-16 14:58:07.540117] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22080, task_cnt=60, avg_time=1) [2024-10-16 14:58:07.540120] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22141, task_cnt=104, avg_time=537) [2024-10-16 14:58:07.540122] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21732, task_cnt=12, avg_time=8) [2024-10-16 14:58:07.540127] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=21747, task_cnt=12, avg_time=2356) [2024-10-16 14:58:07.540129] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21806, task_cnt=12, avg_time=2421) [2024-10-16 14:58:07.540131] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22315, task_cnt=1983, avg_time=8) [2024-10-16 14:58:07.540134] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22399, task_cnt=2971, avg_time=1) [2024-10-16 14:58:07.540136] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22224, task_cnt=60, avg_time=878) [2024-10-16 14:58:07.540138] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22259, task_cnt=248, avg_time=2) [2024-10-16 14:58:07.540143] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=22318, task_cnt=120, avg_time=4) [2024-10-16 14:58:07.540145] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22337, task_cnt=60, avg_time=0) [2024-10-16 14:58:07.540148] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21896, task_cnt=12, avg_time=1037) [2024-10-16 14:58:07.540152] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21897, task_cnt=12, avg_time=175) [2024-10-16 14:58:07.540154] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21899, task_cnt=12, avg_time=36) [2024-10-16 14:58:07.540157] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21900, task_cnt=12, avg_time=29) [2024-10-16 14:58:07.540161] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=22238, task_cnt=30, avg_time=46) [2024-10-16 14:58:07.540168] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=7] TIMER THREAD STAT: (thread_id=22398, task_cnt=72, avg_time=660) [2024-10-16 14:58:07.540171] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22135, task_cnt=26, avg_time=3469) [2024-10-16 14:58:07.540173] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22445, task_cnt=60, avg_time=853) [2024-10-16 14:58:07.540176] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22475, task_cnt=248, avg_time=2) [2024-10-16 14:58:07.540182] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21541, task_cnt=6, avg_time=106) [2024-10-16 14:58:07.540184] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22062, task_cnt=12, avg_time=12) [2024-10-16 14:58:07.540187] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22079, task_cnt=12, avg_time=2401) [2024-10-16 14:58:07.540192] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=22137, task_cnt=12, avg_time=2581) [2024-10-16 14:58:07.540194] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22458, task_cnt=30, avg_time=43) [2024-10-16 14:58:07.540197] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22615, task_cnt=1983, avg_time=10) [2024-10-16 14:58:07.540202] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=22704, task_cnt=2970, avg_time=1) [2024-10-16 14:58:07.540204] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22237, task_cnt=12, avg_time=504) [2024-10-16 14:58:07.540207] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22236, task_cnt=12, avg_time=3630) [2024-10-16 14:58:07.540209] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22239, task_cnt=12, avg_time=53) [2024-10-16 14:58:07.540215] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=22240, task_cnt=12, avg_time=37) [2024-10-16 14:58:07.540217] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22392, task_cnt=26, avg_time=2927) [2024-10-16 14:58:07.540220] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22618, task_cnt=120, avg_time=251) [2024-10-16 14:58:07.540225] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21820, task_cnt=6, avg_time=92) [2024-10-16 14:58:07.540227] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22321, task_cnt=12, avg_time=10) [2024-10-16 14:58:07.540230] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22336, task_cnt=12, avg_time=2407) [2024-10-16 14:58:07.540235] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=22394, task_cnt=12, avg_time=2373) [2024-10-16 14:58:07.540237] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22641, task_cnt=60, avg_time=1) [2024-10-16 14:58:07.540240] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22703, task_cnt=104, avg_time=521) [2024-10-16 14:58:07.540245] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=22456, task_cnt=12, avg_time=982) [2024-10-16 14:58:07.540247] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22457, task_cnt=12, avg_time=159) [2024-10-16 14:58:07.540250] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22459, task_cnt=12, avg_time=34) [2024-10-16 14:58:07.540255] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=22460, task_cnt=12, avg_time=30) [2024-10-16 14:58:07.540257] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22784, task_cnt=60, avg_time=904) [2024-10-16 14:58:07.540260] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22819, task_cnt=248, avg_time=2) [2024-10-16 14:58:07.540265] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=22798, task_cnt=30, avg_time=46) [2024-10-16 14:58:07.540267] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22151, task_cnt=6, avg_time=132) [2024-10-16 14:58:07.540270] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22696, task_cnt=26, avg_time=3624) [2024-10-16 14:58:07.540275] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=22621, task_cnt=12, avg_time=10) [2024-10-16 14:58:07.540278] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22640, task_cnt=12, avg_time=2404) [2024-10-16 14:58:07.540280] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22698, task_cnt=12, avg_time=2346) [2024-10-16 14:58:07.540284] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22408, task_cnt=6, avg_time=89) [2024-10-16 14:58:07.540296] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=12] TIMER THREAD STAT: (thread_id=22796, task_cnt=12, avg_time=3833) [2024-10-16 14:58:07.540299] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22797, task_cnt=12, avg_time=740) [2024-10-16 14:58:07.540302] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22799, task_cnt=12, avg_time=56) [2024-10-16 14:58:07.540305] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22800, task_cnt=12, avg_time=34) [2024-10-16 14:58:07.540309] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22713, task_cnt=6, avg_time=132) [2024-10-16 14:58:07.540312] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21604, task_cnt=2, avg_time=20362) [2024-10-16 14:58:07.540316] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21605, task_cnt=2, avg_time=23) [2024-10-16 14:58:07.540320] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21909, task_cnt=2, avg_time=11415) [2024-10-16 14:58:07.540323] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=21910, task_cnt=2, avg_time=21) [2024-10-16 14:58:07.540326] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22251, task_cnt=2, avg_time=27906) [2024-10-16 14:58:07.540330] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22253, task_cnt=2, avg_time=20) [2024-10-16 14:58:07.540334] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=22469, task_cnt=2, avg_time=11205) [2024-10-16 14:58:07.540337] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=22470, task_cnt=2, avg_time=27) [2024-10-16 14:58:07.540341] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22809, task_cnt=2, avg_time=31655) [2024-10-16 14:58:07.540344] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22811, task_cnt=2, avg_time=20) [2024-10-16 14:58:07.540346] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21263, task_cnt=1, avg_time=2) [2024-10-16 14:58:07.540356] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=11] TIMER THREAD STAT: (thread_id=21203, task_cnt=30, avg_time=0) [2024-10-16 14:58:07.540361] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21079, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540364] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=24970, task_cnt=36, avg_time=1102) [2024-10-16 14:58:07.540367] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=24949, task_cnt=60, avg_time=12) [2024-10-16 14:58:07.540372] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=24963, task_cnt=60, avg_time=2931) [2024-10-16 14:58:07.540375] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21248, task_cnt=60, avg_time=1296) [2024-10-16 14:58:07.540377] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=24944, task_cnt=30, avg_time=571) [2024-10-16 14:58:07.540379] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21247, task_cnt=12, avg_time=2842) [2024-10-16 14:58:07.540382] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21260, task_cnt=6, avg_time=20) [2024-10-16 14:58:07.540386] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21270, task_cnt=6, avg_time=110) [2024-10-16 14:58:07.540389] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21249, task_cnt=6, avg_time=172) [2024-10-16 14:58:07.540391] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=25362, task_cnt=12, avg_time=1311) [2024-10-16 14:58:07.540397] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21478, task_cnt=1, avg_time=20) [2024-10-16 14:58:07.540399] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21602, task_cnt=1, avg_time=24637) [2024-10-16 14:58:07.540401] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21758, task_cnt=1, avg_time=20) [2024-10-16 14:58:07.540404] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21220, task_cnt=2, avg_time=13) [2024-10-16 14:58:07.540406] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21907, task_cnt=1, avg_time=13919) [2024-10-16 14:58:07.540411] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=22090, task_cnt=1, avg_time=20) [2024-10-16 14:58:07.540413] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22247, task_cnt=1, avg_time=138206) [2024-10-16 14:58:07.540416] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22347, task_cnt=1, avg_time=21) [2024-10-16 14:58:07.540422] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=21234, task_cnt=48, avg_time=3) [2024-10-16 14:58:07.540424] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22467, task_cnt=1, avg_time=13426) [2024-10-16 14:58:07.540427] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=26142, task_cnt=6, avg_time=801) [2024-10-16 14:58:07.540429] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=26143, task_cnt=6, avg_time=780) [2024-10-16 14:58:07.540432] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=26147, task_cnt=6, avg_time=37) [2024-10-16 14:58:07.540436] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=26148, task_cnt=6, avg_time=37) [2024-10-16 14:58:07.540439] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22651, task_cnt=1, avg_time=17) [2024-10-16 14:58:07.540441] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22807, task_cnt=1, avg_time=34235) [2024-10-16 14:58:07.540443] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=26169, task_cnt=6, avg_time=239) [2024-10-16 14:58:07.540446] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=26171, task_cnt=6, avg_time=267) [2024-10-16 14:58:07.540448] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=26252, task_cnt=2, avg_time=72) [2024-10-16 14:58:07.540450] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21225, task_cnt=2, avg_time=20) [2024-10-16 14:58:07.540453] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=26251, task_cnt=1, avg_time=7359) [2024-10-16 14:58:07.540455] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=31896, task_cnt=6, avg_time=30) [2024-10-16 14:58:07.540458] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=31906, task_cnt=6, avg_time=27) [2024-10-16 14:58:07.540461] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=21258, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540464] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21578, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540467] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=21886, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540470] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=22225, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540472] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=22446, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540478] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=22785, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540480] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=24952, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540483] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=21389, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540487] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=26217, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540489] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=108056, task_cnt=5897, avg_time=2) [2024-10-16 14:58:07.540493] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=105511, task_cnt=5899, avg_time=2) [2024-10-16 14:58:07.540496] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=108403, task_cnt=5905, avg_time=2) [2024-10-16 14:58:07.540499] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125547, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540501] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125657, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540506] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=125656, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540508] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125746, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540517] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=8] TIMER THREAD STAT: (thread_id=125830, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540520] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125550, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540522] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125829, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540526] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=125749, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540528] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125571, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540531] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125679, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540536] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=125711, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540538] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125768, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540541] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125882, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540546] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=125912, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540549] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125693, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540551] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125895, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540555] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=125636, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540557] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125823, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540560] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=125553, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540563] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125570, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540566] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125643, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540569] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125691, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540572] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125692, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540574] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125694, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540578] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125695, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540580] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125752, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540582] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125767, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540586] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125825, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540588] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125893, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540592] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=125894, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540594] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125896, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540598] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=125897, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540605] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=7] TIMER THREAD STAT: (thread_id=125666, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540607] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125839, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540610] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125704, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540613] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125705, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540616] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125906, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540619] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125907, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540622] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=125586, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540627] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=125702, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540630] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125778, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540632] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=125904, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540638] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=15782, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540640] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=15772, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540643] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=51827, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540648] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=51921, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540651] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=51920, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540662] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=11] TIMER THREAD STAT: (thread_id=52039, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540666] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=52123, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540668] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=51830, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540671] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=52122, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540676] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=52042, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540679] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=51852, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540684] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=51942, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540687] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=51979, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540692] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=52061, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540695] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=52171, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540697] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=52205, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540703] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=51957, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540705] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=52188, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540708] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=51911, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540713] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=52116, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540715] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=51833, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540730] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=14] TIMER THREAD STAT: (thread_id=51850, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540733] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=51916, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540738] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=51955, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540741] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=51956, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540746] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=51958, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540749] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=51960, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540751] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=52045, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540757] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=52060, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540759] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=52118, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540761] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=52186, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540764] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=52187, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540769] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=52189, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540772] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=52190, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540777] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=51930, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540780] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=52132, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540785] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=51970, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540795] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=10] TIMER THREAD STAT: (thread_id=51971, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540798] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=52199, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540800] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=52200, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540805] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=51866, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540808] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=51968, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540810] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=52071, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540816] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=52197, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540818] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=51943, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540820] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=52173, task_cnt=0, avg_time=0) [2024-10-16 14:58:07.540826] INFO dump (ob_timer_monitor.cpp:200) [21270][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=55755, task_cnt=12, avg_time=1346) [2024-10-16 14:58:07.542775] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:07.542789] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:07.550310] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601506, offset=9654438}, end_cursor:ObLogCursor{file_id=192, log_id=47601507, offset=9654722}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db733e050, len:3930, log_data_len:284, seq:47601506, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:07.552154] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4BD-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061887551781000, start_id=1729061887551781000, end_id=1729061887551781001) [2024-10-16 14:58:07.553055] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601507, offset=9654722}, end_cursor:ObLogCursor{file_id=192, log_id=47601508, offset=9654996}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2164050, len:3646, log_data_len:274, seq:47601507, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:07.555125] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601508, offset=9654996}, end_cursor:ObLogCursor{file_id=192, log_id=47601509, offset=9655270}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21bc050, len:3372, log_data_len:274, seq:47601508, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:07.555279] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601509, offset=9655270}, end_cursor:ObLogCursor{file_id=192, log_id=47601510, offset=9655544}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2028050, len:3098, log_data_len:274, seq:47601509, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:07.555340] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561538, miss_cnt:1, cache_read_size:152590714562, hit_ratio:"9.999999928347020894e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:07.555356] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599124, miss_cnt:9, cache_read_size:149777944843, hit_ratio:"9.999999345926111616e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:07.557860] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [22231][T1002_ObXAHbWor][T1002][Y0-0000000000000000-0-0] [lt=13] XA scheduler heartbeat task statistics(avg_time=1) [2024-10-16 14:58:07.560224] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061887559197000, start_id=1729061887559197000, end_id=1729061887559197001) [2024-10-16 14:58:07.567633] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257854, offset=39058998}, end_cursor:ObLogCursor{file_id=197, log_id=49257855, offset=39059281}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2160050, len:458, log_data_len:283, seq:49257854, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:07.567878] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:58:07.568878] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.569194] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:07.569209] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:07.569233] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:07.569241] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:58:07.569363] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.569710] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB388-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061887568902000, start_id=1729061887568902000, end_id=1729061887568902001) [2024-10-16 14:58:07.569791] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.569809] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.569934] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4414-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:07.570092] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:07.570101] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=662, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:07.570721] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257855, offset=39059281}, end_cursor:ObLogCursor{file_id=197, log_id=49257856, offset=39059563}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7352050, len:4271, log_data_len:282, seq:49257855, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:07.570856] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061887570001000, start_id=1729061887570001000, end_id=1729061887570001001) [2024-10-16 14:58:07.571028] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061887571023], gts:1729061887570001000, latest_srr:[mts=1729061887571023]}) [2024-10-16 14:58:07.573062] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257856, offset=39059563}, end_cursor:ObLogCursor{file_id=197, log_id=49257857, offset=39059845}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2054050, len:3989, log_data_len:282, seq:49257856, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:07.573221] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257857, offset=39059845}, end_cursor:ObLogCursor{file_id=197, log_id=49257858, offset=39060127}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73ae050, len:3707, log_data_len:282, seq:49257857, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:07.573268] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650471, miss_cnt:32, cache_read_size:8050448858, hit_ratio:"9.999994892299587379e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:07.573293] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657211, miss_cnt:17, cache_read_size:8050196351, hit_ratio:"9.999997286825392795e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:07.576525] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061887575606000, start_id=1729061887575606000, end_id=1729061887575606001) [2024-10-16 14:58:07.582114] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55B5-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:07.582126] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55B5-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=421, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:07.582126] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:07.582162] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:07.582173] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:07.584478] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.598089] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=9] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:58:07.598182] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:07.598197] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:07.598355] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCB8E-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061887597995000, start_id=1729061887597995000, end_id=1729061887597995001) [2024-10-16 14:58:07.598435] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1001}, {id:1002}]) [2024-10-16 14:58:07.602560] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.603674] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=17) [2024-10-16 14:58:07.603880] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851A3-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:07.604066] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=209, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:07.604083] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=176, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:07.604123] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:07.604304] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45F6-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:07.604313] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C45F6-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=708, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:07.611710] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5893, mem_used=10398720) [2024-10-16 14:58:07.636458] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061887636039000, start_id=1729061887636039000, end_id=1729061887636039001) [2024-10-16 14:58:07.637124] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061887434403000, v:0}, new_max_version={val:1729061887434403000, v:0}, last_min_version={val:1729061887235207000, v:0}, last_max_version={val:1729061887235207000, v:0}, total_time_us=1541, delta_us=202719) [2024-10-16 14:58:07.637180] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=54] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061887434403000, v:0}, min_version={val:1729061887434403000, v:0}, max_version={val:1729061887434403000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:58:07.638641] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061887638156000, start_id=1729061887638156000, end_id=1729061887638156001) [2024-10-16 14:58:07.656829] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=20] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:58:07.665258] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BACE-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061887664479000, start_id=1729061887664479000, end_id=1729061887664479001) [2024-10-16 14:58:07.665333] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BACF-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061887664479000, start_id=1729061887664479000, end_id=1729061887664479001) [2024-10-16 14:58:07.665356] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAD2-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061887664479000, start_id=1729061887664479000, end_id=1729061887664479001) [2024-10-16 14:58:07.665556] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0D5C-0-0] [lt=33] get number(ret=0, service_type_=0, range=1, base_id=1729061887664479000, start_id=1729061887664479000, end_id=1729061887664479001) [2024-10-16 14:58:07.675429] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:58:07.675459] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=27] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:58:07.675467] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=7] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:58:07.675474] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:07.676795] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:07.677715] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=7] Dumping All Cache Objs(alloc_obj_list.count()=115, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:787547, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787548, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787549, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:787550, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:58:07.677984] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=266] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:07.678333] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] no need to update min/max version(new_version={val:1729061887376371000, v:0}, min_version={val:1729061887376371000, v:0}, max_version={val:1729061887376371000, v:0}) [2024-10-16 14:58:07.680382] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061887680292000, start_id=1729061887680292000, end_id=1729061887680292001) [2024-10-16 14:58:07.682595] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=12] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:58:07.683457] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061887682359000, start_id=1729061887682359000, end_id=1729061887682359001) [2024-10-16 14:58:07.684541] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.689043] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:07.690596] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E55-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:07.691078] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40179-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:07.691100] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40179-0-0] [lt=21] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9531, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:07.694327] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:07.700677] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:58:07.702622] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.719017] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:07.729763] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:07.732151] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=12] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:07.737911] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15F2-0-0] [lt=11] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:07.742634] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:07.742648] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:07.760430] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061887759365000, start_id=1729061887759365000, end_id=1729061887759365001) [2024-10-16 14:58:07.760745] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=207, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:07.760764] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=72, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:07.761492] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:07.761514] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=12] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:07.761523] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:07.767940] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:58:07.770487] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DD-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:07.770515] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DD-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061887664479001, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061887664479001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:07.771563] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9DD-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061887771152000, start_id=1729061887771152000, end_id=1729061887771152001) [2024-10-16 14:58:07.773240] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:58:07.774485] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DD-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061887774297000, start_id=1729061887774297000, end_id=1729061887774297001) [2024-10-16 14:58:07.774552] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DD-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=21, avg_log_batch_cnt=1, total_group_log_size=11703, avg_group_log_size=557) [2024-10-16 14:58:07.774571] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DD-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11704) [2024-10-16 14:58:07.774786] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11703) [2024-10-16 14:58:07.774816] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11704) [2024-10-16 14:58:07.774828] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=22, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:07.774976] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2F6-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=23, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:07.774983] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2F6-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=23, stat_interval=1000000, avg (count/cost)=804, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:07.774995] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD2F6-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=21, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=410, avg_log_submit_to_slide_time=804) [2024-10-16 14:58:07.775060] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:07.780353] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=7] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:07.780369] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=12] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:07.780376] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=56) [2024-10-16 14:58:07.783008] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:07.784609] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.788245] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.788260] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.788268] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.788290] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.788646] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.788684] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.788935] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.789256] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.789518] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.789574] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.789617] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:07.802677] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.809691] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4814-0-0] [lt=5] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061887777389, keepalive_ts:-1}) [2024-10-16 14:58:07.809711] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4814-0-0] [lt=18] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061887777389, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:07.818610] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6A-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:07.818636] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6A-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061887664479001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061887664479001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:07.819633] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C6A-0-0] [lt=29] get number(ret=0, service_type_=0, range=1, base_id=1729061887819610000, start_id=1729061887819610000, end_id=1729061887819610001) [2024-10-16 14:58:07.822165] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C6A-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061887821731000, start_id=1729061887821731000, end_id=1729061887821731001) [2024-10-16 14:58:07.822435] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A14-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061887813585, keepalive_ts:-1}) [2024-10-16 14:58:07.822457] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A14-0-0] [lt=22] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061887813585, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:07.829786] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:58:07.829803] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:58:07.829816] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52689750, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:07.833930] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1}, {id:1003}]) [2024-10-16 14:58:07.836488] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061887835468000, start_id=1729061887835468000, end_id=1729061887835468001) [2024-10-16 14:58:07.838636] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061887837752000, start_id=1729061887837752000, end_id=1729061887837752001) [2024-10-16 14:58:07.839278] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info(lsn={lsn:207589580768}, scn={val:1729061887664479002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061887804439, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589580768}, next_to_submit_scn_:{val:1729061887664479002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589580646}, end_lsn:{lsn:207589580768}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061887664479001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061887664479001, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:2017261599, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061887664479001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599127, miss_cnt:9, cache_read_size:149777945209, hit_ratio:"9.999999345926126049e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:58:07.842652] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:07.842665] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:07.843116] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:07.862185] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22136][T1002_TntTransf][T1002][YB4A0A000407-00061E9AB24CC6BF-0-0] [lt=11] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:58:07.862451] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6521, mem_used=12478464) [2024-10-16 14:58:07.870952] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:07.870992] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=210, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:07.871000] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=93, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:07.871009] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:07.879416] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061887878871000, start_id=1729061887878871000, end_id=1729061887878871001) [2024-10-16 14:58:07.879710] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FDA-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:07.879725] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FDA-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:07.879959] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061887878871000, start_id=1729061887878871000, end_id=1729061887878871001) [2024-10-16 14:58:07.880176] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F441C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=323, avg_log_submit_to_slide_time=1453) [2024-10-16 14:58:07.880396] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=238, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:07.880408] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=56, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:07.880440] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:07.880695] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33576, start_service_tstamp_=1722494263708085, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:07.884661] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.891123] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9331-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=228, avg_log_submit_to_slide_time=469) [2024-10-16 14:58:07.891141] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:07.894629] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:07.902694] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.906129] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=20] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:58:07.913091] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] submit pre barrier log success(log_type=17, ls_id={id:1003}, lsn={lsn:207589580946}, scn={val:1729061887664479003, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:58:07.913108] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=21] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:8, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061887913035, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589581002}, next_to_submit_scn_:{val:1729061887664479004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589580890}, end_lsn:{lsn:207589581002}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061887664479003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061887664479003, v:0}, data_checksum:3944859216, flag:0}}, init_mode_version:0, accumulate_checksum:1721980246, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061887664479003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599129, miss_cnt:9, cache_read_size:149777945443, hit_ratio:"9.999999345926134930e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589580946}, scn_:{val:1729061887664479003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061887880245, first_handle_ts_:1729061887913107, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f38a91590, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:58:07.913308] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] success to update reserved snapshot(ret=0, ls_id={id:1003}, min_reserved_snapshot_=1729060080208141000) [2024-10-16 14:58:07.913320] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=12] pre barrier log replay succ(replay_task={ls_id_:{id:1003}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:207589580946}, scn_:{val:1729061887664479003, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061887880245, first_handle_ts_:1729061887913295, replay_cost_:23, retry_cost_:-1, read_log_buf_:0x2b0f38a915a0, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061887913035, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589581002}, next_to_submit_scn_:{val:1729061887664479004, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589580890}, end_lsn:{lsn:207589581002}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061887664479003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061887664479003, v:0}, data_checksum:3944859216, flag:0}}, init_mode_version:0, accumulate_checksum:1721980246, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061887664479003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599129, miss_cnt:9, cache_read_size:149777945443, hit_ratio:"9.999999345926134930e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}, replay_queue_idx=21, ret=0) [2024-10-16 14:58:07.918335] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=34] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:58:07.921044] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=52] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:07.921631] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:07.921675] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061887261758000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:58:07.923680] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=17] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:58:07.926580] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:07.926747] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:07.926890] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=19] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):114, sequence:1}) [2024-10-16 14:58:07.942039] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=16] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:07.942729] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:07.942749] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:07.949591] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4234, mem_used=10398720) [2024-10-16 14:58:07.960446] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061887959214000, start_id=1729061887959214000, end_id=1729061887959214001) [2024-10-16 14:58:07.963550] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:58:07.963567] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:58:07.963574] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:58:07.963579] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:07.965775] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:58:07.966116] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=339] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:07.966175] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:07.966193] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:07.966201] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=164) [2024-10-16 14:58:07.966906] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=24] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:07.966917] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:07.967983] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:58:07.973386] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=26] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:58:07.984746] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:07.991432] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:58:07.992164] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=10] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:07.992193] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=29] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:07.992211] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:07.992276] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061899029444000, latest_log_ts:{val:1729061882455772001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:07.992325] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=45] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:58:07.992342] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:58:07.992349] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=0, rec_log_ts={val:1729061882455772001, v:0}) [2024-10-16 14:58:07.992353] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:07.992356] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:07.992362] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:07.992372] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:58:07.992380] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:07.992385] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:07.995518] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:07.995534] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:07.995524] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=16] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:58:07.995763] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=251, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:07.995776] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=34, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:07.996324] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:07.996381] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1139-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=547, avg_log_submit_to_slide_time=712) [2024-10-16 14:58:08.002752] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.023435] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=15] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:08.036566] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061888035846000, start_id=1729061888035846000, end_id=1729061888035846001) [2024-10-16 14:58:08.036669] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=4243, avg_group_log_size=353) [2024-10-16 14:58:08.038729] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=50] get number(ret=0, service_type_=0, range=1, base_id=1729061888037963000, start_id=1729061888037963000, end_id=1729061888037963001) [2024-10-16 14:58:08.042855] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061888042854, bytes: 4446825113875, bw: 0.464507 MB/s, add_ts: 1000600, add_bytes: 487363 [2024-10-16 14:58:08.043152] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD527-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:08.043169] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD527-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=555, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:08.067085] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:08.067418] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9334-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:08.067431] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9334-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=28927, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:08.071224] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061888070867000, start_id=1729061888070867000, end_id=1729061888070867001) [2024-10-16 14:58:08.071248] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=589, get_gts_with_stc_cnt=32, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=30, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:08.071261] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061888071218], gts:1729061888070867000, latest_srr:[mts=1729061888071218]}) [2024-10-16 14:58:08.071270] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:08.071282] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=541, get_gts_with_stc_cnt=8, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:08.079544] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061888079308000, start_id=1729061888079308000, end_id=1729061888079308001) [2024-10-16 14:58:08.080542] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061888079308000, start_id=1729061888079308000, end_id=1729061888079308001) [2024-10-16 14:58:08.080591] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:08.080608] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:08.082023] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE796F-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1695) [2024-10-16 14:58:08.082205] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:08.084788] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:08.084808] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:08.084815] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.089994] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=16) [2024-10-16 14:58:08.090249] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:08.094919] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:08.098411] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Cache wash clean map node details(ret=0, clean_node_count=27, clean_time=0, clean_start_pos=1887420, clean_num=62914) [2024-10-16 14:58:08.102810] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.104757] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061884101238, last_first_register_time_us=1722494150998907, register_time_us=1729061884101238, curr_time_us=1729061888104756) [2024-10-16 14:58:08.104798] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=19] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061888104778, last_submit_register_req_time_us=1729061888104778, register_parent_reason=3) [2024-10-16 14:58:08.104809] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:58:08.109550] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=11] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:08.110326] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D87-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:08.110352] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D87-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061887664479005, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061887664479005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:08.110367] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:08.110426] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061880208141000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:58:08.110493] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=64] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061880208141000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1002, freeze_info:{latest_snapshot_gc_scn:{val:1729061880208141000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}}) [2024-10-16 14:58:08.112069] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D87-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061888111666000, start_id=1729061888111666000, end_id=1729061888111666001) [2024-10-16 14:58:08.115499] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D87-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061888114804000, start_id=1729061888114804000, end_id=1729061888114804001) [2024-10-16 14:58:08.124760] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=15] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97898373120) [2024-10-16 14:58:08.128460] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:08.128476] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:08.138502] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4935, mem_used=10398720) [2024-10-16 14:58:08.143064] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=192, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:08.143082] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=68, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:08.143113] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=257, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:08.143119] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=45, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:08.143132] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:08.143396] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40186-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:08.143402] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40186-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=678, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:08.150134] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=15] gc stale ls task succ [2024-10-16 14:58:08.160692] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061888159669000, start_id=1729061888159669000, end_id=1729061888159669001) [2024-10-16 14:58:08.160786] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=13] current monitor number(seq_=-1) [2024-10-16 14:58:08.165537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BAD3-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061888165164000, start_id=1729061888165164000, end_id=1729061888165164001) [2024-10-16 14:58:08.165672] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D5F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061888165164000, start_id=1729061888165164000, end_id=1729061888165164001) [2024-10-16 14:58:08.165704] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BAD5-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061888165164000, start_id=1729061888165164000, end_id=1729061888165164001) [2024-10-16 14:58:08.165749] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAD7-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061888165164000, start_id=1729061888165164000, end_id=1729061888165164001) [2024-10-16 14:58:08.167216] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FE0-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:08.168054] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:08.169715] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15EA-0-0] [lt=14] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:08.173910] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15F0-0-0] [lt=10] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:08.174312] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15EC-0-0] [lt=12] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:08.184850] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.191003] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:08.195961] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3656) [2024-10-16 14:58:08.195994] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3656) [2024-10-16 14:58:08.196099] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45D04-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:08.196114] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45D04-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=739, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:08.202881] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.205309] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:08.205415] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109203346, max_lsn:17959737711, max_scn:{val:1729061888070867000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7385580775}, reuse_lsn:{lsn:17959737711}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17917794671}}, last_submit_lsn:{lsn:17959737589}, last_submit_end_lsn:{lsn:17959737711}, last_submit_log_id:109203346, last_submit_log_pid:246, max_flushed_lsn:{lsn:17959737589}, max_flushed_end_lsn:{lsn:17959737711}, max_flushed_log_pid:246, committed_end_lsn:{lsn:17959737711}, last_slide_log_id:109203346, last_slide_scn:{val:1729061888070867000, v:0}, last_slide_lsn:{lsn:17959737589}, last_slide_end_lsn:{lsn:17959737711}, last_slide_log_pid:246, last_slide_log_accum_checksum:2112391168, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1d4530}, [StateMgr]={this:0x2b0e0f1d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109203347, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261839590, palf_role_change_cb:0x2b0e0f1df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, reconfig_barrier:{prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_end_lsn:{lsn:7385763025}, prev_mode_pid:246}, persistent_config_version:{proposal_id:246, config_seq:121}, ms_ack_list:0, resend_config_version:{proposal_id:246, config_seq:121}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258131745, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258131745, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262039736, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728994126835081001, v:0}, min_block_id:265, min_block_min_scn:{val:1728952113212346001, v:0}, base_lsn_for_block_gc:{lsn:17849868288}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:246}, log_config_meta:{version:3, proposal_id:246, prev:{version:1, config:{config_version:{proposal_id:245, config_seq:120}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:246, config_seq:121}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:245, prev_lsn:{lsn:7385762903}, prev_mode_pid:246}, log_snapshot_meta:{version:1, base_lsn:{lsn:17849868288}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:246, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1798144}, readable_log_tail:{lsn:1798144}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17984077824}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/meta", dir_fd:167, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65306624, block_header_serialize_buf_:0x2b0e0f1da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17959737711}, readable_log_tail:{lsn:17959737711}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17916973056}, min_scn:{val:1729035023717936004, v:0}, curr_block_id:267, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1/log", dir_fd:169, min_block_id:265, max_block_id:268, curr_writable_block_id:267}, logical_block_size_:67104768, curr_block_writable_size_:24340113, block_header_serialize_buf_:0x2b0e0f1dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261838799, this:0x2b0e0f1d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:246, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:245, majority_max_lsn:{lsn:7385763025}, saved_end_lsn:{lsn:7385763025}, last_submit_prepare_req_time_us:1722494261839744, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1d89b0}) [2024-10-16 14:58:08.205568] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=150] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888174755, last_flushed_end_lsn:{lsn:17959737711}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888171708, last_flushed_end_lsn:{lsn:17959737711}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888171696, last_flushed_end_lsn:{lsn:17959737711}}]) [2024-10-16 14:58:08.205579] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=109203347, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888174755, last_flushed_end_lsn:{lsn:17959737711}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888171708, last_flushed_end_lsn:{lsn:17959737711}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888171696, last_flushed_end_lsn:{lsn:17959737711}}]) [2024-10-16 14:58:08.207503] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5046-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:08.207521] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5046-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:08.236723] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061888235437000, start_id=1729061888235437000, end_id=1729061888235437001) [2024-10-16 14:58:08.238996] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061888237999000, start_id=1729061888237999000, end_id=1729061888237999001) [2024-10-16 14:58:08.239059] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:08.239076] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:08.239270] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=46] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:213058276087}, scn={val:1729061888165164002, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:58:08.239324] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=60] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060080208141000) [2024-10-16 14:58:08.239337] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:213058276087}, scn_:{val:1729061888165164002, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061888229997, first_handle_ts_:1729061888239316, replay_cost_:19, retry_cost_:-1, read_log_buf_:0x2b0f38a93330, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061888239216, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058276143}, next_to_submit_scn_:{val:1729061888165164003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058276031}, end_lsn:{lsn:213058276143}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061888165164002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061888165164002, v:0}, data_checksum:2466643087, flag:1}}, init_mode_version:0, accumulate_checksum:3615532369, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061888165164002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561546, miss_cnt:1, cache_read_size:152590716125, hit_ratio:"9.999999928347025335e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}, replay_queue_idx=13, ret=0) [2024-10-16 14:58:08.242857] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:08.242873] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:08.243257] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD52C-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=284, avg_log_submit_to_slide_time=521) [2024-10-16 14:58:08.263374] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061888263372, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007614, add_bytes: 0 [2024-10-16 14:58:08.266975] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FE2-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=313, avg_log_submit_to_slide_time=38847) [2024-10-16 14:58:08.270928] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:08.270951] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:08.271349] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C460A-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:08.271366] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C460A-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2086, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:08.278797] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=29] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 135.75; ) [2024-10-16 14:58:08.278823] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:08.278865] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 158.00; ) [2024-10-16 14:58:08.278874] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:08.278885] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:08.280680] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061888279402000, start_id=1729061888279402000, end_id=1729061888279402001) [2024-10-16 14:58:08.280856] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061888279402000, start_id=1729061888279402000, end_id=1729061888279402001) [2024-10-16 14:58:08.284909] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.298710] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:08.302912] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.304308] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851B1-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:08.304766] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] succ to submit log task to replay service(to_submit_lsn={lsn:207589581424}, to_submit_scn={val:1729061888165164001, v:0}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061888304740, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589581490}, next_to_submit_scn_:{val:1729061888165164002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589581368}, end_lsn:{lsn:207589581490}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061888165164001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061888165164001, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:3918748399, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061888165164001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599133, miss_cnt:9, cache_read_size:149777945931, hit_ratio:"9.999999345926153804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:58:08.308609] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DF9-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510356176993, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410654, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:08.314268] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:08.314288] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:08.314307] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373749239, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:08.330439] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:08.330494] WDIAG read_multi_row (ob_table_access_helper.h:238) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1002, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:08.330535] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1289, mem_used=4159488) [2024-10-16 14:58:08.331240] INFO delete_row (ob_table_access_helper.h:312) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=8] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1002, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061888330502) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061888330502) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:58:08.331335] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=17] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:58:08.331408] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=9] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:58:08.331458] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22222][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=6] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:58:08.336339] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=12] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:08.343169] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:08.343244] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:08.347451] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:58:08.347470] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=12582912, ctx.total_memstore_used_=46137344, ctx.total_memstore_hold_=46137344, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:58:08.347487] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:08.360753] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061888359981000, start_id=1729061888359981000, end_id=1729061888359981001) [2024-10-16 14:58:08.369620] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] Cache replace map node details(ret=0, replace_node_count=352, replace_time=0, replace_start_pos=2516560, replace_num=31457) [2024-10-16 14:58:08.369639] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=352, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:08.369894] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89A-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:08.369916] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89A-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061888165164001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061888165164001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:08.371137] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=181, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:08.371151] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=453, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:08.371169] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA89A-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061888370538000, start_id=1729061888370538000, end_id=1729061888370538001) [2024-10-16 14:58:08.371182] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:08.372232] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A27AFA89A-0-0] [lt=33] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.060951662063598633e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=303, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.065452636664601869e+03) [2024-10-16 14:58:08.373729] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89A-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061888373709000, start_id=1729061888373709000, end_id=1729061888373709001) [2024-10-16 14:58:08.373791] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89A-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=9690, avg_group_log_size=570) [2024-10-16 14:58:08.373803] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89A-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9690) [2024-10-16 14:58:08.374033] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=267, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:08.374045] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9690) [2024-10-16 14:58:08.374056] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=86, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:08.374072] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9690) [2024-10-16 14:58:08.374083] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:08.374212] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59EF-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:08.374224] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59EF-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=606, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:08.374239] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D59EF-0-0] [lt=15] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=383, avg_log_submit_to_slide_time=577) [2024-10-16 14:58:08.374245] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:08.381765] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:58:08.381781] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54432140, tenant_id=1003) [2024-10-16 14:58:08.381796] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=12] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:08.382419] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1929) [2024-10-16 14:58:08.384946] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.391032] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E63-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:08.391261] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=187, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:08.391279] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=114, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:08.391315] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:08.393282] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=15] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:08.395862] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:08.402971] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.404335] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851B3-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:58:08.408116] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DC-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061888399318, keepalive_ts:-1}) [2024-10-16 14:58:08.408135] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DC-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061888399318, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:08.436873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061888436128000, start_id=1729061888436128000, end_id=1729061888436128001) [2024-10-16 14:58:08.436946] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3544) [2024-10-16 14:58:08.438991] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061888438210000, start_id=1729061888438210000, end_id=1729061888438210001) [2024-10-16 14:58:08.439874] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:58:08.443738] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD532-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=331, avg_log_submit_to_slide_time=549) [2024-10-16 14:58:08.466266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C123B-0-0] [lt=41] get number(ret=0, service_type_=0, range=1, base_id=1729061888465975000, start_id=1729061888465975000, end_id=1729061888465975001) [2024-10-16 14:58:08.467142] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:08.471032] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:08.471058] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:08.471261] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:08.471313] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:08.471713] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F442F-0-0] [lt=17] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=653, avg_log_submit_to_slide_time=2065) [2024-10-16 14:58:08.480918] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061888479788000, start_id=1729061888479788000, end_id=1729061888479788001) [2024-10-16 14:58:08.480967] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061888479788000, start_id=1729061888479788000, end_id=1729061888479788001) [2024-10-16 14:58:08.482253] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7977-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:08.482470] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=265, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:08.482485] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:08.484995] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.492832] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:08.494845] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9A-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:08.494863] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9A-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:08.498979] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:08.503012] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.504617] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:58:08.513195] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:58:08.513964] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:08.513991] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=27] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:08.514012] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:08.514078] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061895590307000, latest_log_ts:{val:1729061878965510001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:08.514137] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=55] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:08.514154] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:58:08.514161] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061878965510001, v:0}) [2024-10-16 14:58:08.514165] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:58:08.514168] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:08.514175] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:58:08.514185] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:58:08.514196] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:08.514201] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:08.514606] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB900-0-0] [lt=10] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:58:08.514621] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB900-0-0] [lt=11] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:08.514638] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000408-00061E9B68AEB900-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:58:08.524004] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E998FCD75CC-0-0] [lt=37] get number(ret=0, service_type_=0, range=1, base_id=1729061888523561000, start_id=1729061888523561000, end_id=1729061888523561001) [2024-10-16 14:58:08.534616] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:08.534635] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:58:08.534651] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444678438, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:08.543849] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD536-0-0] [lt=18] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:08.544325] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:08.556247] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601510, offset=9655544}, end_cursor:ObLogCursor{file_id=192, log_id=47601511, offset=9655828}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db7392050, len:2824, log_data_len:284, seq:47601510, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:08.558061] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4BE-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061888556158000, start_id=1729061888556158000, end_id=1729061888556158001) [2024-10-16 14:58:08.559677] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601511, offset=9655828}, end_cursor:ObLogCursor{file_id=192, log_id=47601512, offset=9656102}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73c6050, len:2540, log_data_len:274, seq:47601511, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:08.560821] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061888559262000, start_id=1729061888559262000, end_id=1729061888559262001) [2024-10-16 14:58:08.561677] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=11] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:58:08.561697] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] get_min_unreplayed_log_info(lsn={lsn:14538520993}, scn={val:1729061888165164005, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061888491596, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538520993}, next_to_submit_scn_:{val:1729061888165164005, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538520871}, end_lsn:{lsn:14538520993}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061888165164004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061888165164004, v:0}, data_checksum:3348837161, flag:1}}, init_mode_version:0, accumulate_checksum:4092326486, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061888165164004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650481, miss_cnt:32, cache_read_size:8050450078, hit_ratio:"9.999994892300403393e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:58:08.561754] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=53] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:58:08.561801] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601512, offset=9656102}, end_cursor:ObLogCursor{file_id=192, log_id=47601513, offset=9656376}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd209c050, len:2266, log_data_len:274, seq:47601512, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:08.562005] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=26] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601513, offset=9656376}, end_cursor:ObLogCursor{file_id=192, log_id=47601514, offset=9656650}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2184050, len:1992, log_data_len:274, seq:47601513, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:08.562066] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561549, miss_cnt:1, cache_read_size:152590716491, hit_ratio:"9.999999928347026446e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:08.562075] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=49] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599135, miss_cnt:9, cache_read_size:149777946175, hit_ratio:"9.999999345926163796e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:08.567116] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FE8-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173419, log_task={header:{begin_lsn:{lsn:14352407219}, end_lsn:{lsn:14352407341}, log_id:115173419, min_scn:{val:1729061888165164003, v:0}, max_scn:{val:1729061888165164003, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352407106}, prev_proposal_id:17, committed_end_lsn:{lsn:14352407219}, data_checksum:940009244, accum_checksum:2743864830, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061888467056, freeze_ts:1729061888467056, submit_ts:1729061888467061, flushed_ts:1729061888467384, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:323}, fs_cb_begin_ts=1729061888567110, log_life_time=100054) [2024-10-16 14:58:08.568624] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.568692] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.568881] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.569358] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.571173] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=419, replace_time=0, replace_start_pos=2548017, replace_num=31457) [2024-10-16 14:58:08.571210] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=419, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:08.571306] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:08.571320] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:08.571357] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:08.571365] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:08.571590] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061888570581000, start_id=1729061888570581000, end_id=1729061888570581001) [2024-10-16 14:58:08.571608] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:08.571613] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=20] ts response task statistics(alloc_count=261365514, free_count=130682756) [2024-10-16 14:58:08.571803] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4432-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:08.571811] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4432-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2042, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:08.571908] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=24, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:58:08.571918] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=1098, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:58:08.571922] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:58:08.571925] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=-595, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:58:08.571929] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=4] [APPLY STAT CB TOTAL TIME](cur_stat_count=19, stat_interval=5000000, avg (count/cost)=527, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:58:08.575003] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257858, offset=39060127}, end_cursor:ObLogCursor{file_id=197, log_id=49257859, offset=39060410}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd206c050, len:3425, log_data_len:283, seq:49257858, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:08.577017] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB389-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061888576187000, start_id=1729061888576187000, end_id=1729061888576187001) [2024-10-16 14:58:08.577812] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257859, offset=39060410}, end_cursor:ObLogCursor{file_id=197, log_id=49257860, offset=39060692}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd22c4050, len:3142, log_data_len:282, seq:49257859, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:08.579712] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257860, offset=39060692}, end_cursor:ObLogCursor{file_id=197, log_id=49257861, offset=39060974}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db738e050, len:2860, log_data_len:282, seq:49257860, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:08.579911] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257861, offset=39060974}, end_cursor:ObLogCursor{file_id=197, log_id=49257862, offset=39061256}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd214c050, len:2578, log_data_len:282, seq:49257861, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:08.579966] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650481, miss_cnt:32, cache_read_size:8050450078, hit_ratio:"9.999994892300403393e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:08.579975] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657222, miss_cnt:17, cache_read_size:8050197684, hit_ratio:"9.999997286825869081e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:08.582330] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7979-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=12, avg_log_batch_cnt=0, total_group_log_size=2051, avg_group_log_size=170) [2024-10-16 14:58:08.582705] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55CB-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:08.582719] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55CB-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9510, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:08.582716] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1929, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:08.582752] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=52] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=84, stat_interval=1000000, avg (count/cost)=31, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:08.582758] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=84, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:08.585030] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.590315] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115178725, max_lsn:14538520993, max_scn:{val:1729061888165164004, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6487749074}, reuse_lsn:{lsn:14538520993}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14496577953}}, last_submit_lsn:{lsn:14538520871}, last_submit_end_lsn:{lsn:14538520993}, last_submit_log_id:115178725, last_submit_log_pid:23, max_flushed_lsn:{lsn:14538520871}, max_flushed_end_lsn:{lsn:14538520993}, max_flushed_log_pid:23, committed_end_lsn:{lsn:14538520993}, last_slide_log_id:115178725, last_slide_scn:{val:1729061888165164004, v:0}, last_slide_lsn:{lsn:14538520871}, last_slide_end_lsn:{lsn:14538520993}, last_slide_log_pid:23, last_slide_log_accum_checksum:4092326486, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43cc0530}, [StateMgr]={this:0x2b0e43cc4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, leader:"10.0.4.8:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6488080380}, scan_disk_log_finished:true, last_check_start_id:115178726, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261653045, palf_role_change_cb:0x2b0e43ccb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, reconfig_barrier:{prev_log_proposal_id:21, prev_lsn:{lsn:6487893471}, prev_end_lsn:{lsn:6487893593}, prev_mode_pid:22}, persistent_config_version:{proposal_id:23, config_seq:34}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061886088435, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061886088435, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43cc1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, accepted_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1728259580444569001, v:0}, min_block_id:201, min_block_min_scn:{val:1728204893605525001, v:0}, base_lsn_for_block_gc:{lsn:14427525120}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:23}, log_config_meta:{version:3, proposal_id:23, prev:{version:1, config:{config_version:{proposal_id:23, config_seq:33}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:13, lock_time:1725017521087102}}, curr:{version:1, config:{config_version:{proposal_id:23, config_seq:34}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:13, lock_time:1725017521253864}}, prev_log_proposal_id:23, prev_lsn:{lsn:9579163212}, prev_mode_pid:23}, log_snapshot_meta:{version:1, base_lsn:{lsn:14427525120}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:23, mode_version:0, access_mode:1, ref_scn:{val:1717582266388391000, v:0}}}, log_meta_storage:{log_tail:{lsn:1134592}, readable_log_tail:{lsn:1134592}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14561734656}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/meta", dir_fd:166, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65970176, block_header_serialize_buf_:0x2b0e43cc69f8, flashback_version:0}, log_storage:{log_tail:{lsn:14538520993}, readable_log_tail:{lsn:14538520993}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14494629888}, min_scn:{val:1729026040007117003, v:0}, curr_block_id:216, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1002/log", dir_fd:201, min_block_id:201, max_block_id:217, curr_writable_block_id:216}, logical_block_size_:67104768, curr_block_writable_size_:23213663, block_header_serialize_buf_:0x2b0e43cc86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261652788, this:0x2b0e43cc4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:22, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:21, majority_max_lsn:{lsn:6487893593}, saved_end_lsn:{lsn:6487893593}, last_submit_prepare_req_time_us:1722494261653224, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43cc49b0}) [2024-10-16 14:58:08.590463] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=145] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:58:08.590475] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=115178726, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433766130, last_flushed_end_lsn:{lsn:6488080380}}]) [2024-10-16 14:58:08.591093] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E67-0-0] [lt=25] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:08.598279] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=25][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:08.598301] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:08.598532] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1}, {id:1001}]) [2024-10-16 14:58:08.603071] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.604463] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851B7-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:58:08.604672] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=192, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:08.604689] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:08.604717] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:08.604911] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55CC-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:08.604918] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55CC-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=10269, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:08.605666] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1002, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:220938543, max_lsn:213058276631, max_scn:{val:1729061888165164006, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:60464363030}, reuse_lsn:{lsn:213058276631}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:213016333591}}, last_submit_lsn:{lsn:213058276509}, last_submit_end_lsn:{lsn:213058276631}, last_submit_log_id:220938543, last_submit_log_pid:88, max_flushed_lsn:{lsn:213058276509}, max_flushed_end_lsn:{lsn:213058276631}, max_flushed_log_pid:88, committed_end_lsn:{lsn:213058276631}, last_slide_log_id:220938543, last_slide_scn:{val:1729061888165164006, v:0}, last_slide_lsn:{lsn:213058276509}, last_slide_end_lsn:{lsn:213058276631}, last_slide_log_pid:88, last_slide_log_accum_checksum:1670534291, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0dac0530}, [StateMgr]={this:0x2b0e0dac4830, self:"10.0.4.7:2890", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, leader:"10.0.4.8:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:60468525403}, scan_disk_log_finished:true, last_check_start_id:220938544, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261639483, palf_role_change_cb:0x2b0e0dacb230, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, reconfig_barrier:{prev_log_proposal_id:86, prev_lsn:{lsn:60466037954}, prev_end_lsn:{lsn:60466038076}, prev_mode_pid:87}, persistent_config_version:{proposal_id:88, config_seq:21}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061888104778, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061888104778, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0dac1c70}, [ModeMgr]={palf_id:1002, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, accepted_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1729009096591759001, v:0}, min_block_id:3154, min_block_min_scn:{val:1729006455168279002, v:0}, base_lsn_for_block_gc:{lsn:212587905024}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:88}, log_config_meta:{version:3, proposal_id:88, prev:{version:1, config:{config_version:{proposal_id:87, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:88, config_seq:21}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:87, prev_lsn:{lsn:60468525281}, prev_mode_pid:88}, log_snapshot_meta:{version:1, base_lsn:{lsn:212587905024}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:88, mode_version:0, access_mode:1, ref_scn:{val:1717560293733356000, v:0}}}, log_meta_storage:{log_tail:{lsn:5767168}, readable_log_tail:{lsn:5767168}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213124743168}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/meta", dir_fd:175, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61337600, block_header_serialize_buf_:0x2b0e0dac69f8, flashback_version:0}, log_storage:{log_tail:{lsn:213058276631}, readable_log_tail:{lsn:213058276631}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:213057638400}, min_scn:{val:1729061839666529002, v:0}, curr_block_id:3175, palf_id:1002}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1002/log", dir_fd:177, min_block_id:3154, max_block_id:3176, curr_writable_block_id:3175}, logical_block_size_:67104768, curr_block_writable_size_:66466537, block_header_serialize_buf_:0x2b0e0dac86b8, flashback_version:0}, palf_epoch:3, last_purge_throttling_ts:1722494261637707, this:0x2b0e0dac4f70}, [Reconfirm]={palf_id:1002, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:87, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:86, majority_max_lsn:{lsn:60466038076}, saved_end_lsn:{lsn:60466038076}, last_submit_prepare_req_time_us:1722494261639634, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0dac49b0}) [2024-10-16 14:58:08.605798] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=129] [PALF_DUMP](palf_id=1002, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:58:08.605809] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1002, self="10.0.4.7:2890", log_id=220938544, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494433991433, last_flushed_end_lsn:{lsn:60468525403}}]) [2024-10-16 14:58:08.605832] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:211147565, max_lsn:204023140480, max_scn:{val:1729061888465975001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57922170954}, reuse_lsn:{lsn:204023140480}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:203981197440}}, last_submit_lsn:{lsn:204023140358}, last_submit_end_lsn:{lsn:204023140480}, last_submit_log_id:211147565, last_submit_log_pid:130, max_flushed_lsn:{lsn:204023140358}, max_flushed_end_lsn:{lsn:204023140480}, max_flushed_log_pid:130, committed_end_lsn:{lsn:204023140480}, last_slide_log_id:211147565, last_slide_scn:{val:1729061888465975001, v:0}, last_slide_lsn:{lsn:204023140358}, last_slide_end_lsn:{lsn:204023140480}, last_slide_log_pid:130, last_slide_log_accum_checksum:3784462130, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0f1ec530}, [StateMgr]={this:0x2b0e0f1f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:211147566, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261539368, palf_role_change_cb:0x2b0e0f1f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, reconfig_barrier:{prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_end_lsn:{lsn:57923794664}, prev_mode_pid:130}, persistent_config_version:{proposal_id:130, config_seq:17}, ms_ack_list:0, resend_config_version:{proposal_id:130, config_seq:17}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258231773, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258231773, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0f1edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, accepted_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261739861, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1729008439619722001, v:0}, min_block_id:3020, min_block_min_scn:{val:1729005678665693002, v:0}, base_lsn_for_block_gc:{lsn:203595866112}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:130}, log_config_meta:{version:3, proposal_id:130, prev:{version:1, config:{config_version:{proposal_id:129, config_seq:16}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:130, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:129, prev_lsn:{lsn:57923794542}, prev_mode_pid:130}, log_snapshot_meta:{version:1, base_lsn:{lsn:203595866112}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:130, mode_version:0, access_mode:1, ref_scn:{val:1717560293720964000, v:0}}}, log_meta_storage:{log_tail:{lsn:6397952}, readable_log_tail:{lsn:6397952}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:204065599488}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/meta", dir_fd:145, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:60706816, block_header_serialize_buf_:0x2b0e0f1f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:204023140480}, readable_log_tail:{lsn:204023140480}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:203998494720}, min_scn:{val:1729060878194301003, v:0}, curr_block_id:3040, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1001/log", dir_fd:173, min_block_id:3020, max_block_id:3041, curr_writable_block_id:3040}, logical_block_size_:67104768, curr_block_writable_size_:42459008, block_header_serialize_buf_:0x2b0e0f1f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261539352, this:0x2b0e0f1f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:130, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:129, majority_max_lsn:{lsn:57923794664}, saved_end_lsn:{lsn:57923794664}, last_submit_prepare_req_time_us:1722494261539574, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0f1f09b0}) [2024-10-16 14:58:08.605939] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=105] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888571595, last_flushed_end_lsn:{lsn:204023140480}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888572002, last_flushed_end_lsn:{lsn:204023140480}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888571756, last_flushed_end_lsn:{lsn:204023140480}}]) [2024-10-16 14:58:08.605947] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=211147566, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888571595, last_flushed_end_lsn:{lsn:204023140480}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888572002, last_flushed_end_lsn:{lsn:204023140480}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888571756, last_flushed_end_lsn:{lsn:204023140480}}]) [2024-10-16 14:58:08.605964] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:218497689, max_lsn:207589581856, max_scn:{val:1729061888165164004, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:57807896114}, reuse_lsn:{lsn:207589581856}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:207547638816}}, last_submit_lsn:{lsn:207589581734}, last_submit_end_lsn:{lsn:207589581856}, last_submit_log_id:218497689, last_submit_log_pid:81, max_flushed_lsn:{lsn:207589581734}, max_flushed_end_lsn:{lsn:207589581856}, max_flushed_log_pid:81, committed_end_lsn:{lsn:207589581856}, last_slide_log_id:218497689, last_slide_scn:{val:1729061888165164004, v:0}, last_slide_lsn:{lsn:207589581734}, last_slide_end_lsn:{lsn:207589581856}, last_slide_log_pid:81, last_slide_log_accum_checksum:4127653910, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e0db32530}, [StateMgr]={this:0x2b0e0db36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, leader:"10.0.4.9:2890", leader_epoch:7, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:57814812323}, scan_disk_log_finished:true, last_check_start_id:218497690, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433991656, palf_role_change_cb:0x2b0e0db3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, reconfig_barrier:{prev_log_proposal_id:79, prev_lsn:{lsn:57811902920}, prev_end_lsn:{lsn:57811903042}, prev_mode_pid:80}, persistent_config_version:{proposal_id:81, config_seq:20}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061885101776, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061885101776, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e0db33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, accepted_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1729009518284639001, v:0}, min_block_id:3073, min_block_min_scn:{val:1729006819670466001, v:0}, base_lsn_for_block_gc:{lsn:207152418816}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:81}, log_config_meta:{version:3, proposal_id:81, prev:{version:1, config:{config_version:{proposal_id:80, config_seq:19}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:81, config_seq:20}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:80, prev_lsn:{lsn:57814812201}, prev_mode_pid:81}, log_snapshot_meta:{version:1, base_lsn:{lsn:207152418816}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:81, mode_version:0, access_mode:1, ref_scn:{val:1717560293739274000, v:0}}}, log_meta_storage:{log_tail:{lsn:5660672}, readable_log_tail:{lsn:5660672}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207622152192}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/meta", dir_fd:180, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:61444096, block_header_serialize_buf_:0x2b0e0db389f8, flashback_version:0}, log_storage:{log_tail:{lsn:207589581856}, readable_log_tail:{lsn:207589581856}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:207555047424}, min_scn:{val:1729060516440786001, v:0}, curr_block_id:3093, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002/1003/log", dir_fd:182, min_block_id:3073, max_block_id:3094, curr_writable_block_id:3093}, logical_block_size_:67104768, curr_block_writable_size_:32570336, block_header_serialize_buf_:0x2b0e0db3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433990608, this:0x2b0e0db36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:80, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:79, majority_max_lsn:{lsn:57811903042}, saved_end_lsn:{lsn:57811903042}, last_submit_prepare_req_time_us:1722494433991756, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e0db369b0}) [2024-10-16 14:58:08.606056] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=91] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:58:08.606064] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=218497690, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494599977855, last_flushed_end_lsn:{lsn:57814812323}}]) [2024-10-16 14:58:08.611820] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5911, mem_used=12478464) [2024-10-16 14:58:08.637139] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061888637090000, start_id=1729061888637090000, end_id=1729061888637090001) [2024-10-16 14:58:08.639144] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061888638149000, start_id=1729061888638149000, end_id=1729061888638149001) [2024-10-16 14:58:08.639813] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=27] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061888070867002, v:0}, new_max_version={val:1729061888070867002, v:0}, last_min_version={val:1729061887664479004, v:0}, last_max_version={val:1729061887664479004, v:0}, total_time_us=1503, delta_us=568944) [2024-10-16 14:58:08.639829] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061888070867002, v:0}, min_version={val:1729061888070867002, v:0}, max_version={val:1729061888070867002, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:58:08.666000] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D63-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061888665614000, start_id=1729061888665614000, end_id=1729061888665614001) [2024-10-16 14:58:08.666109] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BADB-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061888665614000, start_id=1729061888665614000, end_id=1729061888665614001) [2024-10-16 14:58:08.666138] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D64-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061888665614000, start_id=1729061888665614000, end_id=1729061888665614001) [2024-10-16 14:58:08.666246] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D67-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061888665614000, start_id=1729061888665614000, end_id=1729061888665614001) [2024-10-16 14:58:08.666323] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D65-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061888665614000, start_id=1729061888665614000, end_id=1729061888665614001) [2024-10-16 14:58:08.672970] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:58:08.672988] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=16] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:58:08.672995] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:58:08.673002] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:08.675306] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=108, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}, {obj_id:616254, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616255, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616256, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:616257, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}]) [2024-10-16 14:58:08.675459] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=151] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:08.676889] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:08.680794] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061888679373000, start_id=1729061888679373000, end_id=1729061888679373001) [2024-10-16 14:58:08.681158] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061888680987000, start_id=1729061888680987000, end_id=1729061888680987001) [2024-10-16 14:58:08.685061] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=27][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.690719] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1001, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:109593988, max_lsn:13645451247, max_scn:{val:1729061888570581000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6198843846}, reuse_lsn:{lsn:13645451247}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:13603508207}}, last_submit_lsn:{lsn:13645451125}, last_submit_end_lsn:{lsn:13645451247}, last_submit_log_id:109593988, last_submit_log_pid:17, max_flushed_lsn:{lsn:13645451125}, max_flushed_end_lsn:{lsn:13645451247}, max_flushed_log_pid:17, committed_end_lsn:{lsn:13645451247}, last_slide_log_id:109593988, last_slide_scn:{val:1729061888570581000, v:0}, last_slide_lsn:{lsn:13645451125}, last_slide_end_lsn:{lsn:13645451247}, last_slide_log_pid:17, last_slide_log_accum_checksum:3778690568, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431ec530}, [StateMgr]={this:0x2b0e431f0830, self:"10.0.4.7:2890", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:109593989, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261552988, palf_role_change_cb:0x2b0e431f7230, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:14, lock_type:0, added_list:0, removed_list:0, type:"UNLOCK_CONFIG_CHANGE"}, state:0, checking_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_end_lsn:{lsn:9121864909}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:30}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:30}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494257649102, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494257649102, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431edc70}, [ModeMgr]={palf_id:1001, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261753456, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1728296971089337004, v:0}, min_block_id:189, min_block_min_scn:{val:1728240898542283004, v:0}, base_lsn_for_block_gc:{lsn:13555163136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:29}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:14, lock_time:1725269661734767}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:30}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:14, lock_time:1725269662094999}}, prev_log_proposal_id:17, prev_lsn:{lsn:9121864738}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:13555163136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266377168000, v:0}}}, log_meta_storage:{log_tail:{lsn:1011712}, readable_log_tail:{lsn:1011712}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13689372672}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/meta", dir_fd:198, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66093056, block_header_serialize_buf_:0x2b0e431f29f8, flashback_version:0}, log_storage:{log_tail:{lsn:13645451247}, readable_log_tail:{lsn:13645451247}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:13622267904}, min_scn:{val:1729042537979987002, v:0}, curr_block_id:203, palf_id:1001}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1001/log", dir_fd:200, min_block_id:189, max_block_id:204, curr_writable_block_id:203}, logical_block_size_:67104768, curr_block_writable_size_:43921425, block_header_serialize_buf_:0x2b0e431f46b8, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1722494261552391, this:0x2b0e431f0f70}, [Reconfirm]={palf_id:1001, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:6198988121}, saved_end_lsn:{lsn:6198988121}, last_submit_prepare_req_time_us:1722494261553147, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431f09b0}) [2024-10-16 14:58:08.690859] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=137] [PALF_DUMP](palf_id=1001, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888643518, last_flushed_end_lsn:{lsn:13645451247}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888643895, last_flushed_end_lsn:{lsn:13645451247}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888643798, last_flushed_end_lsn:{lsn:13645451247}}]) [2024-10-16 14:58:08.690871] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1001, self="10.0.4.7:2890", log_id=109593989, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888643518, last_flushed_end_lsn:{lsn:13645451247}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888643895, last_flushed_end_lsn:{lsn:13645451247}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061888643798, last_flushed_end_lsn:{lsn:13645451247}}]) [2024-10-16 14:58:08.691180] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E69-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:08.693257] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:58:08.694017] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061888693706000, start_id=1729061888693706000, end_id=1729061888693706001) [2024-10-16 14:58:08.699265] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:08.701592] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:58:08.703129] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.729867] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=19] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:08.732168] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=19] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:08.732128] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22859][T1004_L0_G2][T1004][YB4A0A000409-00061E9A39C4D906-0-0] [lt=61] get_min_unreplayed_log_info(lsn={lsn:14352407585}, scn={val:1729061888665614002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061888667720, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352407585}, next_to_submit_scn_:{val:1729061888665614002, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352407463}, end_lsn:{lsn:14352407585}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061888665614001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061888665614001, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:3919084961, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061888665614001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657223, miss_cnt:17, cache_read_size:8050197806, hit_ratio:"9.999997286825912379e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:58:08.743132] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=3851918, this={dio_aligned_buf:{buf_write_offset:2496, buf_padding_size:1600, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80963028, truncate_used_ts:29163628}, log_block_size:67108864, dir_fd:148, io_fd:171}, aligned_buf_len=4096, aligned_block_offset=3850240, buf_len=818, total_write_size_=99487, total_write_size_after_dio_=745472, ob_pwrite_used_ts=35142, count=158, each_pwrite_cost=222) [2024-10-16 14:58:08.743279] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:58:08.743289] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:58:08.743292] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:08.743333] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] get_max_applied_scn(scn={val:1729061888570581000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493510534}, palf_committed_end_scn_:{val:1729061888693706000, v:0}, last_check_scn_:{val:1729061888693706000, v:0}, max_applied_cb_scn_:{val:1729061888570581000, v:0}}) [2024-10-16 14:58:08.743353] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:58:08.744017] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=57] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900721265000, latest_log_ts:{val:1729061884067775001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:08.744110] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=39] get_max_applied_scn(scn={val:1729061888570581000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645451247}, palf_committed_end_scn_:{val:1729061888570581000, v:0}, last_check_scn_:{val:1729061888570581000, v:0}, max_applied_cb_scn_:{val:1729061888570581000, v:0}}) [2024-10-16 14:58:08.744139] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:58:08.744895] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=32] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:08.744978] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=39] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:08.744988] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:08.745703] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:08.746351] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=40] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:08.749780] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=21] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:58:08.761007] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061888760706000, start_id=1729061888760706000, end_id=1729061888760706001) [2024-10-16 14:58:08.761314] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=214, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:08.761330] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=59, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:08.761563] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:08.761583] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:08.761590] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:08.772691] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=8] Cache replace map node details(ret=0, replace_node_count=420, replace_time=0, replace_start_pos=2579474, replace_num=31457) [2024-10-16 14:58:08.772704] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=420, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:08.773272] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:08.776794] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DE-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:08.776819] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DE-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061888665614002, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061888665614002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:08.777873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9DE-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061888777295000, start_id=1729061888777295000, end_id=1729061888777295001) [2024-10-16 14:58:08.780360] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22493][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DE-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061888779412000, start_id=1729061888779412000, end_id=1729061888779412001) [2024-10-16 14:58:08.780417] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22493][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DE-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10338, avg_group_log_size=574) [2024-10-16 14:58:08.780427] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22493][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DE-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10338) [2024-10-16 14:58:08.780651] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10338) [2024-10-16 14:58:08.780678] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10338) [2024-10-16 14:58:08.780686] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:08.780927] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD308-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:08.780937] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD308-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=698, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:08.780942] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD308-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=29, avg_log_submit_to_flush_time=294, avg_log_submit_to_slide_time=669) [2024-10-16 14:58:08.780979] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:08.783109] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:08.784719] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15F4-0-0] [lt=18] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:08.785074] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.788068] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.788253] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.788265] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.788276] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.788598] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.788679] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=25] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.788957] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.789167] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.789487] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.789489] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.789617] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:08.791170] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E6B-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:08.791180] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E6B-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21223, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:08.803163] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.824760] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6B-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:08.824791] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6B-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061888665614001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061888665614001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:08.825877] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C6B-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061888824883000, start_id=1729061888824883000, end_id=1729061888824883001) [2024-10-16 14:58:08.828344] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C6B-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061888828238000, start_id=1729061888828238000, end_id=1729061888828238001) [2024-10-16 14:58:08.834030] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1001}, {id:1003}]) [2024-10-16 14:58:08.837110] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061888836629000, start_id=1729061888836629000, end_id=1729061888836629001) [2024-10-16 14:58:08.837749] INFO [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:194) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get wrs ts(ls_id={id:1}, delta=200608, log_service_ts=1729061888637090, min_tx_service_ts=4611686018427387, timestamp={val:1729061888637090000, v:0}) [2024-10-16 14:58:08.837763] INFO [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:214) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=0, Near_To_GTS_Cnt=49, Other_Error_Cnt=0, Submit_Succ_Cnt=51, last_scn="{val:1729061885634754001, v:0}", last_lsn={lsn:66390219025}, last_gts={val:1729061888637090000, v:0}, min_start_scn="{val:0, v:0}", min_start_status=1, sys_ls_end_scn={val:0, v:0}) [2024-10-16 14:58:08.839167] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061888838713000, start_id=1729061888838713000, end_id=1729061888838713001) [2024-10-16 14:58:08.862558] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=6583, mem_used=12478464) [2024-10-16 14:58:08.870662] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=12] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:08.871254] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=15] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:08.871299] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061884443293000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:58:08.873107] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:08.873136] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=196, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:08.873144] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=411, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:08.873154] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:08.880896] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061888880023000, start_id=1729061888880023000, end_id=1729061888880023001) [2024-10-16 14:58:08.881632] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=40] get number(ret=0, service_type_=0, range=1, base_id=1729061888880023000, start_id=1729061888880023000, end_id=1729061888880023001) [2024-10-16 14:58:08.885103] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.891657] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9345-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=424, avg_log_submit_to_slide_time=21225) [2024-10-16 14:58:08.891669] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:08.894566] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:08.899505] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:08.903223] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.912704] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:08.912726] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=17] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:58:08.912735] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:08.919657] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=40] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:08.920232] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=13] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:08.920272] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=9] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061881847501000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:58:08.926615] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=11] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:08.926838] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:08.942157] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=30] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:08.943416] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=39] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1758, avg_group_log_size=159) [2024-10-16 14:58:08.943430] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1758) [2024-10-16 14:58:08.943479] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:08.943489] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:08.943712] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1758) [2024-10-16 14:58:08.945744] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF2-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061888913448, keepalive_ts:-1}) [2024-10-16 14:58:08.945765] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF2-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061888913448, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:08.949711] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=25] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4271, mem_used=10398720) [2024-10-16 14:58:08.961010] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061888960998000, start_id=1729061888960998000, end_id=1729061888960998001) [2024-10-16 14:58:08.961725] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:08.967349] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FF0-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:08.967368] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FF0-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1333) [2024-10-16 14:58:08.967576] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=210, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:08.967597] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:08.967643] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:08.967873] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=24] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:08.967884] INFO [LIB] stat (utility.h:1165) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:08.974257] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=314, replace_time=0, replace_start_pos=2610931, replace_num=31457) [2024-10-16 14:58:08.974294] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=314, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:08.974955] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F443F-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=601, avg_log_submit_to_slide_time=1120) [2024-10-16 14:58:08.985225] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:08.987307] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=10] ====== tenant freeze timer task ====== [2024-10-16 14:58:08.990900] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1003, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:115173424, max_lsn:14352407951, max_scn:{val:1729061888665614004, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:6301879685}, reuse_lsn:{lsn:14352407951}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:14310464911}}, last_submit_lsn:{lsn:14352407829}, last_submit_end_lsn:{lsn:14352407951}, last_submit_log_id:115173424, last_submit_log_pid:17, max_flushed_lsn:{lsn:14352407829}, max_flushed_end_lsn:{lsn:14352407951}, max_flushed_log_pid:17, committed_end_lsn:{lsn:14352407951}, last_slide_log_id:115173424, last_slide_scn:{val:1729061888665614004, v:0}, last_slide_lsn:{lsn:14352407829}, last_slide_end_lsn:{lsn:14352407951}, last_slide_log_pid:17, last_slide_log_accum_checksum:1915673091, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e43d32530}, [StateMgr]={this:0x2b0e43d36830, self:"10.0.4.7:2890", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.9:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:6302436797}, scan_disk_log_finished:true, last_check_start_id:115173425, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494433837169, palf_role_change_cb:0x2b0e43d3d230, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"Invalid"}, state:0, checking_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, reconfig_barrier:{prev_log_proposal_id:15, prev_lsn:{lsn:6302238307}, prev_end_lsn:{lsn:6302238429}, prev_mode_pid:16}, persistent_config_version:{proposal_id:17, config_seq:26}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1729061885087605, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1729061885087605, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e43d33c70}, [ModeMgr]={palf_id:1003, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1728301961314447004, v:0}, min_block_id:199, min_block_min_scn:{val:1728247158357854003, v:0}, base_lsn_for_block_gc:{lsn:14293315584}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:17, config_seq:25}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:1, lock_owner:16, lock_time:1725269666117759}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:26}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:16, lock_time:1725269666314605}}, prev_log_proposal_id:17, prev_lsn:{lsn:9703181778}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:14293315584}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1717582266392644000, v:0}}}, log_meta_storage:{log_tail:{lsn:1064960}, readable_log_tail:{lsn:1064960}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14360420352}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/meta", dir_fd:192, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66039808, block_header_serialize_buf_:0x2b0e43d389f8, flashback_version:0}, log_storage:{log_tail:{lsn:14352407951}, readable_log_tail:{lsn:14352407951}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:14293315584}, min_scn:{val:1729013741169122001, v:0}, curr_block_id:213, palf_id:1003}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1003/log", dir_fd:207, min_block_id:199, max_block_id:214, curr_writable_block_id:213}, logical_block_size_:67104768, curr_block_writable_size_:8012401, block_header_serialize_buf_:0x2b0e43d3a6b8, flashback_version:0}, palf_epoch:4, last_purge_throttling_ts:1722494433836510, this:0x2b0e43d36f70}, [Reconfirm]={palf_id:1003, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:16, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:15, majority_max_lsn:{lsn:6302238429}, saved_end_lsn:{lsn:6302238429}, last_submit_prepare_req_time_us:1722494433837290, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e43d369b0}) [2024-10-16 14:58:08.991032] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=129] [PALF_DUMP](palf_id=1003, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:58:08.991043] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1003, self="10.0.4.7:2890", log_id=115173425, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1722494600485693, last_flushed_end_lsn:{lsn:6302436797}}]) [2024-10-16 14:58:08.993354] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:08.993380] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=217) [2024-10-16 14:58:08.996092] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:08.996110] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:08.996323] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=235, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:08.996338] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=51, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:08.996358] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:08.996588] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1143-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=309, avg_log_submit_to_slide_time=524) [2024-10-16 14:58:09.003279] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.036368] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] no need to update min/max version(new_version={val:1729061888637090000, v:0}, min_version={val:1729061888637090000, v:0}, max_version={val:1729061888637090000, v:0}) [2024-10-16 14:58:09.039356] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] submit log success(service_type_=0, last_id=1729061905590307000, limited_id=1729061905590307000) [2024-10-16 14:58:09.039369] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] submit log with lock success(service_type_=0, last_id=1729061905590307000, limited_id=1729061905590307000) [2024-10-16 14:58:09.039375] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061889039053000, start_id=1729061889039053000, end_id=1729061889039053001) [2024-10-16 14:58:09.040068] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=5] update id meta(ret=0, service_type=0, limited_id=1729061905590307000, latest_scn={val:1729061889039053001, v:0}, *this={tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061905590307000, latest_log_ts:{val:1729061889039053001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:58:09.040123] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=54] handle submit callback(service_type_=0, success=true, used_ts=767, log_ts={val:1729061889039053001, v:0}, limited_id=1729061905590307000) [2024-10-16 14:58:09.040133] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=10] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:58:09.040141] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=8] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:58:09.040419] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061889040257000, start_id=1729061889040257000, end_id=1729061889040257001) [2024-10-16 14:58:09.040826] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:58:09.040838] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151364800, tenant_id=1) [2024-10-16 14:58:09.040852] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:09.043567] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=2] ====== checkpoint timer task ====== [2024-10-16 14:58:09.043569] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061889043567, bytes: 4446825499249, bw: 0.367259 MB/s, add_ts: 1000713, add_bytes: 385374 [2024-10-16 14:58:09.044095] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD544-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:09.044105] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD544-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=611, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:09.044316] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=13] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.044340] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=24] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.044364] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:09.044517] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886446268000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900098662000, latest_log_ts:{val:1729061883567257001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:09.044565] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=45] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886446268000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900098662000, latest_log_ts:{val:1729061883567257001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:58:09.044607] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=32] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:58:09.044617] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061883567257001, v:0}) [2024-10-16 14:58:09.044621] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:58:09.044632] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:09.044645] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:58:09.044655] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:09.045356] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.045374] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=19] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.045389] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:58:09.045430] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886446268000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:09.045463] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886446268000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:58:09.045490] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:58:09.045497] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:58:09.045507] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:58:09.045512] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:58:09.046243] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.046261] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=17] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.046271] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:58:09.046308] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886446268000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:09.046341] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=31] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886446268000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:58:09.046367] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:58:09.046376] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:58:09.046385] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:58:09.046393] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:58:09.047098] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.047119] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=20] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.047132] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:58:09.047168] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886446268000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:09.047199] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=29] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886446268000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:58:09.047224] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=24] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:58:09.047233] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:58:09.047243] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:58:09.047252] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:58:09.047256] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:58:09.047279] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:09.047290] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:09.047303] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=292034612, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:58:09.059253] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=14] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:09.067704] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:09.067959] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD545-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:09.067974] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD545-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30073, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:09.071948] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061889071733000, start_id=1729061889071733000, end_id=1729061889071733001) [2024-10-16 14:58:09.071970] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:09.071995] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061889071991], gts:1729061889071733000, latest_srr:[mts=1729061889071991]}) [2024-10-16 14:58:09.081069] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061889080153000, start_id=1729061889080153000, end_id=1729061889080153001) [2024-10-16 14:58:09.081128] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:09.081140] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:09.081948] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061889081200000, start_id=1729061889081200000, end_id=1729061889081200001) [2024-10-16 14:58:09.082672] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7983-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1332) [2024-10-16 14:58:09.083542] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:09.085260] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.090957] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061885087605, last_first_register_time_us=1722494157071428, register_time_us=1729061885087605, curr_time_us=1729061889090956) [2024-10-16 14:58:09.090990] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061889090974, last_submit_register_req_time_us=1729061889090974, register_parent_reason=3) [2024-10-16 14:58:09.091000] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:09.096121] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3079, avg_group_log_size=307) [2024-10-16 14:58:09.099808] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:09.103340] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.105844] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=20] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061885101776, last_first_register_time_us=1722494150998907, register_time_us=1729061885101776, curr_time_us=1729061889105843) [2024-10-16 14:58:09.105871] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061889105859, last_submit_register_req_time_us=1729061889105859, register_parent_reason=3) [2024-10-16 14:58:09.105880] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:09.120973] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D88-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:09.121001] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D88-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061888665614004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061888665614004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:09.122039] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D88-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061889120996000, start_id=1729061889120996000, end_id=1729061889120996001) [2024-10-16 14:58:09.124610] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D88-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061889124171000, start_id=1729061889124171000, end_id=1729061889124171001) [2024-10-16 14:58:09.128424] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:09.128447] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:09.137204] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061889137131000, start_id=1729061889137131000, end_id=1729061889137131001) [2024-10-16 14:58:09.138566] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4984, mem_used=10398720) [2024-10-16 14:58:09.146484] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=186, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:09.146505] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:09.146517] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=589, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:09.146523] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=32, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:09.146540] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:09.146876] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE401A4-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:09.146884] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE401A4-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=965, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:09.150196] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:09.161218] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061889159254000, start_id=1729061889159254000, end_id=1729061889159254001) [2024-10-16 14:58:09.166446] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D68-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061889165641000, start_id=1729061889165641000, end_id=1729061889165641001) [2024-10-16 14:58:09.166513] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D6A-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061889165641000, start_id=1729061889165641000, end_id=1729061889165641001) [2024-10-16 14:58:09.166633] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D6C-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061889165641000, start_id=1729061889165641000, end_id=1729061889165641001) [2024-10-16 14:58:09.166667] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BADE-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061889165641000, start_id=1729061889165641000, end_id=1729061889165641001) [2024-10-16 14:58:09.167432] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD53FF4-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:09.175996] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=8] Cache replace map node details(ret=0, replace_node_count=382, replace_time=0, replace_start_pos=2642388, replace_num=31457) [2024-10-16 14:58:09.176013] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=382, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:09.185321] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.203398] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.205395] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:09.208310] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5047-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:09.208326] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5047-0-0] [lt=15] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:09.239436] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061889238651000, start_id=1729061889238651000, end_id=1729061889238651001) [2024-10-16 14:58:09.239510] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:09.239535] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:09.246250] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:09.246270] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:09.246765] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD549-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=22, avg_log_submit_to_flush_time=325, avg_log_submit_to_slide_time=593) [2024-10-16 14:58:09.260130] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:58:09.260890] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.260916] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.260937] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:09.261066] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900721265000, latest_log_ts:{val:1729061884067775001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:09.261112] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=43] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:09.261133] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:58:09.261142] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061884067775001, v:0}) [2024-10-16 14:58:09.261149] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.261157] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:09.261170] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:58:09.261181] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:09.261903] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.261919] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.261930] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:58:09.261998] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:09.262033] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=33] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:58:09.262061] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:58:09.262067] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:58:09.262077] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:58:09.262085] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:58:09.262818] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.262836] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=18] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.262851] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:58:09.262925] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:09.262958] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=32] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:58:09.262987] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=27] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:58:09.262993] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:58:09.263000] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:58:09.263006] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:58:09.263764] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.263780] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:09.263791] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:58:09.263861] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:09.263893] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=30] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061886069616004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:58:09.263917] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:58:09.263923] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:58:09.263931] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:58:09.263936] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:58:09.263940] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:58:09.264014] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E11-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061889263101000, start_id=1729061889263101000, end_id=1729061889263101001) [2024-10-16 14:58:09.264330] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4090) [2024-10-16 14:58:09.264622] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3666) [2024-10-16 14:58:09.264983] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061889264154000, start_id=1729061889264154000, end_id=1729061889264154001) [2024-10-16 14:58:09.265238] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1146-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:09.265250] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1146-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=603, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:09.267890] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B934C-0-0] [lt=34] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=249, avg_log_submit_to_slide_time=18343) [2024-10-16 14:58:09.270988] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=16] PNIO [ratelimit] time: 1729061889270987, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007615, add_bytes: 0 [2024-10-16 14:58:09.274552] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:09.274566] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:09.274990] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4447-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:09.274999] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4447-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=603, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:09.280663] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 145.75; ) [2024-10-16 14:58:09.280680] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:09.280715] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 133.50; ) [2024-10-16 14:58:09.280720] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:09.280728] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:09.281085] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061889279988000, start_id=1729061889279988000, end_id=1729061889279988001) [2024-10-16 14:58:09.282734] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061889281902000, start_id=1729061889281902000, end_id=1729061889281902001) [2024-10-16 14:58:09.285370] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.291696] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:58:09.295786] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=26] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:09.300118] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:09.303889] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1E06-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061889302713000, start_id=1729061889302713000, end_id=1729061889302713001) [2024-10-16 14:58:09.303913] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.304175] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=17] dump tenant info(tenant={id:1, tenant_meta:{unit:{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, super_block:{tenant_id:1, replay_start_point:ObLogCursor{file_id=12, log_id=1497048, offset=40358755}, ls_meta_entry:[2738](ver=0,mode=0,seq=7639021), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"3.000000000000000000e+00", unit_max_cpu:"3.000000000000000000e+00", total_worker_cnt:54, min_worker_cnt:32, max_worker_cnt:150, stopped:0, worker_us:1096711260762, recv_hp_rpc_cnt:5717699, recv_np_rpc_cnt:154802166, recv_lp_rpc_cnt:0, recv_mysql_cnt:62154189, recv_task_cnt:6664611, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:19623955298, workers:32, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=7 cnt[3]=0 cnt[4]=0 cnt[5]=228765716 cnt[6]=10625 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541241, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143148686}{group_id:10, queue_size:0, recv_req_cnt:19682155, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143048442}{group_id:20, queue_size:0, recv_req_cnt:401, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015260372185}{group_id:15, queue_size:0, recv_req_cnt:5821, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723554329}{group_id:5, queue_size:0, recv_req_cnt:35746269, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494262337428}{group_id:19, queue_size:0, recv_req_cnt:7245714, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061834968674}{group_id:9, queue_size:0, recv_req_cnt:15758150, min_worker_cnt:3, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:3, nesting_worker_cnt:0, token_change:1722494143222379}{group_id:1, queue_size:0, recv_req_cnt:153511602, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061854965187}, rpc_stat_info: pcode=0x51c:cnt=333 pcode=0x1502:cnt=218 pcode=0x717:cnt=185 pcode=0x150b:cnt=60 pcode=0x710:cnt=54, token_change_ts:1722494237022007, tenant_role:1}) [2024-10-16 14:58:09.304700] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=524] dump throttled time info(id_=1, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 10, group: OBCG_LOC_CACHE, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 9, group: OBCG_DETECT_RS, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1, tenant_throttled_time: 0;) [2024-10-16 14:58:09.304806] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851C5-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:09.304714] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=14] dump tenant info(tenant={id:508, tenant_meta:{unit:{tenant_id:508, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140910089, is_removed:false}, super_block:{tenant_id:508, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:15574151947, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:51691981, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6536510677, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:15, queue_size:0, recv_req_cnt:53884, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060722522359}, rpc_stat_info: pcode=0x5c0:cnt=96, token_change_ts:1722494140914497, tenant_role:0}) [2024-10-16 14:58:09.305144] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=428] dump throttled time info(id_=508, throttled_time_log=group_id: 15, group: OBCG_WR, throttled_time: 0;tenant_id: 508, tenant_throttled_time: 0;) [2024-10-16 14:58:09.305149] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=6] dump tenant info(tenant={id:509, tenant_meta:{unit:{tenant_id:509, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1722494140901337, is_removed:false}, super_block:{tenant_id:509, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:0, worker_us:12839528149, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6441848765, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1722494140916657, tenant_role:0}) [2024-10-16 14:58:09.305560] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=409] dump throttled time info(id_=509, throttled_time_log=tenant_id: 509, tenant_throttled_time: 0;) [2024-10-16 14:58:09.305565] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=5] dump tenant info(tenant={id:1001, tenant_meta:{unit:{tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1001, replay_start_point:ObLogCursor{file_id=11, log_id=1059563, offset=59267180}, ls_meta_entry:[7452](ver=0,mode=0,seq=7431948), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:468366017480, recv_hp_rpc_cnt:13589336, recv_np_rpc_cnt:126529993, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2784384336, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=9 cnt[3]=0 cnt[4]=0 cnt[5]=167012158 cnt[6]=3244 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541240, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494145075365}{group_id:20, queue_size:0, recv_req_cnt:306, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015263681355}{group_id:5, queue_size:0, recv_req_cnt:58923792, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726724812431820}{group_id:19, queue_size:0, recv_req_cnt:7662225, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061844973518}{group_id:1, queue_size:0, recv_req_cnt:227494154, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061885513575}, rpc_stat_info: pcode=0x1502:cnt=320 pcode=0x51c:cnt=218 pcode=0x717:cnt=178 pcode=0x710:cnt=89 pcode=0x150b:cnt=60, token_change_ts:1722494144838773, tenant_role:1}) [2024-10-16 14:58:09.306015] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=448] dump throttled time info(id_=1001, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1001, tenant_throttled_time: 0;) [2024-10-16 14:58:09.306024] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=9] dump tenant info(tenant={id:1002, tenant_meta:{unit:{tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, super_block:{tenant_id:1002, replay_start_point:ObLogCursor{file_id=191, log_id=47529733, offset=57901112}, ls_meta_entry:[10485](ver=0,mode=0,seq=7657455), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:998836139323, recv_hp_rpc_cnt:198478912, recv_np_rpc_cnt:379059159, recv_lp_rpc_cnt:0, recv_mysql_cnt:280948193, recv_task_cnt:146658, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:4103780966, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=673 cnt[3]=0 cnt[4]=0 cnt[5]=64429174 cnt[6]=226112 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131355453, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1728209719937557}{group_id:20, queue_size:0, recv_req_cnt:1673, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264869418}{group_id:15, queue_size:0, recv_req_cnt:5733, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723564477}{group_id:5, queue_size:0, recv_req_cnt:401894222, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1727167492226332}{group_id:19, queue_size:0, recv_req_cnt:1399891, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061856166239}{group_id:1, queue_size:0, recv_req_cnt:977162233, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061887423939}, rpc_stat_info: pcode=0x1502:cnt=408 pcode=0x1516:cnt=207 pcode=0x1501:cnt=206 pcode=0x717:cnt=181 pcode=0x150b:cnt=120, token_change_ts:1729004298741539, tenant_role:1}) [2024-10-16 14:58:09.306464] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=440] dump throttled time info(id_=1002, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1002, tenant_throttled_time: 0;) [2024-10-16 14:58:09.306472] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] dump tenant info(tenant={id:1003, tenant_meta:{unit:{tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1003, replay_start_point:ObLogCursor{file_id=11, log_id=1067656, offset=59494798}, ls_meta_entry:[3704](ver=0,mode=0,seq=7380334), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:503360125717, recv_hp_rpc_cnt:13609726, recv_np_rpc_cnt:126132717, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2783981104, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=4 cnt[3]=0 cnt[4]=0 cnt[5]=196318613 cnt[6]=6245 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:52541205, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722517512781421}{group_id:20, queue_size:0, recv_req_cnt:308, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015261556381}{group_id:5, queue_size:0, recv_req_cnt:63972041, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1726083164255294}{group_id:19, queue_size:0, recv_req_cnt:7660924, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061862930562}{group_id:1, queue_size:0, recv_req_cnt:228855218, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061872442043}, rpc_stat_info: pcode=0x1502:cnt=346 pcode=0x51c:cnt=254 pcode=0x717:cnt=182 pcode=0x710:cnt=98 pcode=0x150b:cnt=60, token_change_ts:1722494150252707, tenant_role:1}) [2024-10-16 14:58:09.306921] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=448] dump throttled time info(id_=1003, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1003, tenant_throttled_time: 0;) [2024-10-16 14:58:09.306929] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2429) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] dump tenant info(tenant={id:1004, tenant_meta:{unit:{tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, super_block:{tenant_id:1004, replay_start_point:ObLogCursor{file_id=197, log_id=49173916, offset=15367427}, ls_meta_entry:[12150](ver=0,mode=0,seq=7655944), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:false, version:4, snapshot_cnt:0}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:18, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:603329080209, recv_hp_rpc_cnt:12341824, recv_np_rpc_cnt:113755978, recv_lp_rpc_cnt:0, recv_mysql_cnt:11227, recv_task_cnt:1052, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2741365564, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=99 cnt[3]=0 cnt[4]=0 cnt[5]=72426725 cnt[6]=155203 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:2, queue_size:0, recv_req_cnt:131355311, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1722494154950936}{group_id:20, queue_size:0, recv_req_cnt:798, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729015264132685}{group_id:15, queue_size:0, recv_req_cnt:5423, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1729060723614999}{group_id:21, queue_size:0, recv_req_cnt:15, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269726417109}{group_id:5, queue_size:0, recv_req_cnt:31029737, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1724705543331737}{group_id:19, queue_size:0, recv_req_cnt:1325451, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061845553743}{group_id:16, queue_size:0, recv_req_cnt:18, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1725269721623294}{group_id:1, queue_size:0, recv_req_cnt:509701881, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1729061880166836}, rpc_stat_info: pcode=0x1502:cnt=396 pcode=0x1501:cnt=200 pcode=0x1516:cnt=194 pcode=0x717:cnt=183 pcode=0x150b:cnt=120, token_change_ts:1725266206494273, tenant_role:1}) [2024-10-16 14:58:09.307409] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1672) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=478] dump throttled time info(id_=1004, throttled_time_log=group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 21, group: OBCG_UNLOCK, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 16, group: OBCG_TRANSFER, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;tenant_id: 1004, tenant_throttled_time: 0;) [2024-10-16 14:58:09.312003] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DFB-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510357180397, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410655, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:09.314266] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=16] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:58:09.314961] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985E4-0-0] [lt=11] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=191) [2024-10-16 14:58:09.315061] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061889313443000, start_id=1729061889313443000, end_id=1729061889313443001) [2024-10-16 14:58:09.326391] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1044-0-0] [lt=9] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:58:09.330638] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1348, mem_used=4159488) [2024-10-16 14:58:09.336412] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=12] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:09.337224] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061889336345000, start_id=1729061889336345000, end_id=1729061889336345001) [2024-10-16 14:58:09.338101] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B28-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061889337464000, start_id=1729061889337464000, end_id=1729061889337464001) [2024-10-16 14:58:09.346529] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1089) [2024-10-16 14:58:09.346592] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1636) [2024-10-16 14:58:09.366878] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21658][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE8A8-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510357235272, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410656, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:09.374845] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=206, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:09.374867] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:09.374898] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:09.376040] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89B-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:09.376066] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89B-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061889165641001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061889165641001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:09.377537] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=365, replace_time=0, replace_start_pos=2673845, replace_num=31457) [2024-10-16 14:58:09.377569] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=365, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:09.377674] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA89B-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061889377397000, start_id=1729061889377397000, end_id=1729061889377397001) [2024-10-16 14:58:09.378707] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A27AFA89B-0-0] [lt=11] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.060951662063598633e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=303, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.065452636664601869e+03) [2024-10-16 14:58:09.378994] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABB-0-0] [lt=14] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:58:09.379018] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABB-0-0] [lt=21] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=25) [2024-10-16 14:58:09.380780] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89B-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061889380608000, start_id=1729061889380608000, end_id=1729061889380608001) [2024-10-16 14:58:09.380851] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89B-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=20, avg_log_batch_cnt=1, total_group_log_size=11585, avg_group_log_size=579) [2024-10-16 14:58:09.380862] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89B-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11585) [2024-10-16 14:58:09.381145] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=30] [PALF STAT WRITE LOG COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=278, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:09.381160] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11585) [2024-10-16 14:58:09.381171] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=87, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:09.381190] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11585) [2024-10-16 14:58:09.381200] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:09.381396] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A03-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:09.381404] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A03-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=885, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:09.381408] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A03-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=20, avg_log_gen_to_freeze_time=23, avg_log_gen_to_submit_time=50, avg_log_submit_to_flush_time=459, avg_log_submit_to_slide_time=834) [2024-10-16 14:58:09.381464] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=12] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:09.382941] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:09.385432] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.388766] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A11A789E1-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061889387991000, start_id=1729061889387991000, end_id=1729061889387991001) [2024-10-16 14:58:09.391445] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E77-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:09.391644] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:09.391660] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=112, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:09.391695] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:09.395919] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=16] gc stale ls task succ [2024-10-16 14:58:09.397077] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=10] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:58:09.403975] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.407684] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851C7-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:09.407706] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851C7-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497696, log_task={header:{begin_lsn:{lsn:207589582588}, end_lsn:{lsn:207589582710}, log_id:218497696, min_scn:{val:1729061889165641002, v:0}, max_scn:{val:1729061889165641002, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589582466}, prev_proposal_id:81, committed_end_lsn:{lsn:207589582588}, data_checksum:940009244, accum_checksum:1126432844, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061889304803, freeze_ts:1729061889304803, submit_ts:1729061889304817, flushed_ts:1729061889306904, gen_to_freeze cost time:0, gen_to_submit cost time:14, submit_to_flush cost time:2087}, fs_cb_begin_ts=1729061889407699, log_life_time=102896) [2024-10-16 14:58:09.440053] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061889439735000, start_id=1729061889439735000, end_id=1729061889439735001) [2024-10-16 14:58:09.463674] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] [COMPACTION DAG STATUS] (start_time=1729061879460999, end_time=1729061889463672, dag_status={}) [2024-10-16 14:58:09.463693] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=17] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:09.463700] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:09.463704] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:09.463708] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:09.463711] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:09.463714] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:09.463718] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:09.463721] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:09.463724] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:09.463728] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463734] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463739] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463742] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463746] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463750] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463753] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463757] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463761] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463764] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463768] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463772] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463776] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463780] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463784] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463787] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463792] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463797] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463801] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463805] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463808] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463812] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463816] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463820] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463824] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463827] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463831] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463835] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463839] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463843] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463846] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463850] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463854] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463858] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463861] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463865] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463869] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463873] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463877] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463883] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463886] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463890] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463894] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463899] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463903] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463907] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463914] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463917] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463921] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463928] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463932] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463935] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463942] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463946] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463953] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463957] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463961] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463968] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:09.463973] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:58:09.463979] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:58:09.463983] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:58:09.463986] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:58:09.463991] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:58:09.463993] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:58:09.463998] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:58:09.464000] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:58:09.464003] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:58:09.464009] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:58:09.464012] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21803][T1001_DagSchedu][T1001][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:58:09.465076] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061889464819000, start_id=1729061889464819000, end_id=1729061889464819001) [2024-10-16 14:58:09.466262] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [21810][T1001_TntShared][T1001][Y0-0000000000000000-0-0] [lt=14] clear dtl interm result cost(us)(clear_cost=15, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:58:09.468248] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:09.473173] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C124D-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061889472206000, start_id=1729061889472206000, end_id=1729061889472206001) [2024-10-16 14:58:09.475128] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2194, avg_group_log_size=199) [2024-10-16 14:58:09.475144] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2194) [2024-10-16 14:58:09.475472] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2194) [2024-10-16 14:58:09.475516] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:09.475870] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F444E-0-0] [lt=18] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=264, avg_log_submit_to_slide_time=629) [2024-10-16 14:58:09.481511] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061889479611000, start_id=1729061889479611000, end_id=1729061889479611001) [2024-10-16 14:58:09.482815] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE798B-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:09.483919] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=203, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:09.483934] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=161, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:09.484439] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061889483975000, start_id=1729061889483975000, end_id=1729061889483975001) [2024-10-16 14:58:09.486154] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.491288] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:58:09.491307] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:58:09.491319] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70961093, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:09.494618] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:09.494629] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=11] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:58:09.494632] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A30-0-0] [lt=7] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:09.494639] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:58:09.494645] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:58:09.494648] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:58:09.494645] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21204][SerScheQueue0][T0][YB4A0A000407-00061E9AE6BE9A30-0-0] [lt=12] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:09.494651] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:58:09.494654] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:58:09.494656] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:58:09.494659] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:58:09.494661] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:58:09.494675] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=14][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:58:09.495664] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9B-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:09.495682] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9B-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:09.496358] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3968) [2024-10-16 14:58:09.497045] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8D6-0-0] [lt=6] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:58:09.497063] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8D6-0-0] [lt=15] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:09.497082] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000408-00061E9B68BEB8D6-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:58:09.500399] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:09.504036] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.505158] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:09.516626] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=5] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:09.517263] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:09.517319] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061888741781000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:58:09.518085] INFO [STORAGE.TRANS] refresh_elr_tenant_config_ (ob_tx_elr_util.cpp:47) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C8B-0-0] [lt=11] refresh tenant config success(tenant_id=1004, *this={last_refresh_ts:1729061889517823, can_tenant_elr:true}) [2024-10-16 14:58:09.518243] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C8B-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061889517823000, start_id=1729061889517823000, end_id=1729061889517823001) [2024-10-16 14:58:09.518866] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD552-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=704, avg_log_submit_to_slide_time=883) [2024-10-16 14:58:09.519940] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EBC-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061889487644, keepalive_ts:-1}) [2024-10-16 14:58:09.519953] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EBC-0-0] [lt=12] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061889487644, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:09.519972] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EBD-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061889487684, keepalive_ts:-1}) [2024-10-16 14:58:09.519977] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EBD-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061889487684, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:09.523444] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=11] replay status is not follower(min_unreplayed_lsn={lsn:7385763025}, base_lsn={lsn:7314419712}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:58:09.523492] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=47] get_replay_process success(id={id:1}, replayed_log_size=71343313, unreplayed_log_size=0) [2024-10-16 14:58:09.523500] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:57923794664}, base_lsn={lsn:57911414784}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:58:09.523525] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=25] get_replay_process success(id={id:1001}, replayed_log_size=12379880, unreplayed_log_size=0) [2024-10-16 14:58:09.523539] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=11] get_replay_process success(id={id:1002}, replayed_log_size=152596881639, unreplayed_log_size=0) [2024-10-16 14:58:09.523547] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=5] get_replay_process success(id={id:1003}, replayed_log_size=149812377697, unreplayed_log_size=0) [2024-10-16 14:58:09.523563] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22151][T1002_ReplayPro][T1002][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1002, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=288479, last_replayed_log_size(MB)=288479, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:58:09.537515] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061889536499000, start_id=1729061889536499000, end_id=1729061889536499001) [2024-10-16 14:58:09.545538] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=911101952, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=162639872, wash_ratio=6, tenant_id=1001) [2024-10-16 14:58:09.545597] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=29] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:09.545614] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=15] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:09.545648] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=963407872, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=110333952, wash_ratio=6, tenant_id=1002) [2024-10-16 14:58:09.545685] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:09.545693] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:58:09.545721] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=6] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=929976320, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=143765504, wash_ratio=6, tenant_id=1003) [2024-10-16 14:58:09.545761] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=9] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:09.545769] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011877, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:09.545798] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=958939136, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=114802688, wash_ratio=6, tenant_id=1004) [2024-10-16 14:58:09.545831] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:09.545839] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:09.547591] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:58:09.547622] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:09.550135] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=17][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:09.565945] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601514, offset=9656650}, end_cursor:ObLogCursor{file_id=192, log_id=47601515, offset=9656934}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20c4050, len:1718, log_data_len:284, seq:47601514, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:09.567966] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4BF-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061889567221000, start_id=1729061889567221000, end_id=1729061889567221001) [2024-10-16 14:58:09.568792] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.569015] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601515, offset=9656934}, end_cursor:ObLogCursor{file_id=192, log_id=47601516, offset=9657208}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd201c050, len:1434, log_data_len:274, seq:47601515, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:09.569254] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.569627] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.569670] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.571628] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601516, offset=9657208}, end_cursor:ObLogCursor{file_id=192, log_id=47601517, offset=9657482}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20f8050, len:1160, log_data_len:274, seq:47601516, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:09.572056] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601517, offset=9657482}, end_cursor:ObLogCursor{file_id=192, log_id=47601518, offset=9657756}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2058050, len:886, log_data_len:274, seq:47601517, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:09.572110] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599145, miss_cnt:9, cache_read_size:149777947386, hit_ratio:"9.999999345926211536e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:09.572120] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561558, miss_cnt:1, cache_read_size:152590717589, hit_ratio:"9.999999928347030886e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:09.572326] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061889571647000, start_id=1729061889571647000, end_id=1729061889571647001) [2024-10-16 14:58:09.572350] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=501, get_gts_with_stc_cnt=77, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=75, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:09.572361] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061889572346], gts:1729061889571647000, latest_srr:[mts=1729061889572346]}) [2024-10-16 14:58:09.572531] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=523, get_gts_with_stc_cnt=6, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:09.579135] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=318, replace_time=0, replace_start_pos=2705302, replace_num=31457) [2024-10-16 14:58:09.579154] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=318, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:09.582911] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE798D-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:09.582937] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE798D-0-0] [lt=17] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:09.582944] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE798D-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=33619, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:09.582948] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE798D-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938552, log_task={header:{begin_lsn:{lsn:213058277607}, end_lsn:{lsn:213058277729}, log_id:220938552, min_scn:{val:1729061889165641003, v:0}, max_scn:{val:1729061889165641003, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058277485}, prev_proposal_id:88, committed_end_lsn:{lsn:213058277607}, data_checksum:1715074492, accum_checksum:4248279472, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061889482811, freeze_ts:1729061889482811, submit_ts:1729061889482831, flushed_ts:1729061889483943, gen_to_freeze cost time:0, gen_to_submit cost time:20, submit_to_flush cost time:1112}, fs_cb_begin_ts=1729061889582932, log_life_time=100121) [2024-10-16 14:58:09.583002] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:09.583013] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:09.583825] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=8] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257862, offset=39061256}, end_cursor:ObLogCursor{file_id=197, log_id=49257863, offset=39061539}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21a0050, len:2296, log_data_len:283, seq:49257862, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:09.585869] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB38A-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061889585631000, start_id=1729061889585631000, end_id=1729061889585631001) [2024-10-16 14:58:09.586222] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.586739] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257863, offset=39061539}, end_cursor:ObLogCursor{file_id=197, log_id=49257864, offset=39061821}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2044050, len:2013, log_data_len:282, seq:49257863, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:09.588931] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257864, offset=39061821}, end_cursor:ObLogCursor{file_id=197, log_id=49257865, offset=39062103}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2170050, len:1731, log_data_len:282, seq:49257864, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:09.589106] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257865, offset=39062103}, end_cursor:ObLogCursor{file_id=197, log_id=49257866, offset=39062385}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2110050, len:1449, log_data_len:282, seq:49257865, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:09.589166] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657232, miss_cnt:17, cache_read_size:8050198904, hit_ratio:"9.999997286826302068e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:09.589156] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650491, miss_cnt:32, cache_read_size:8050451298, hit_ratio:"9.999994892301218297e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:09.591541] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E7B-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:09.598184] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:09.598204] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:09.598632] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1001}, {id:1}, {id:1002}]) [2024-10-16 14:58:09.604166] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.605017] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851CB-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:09.605245] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=202, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:09.605273] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=210, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:09.605346] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:09.605624] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55E0-0-0] [lt=10] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:09.605631] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55E0-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10876, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:09.610874] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B32-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061889609815000, start_id=1729061889609815000, end_id=1729061889609815001) [2024-10-16 14:58:09.611545] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=46, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:58:09.611560] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=15] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=684, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:58:09.611564] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:58:09.611567] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=-613, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:58:09.611570] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=44, stat_interval=5000000, avg (count/cost)=118, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:58:09.611931] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5934, mem_used=12478464) [2024-10-16 14:58:09.614882] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1E05-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061889614014000, start_id=1729061889614014000, end_id=1729061889614014001) [2024-10-16 14:58:09.640070] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061889639494000, start_id=1729061889639494000, end_id=1729061889639494001) [2024-10-16 14:58:09.640787] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=30] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061889165641002, v:0}, new_max_version={val:1729061889165641002, v:0}, last_min_version={val:1729061889071733001, v:0}, last_max_version={val:1729061889071733001, v:0}, total_time_us=1501, delta_us=475145) [2024-10-16 14:58:09.640811] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=23] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061889165641002, v:0}, min_version={val:1729061889165641002, v:0}, max_version={val:1729061889165641002, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:58:09.665305] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061889664957000, start_id=1729061889664957000, end_id=1729061889664957001) [2024-10-16 14:58:09.666954] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D6D-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061889666032000, start_id=1729061889666032000, end_id=1729061889666032001) [2024-10-16 14:58:09.666974] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D6E-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061889666032000, start_id=1729061889666032000, end_id=1729061889666032001) [2024-10-16 14:58:09.667033] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0D70-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061889666032000, start_id=1729061889666032000, end_id=1729061889666032001) [2024-10-16 14:58:09.667126] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000409-00061E9A3969BAE4-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061889667117000, start_id=1729061889667117000, end_id=1729061889667117001) [2024-10-16 14:58:09.667200] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D71-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061889667117000, start_id=1729061889667117000, end_id=1729061889667117001) [2024-10-16 14:58:09.675203] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:09.675228] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:09.675265] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:09.675279] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:09.675543] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2194) [2024-10-16 14:58:09.675633] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4633-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:09.675644] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4633-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=639, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:09.676986] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:09.677278] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21726][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=21] BlockGCTimerTask success(ret=0, cost_time_us=8, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):131, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:58:09.681674] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061889679801000, start_id=1729061889679801000, end_id=1729061889679801001) [2024-10-16 14:58:09.683302] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:09.684540] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061889683884000, start_id=1729061889683884000, end_id=1729061889683884001) [2024-10-16 14:58:09.686285] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.691596] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E7D-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:09.698610] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:09.700714] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:09.701616] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=17] gc stale ls task succ [2024-10-16 14:58:09.704228] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:09.704246] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:09.704253] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.709554] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=12] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:58:09.710248] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061889708659000, start_id=1729061889708659000, end_id=1729061889708659001) [2024-10-16 14:58:09.716338] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:09.716363] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=13] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=218) [2024-10-16 14:58:09.720894] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DD-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061889688621, keepalive_ts:-1}) [2024-10-16 14:58:09.720911] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DD-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061889688621, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:09.730268] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B25-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061889730104000, start_id=1729061889730104000, end_id=1729061889730104001) [2024-10-16 14:58:09.730363] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:09.737542] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061889737503000, start_id=1729061889737503000, end_id=1729061889737503001) [2024-10-16 14:58:09.747625] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=8, avg_log_batch_cnt=1, total_group_log_size=976, avg_group_log_size=122) [2024-10-16 14:58:09.747649] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=967) [2024-10-16 14:58:09.761589] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:09.761624] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=24] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:09.761632] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:09.773333] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:09.780738] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=165, replace_time=0, replace_start_pos=2736759, replace_num=31457) [2024-10-16 14:58:09.780756] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=165, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:09.782711] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DF-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:09.782737] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9DF-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061889667117002, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061889667117002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:09.783209] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=15] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:09.783854] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9DF-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061889783290000, start_id=1729061889783290000, end_id=1729061889783290001) [2024-10-16 14:58:09.785731] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=17] LogLoopThread round_cost_time(us)(round_cost_time=9) [2024-10-16 14:58:09.786506] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.787923] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DF-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061889787493000, start_id=1729061889787493000, end_id=1729061889787493001) [2024-10-16 14:58:09.787985] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DF-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10551, avg_group_log_size=620) [2024-10-16 14:58:09.787994] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9DF-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10550) [2024-10-16 14:58:09.788013] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.788227] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=250, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:09.788261] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=33] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10551) [2024-10-16 14:58:09.788273] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=85, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:09.788288] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.788304] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.788318] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.788339] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10550) [2024-10-16 14:58:09.788352] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:09.788443] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD319-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:09.788449] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD319-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=729, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:09.788453] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD319-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=470, avg_log_submit_to_slide_time=696) [2024-10-16 14:58:09.788506] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:09.788634] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.788703] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.788955] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.789094] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.789446] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.789458] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.789694] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:09.792119] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9357-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:09.792142] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9357-0-0] [lt=23] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=467, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:09.804291] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.809102] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79DC-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061889800320, keepalive_ts:-1}) [2024-10-16 14:58:09.809118] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79DC-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061889800320, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:09.819637] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=16] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:09.829892] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:09.829908] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:58:09.829922] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52710639, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:09.831420] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6C-0-0] [lt=11] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:09.831446] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6C-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061889667117002, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061889667117002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:09.832506] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C6C-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061889831585000, start_id=1729061889831585000, end_id=1729061889831585001) [2024-10-16 14:58:09.834884] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=20] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1003}, {id:1001}, {id:1}]) [2024-10-16 14:58:09.835063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C6C-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061889834726000, start_id=1729061889834726000, end_id=1729061889834726001) [2024-10-16 14:58:09.836799] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061889609815000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=226979, in_cluster_service=true, cluster_version={val:1729061889536499000, v:0}, min_cluster_version={val:1729061889536499000, v:0}, max_cluster_version={val:1729061889536499000, v:0}, get_cluster_version_err=0, cluster_version_delta=300295, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65249177, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=387109509, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:09.840185] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061889840029000, start_id=1729061889840029000, end_id=1729061889840029001) [2024-10-16 14:58:09.862646] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6647, mem_used=12478464) [2024-10-16 14:58:09.865383] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061889864779000, start_id=1729061889864779000, end_id=1729061889864779001) [2024-10-16 14:58:09.875578] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=238, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:09.875563] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:58:09.875596] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=41, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:09.875620] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=34] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:09.881805] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061889881537000, start_id=1729061889881537000, end_id=1729061889881537001) [2024-10-16 14:58:09.884638] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061889883689000, start_id=1729061889883689000, end_id=1729061889883689001) [2024-10-16 14:58:09.886557] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.901018] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:09.904358] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.905328] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=34539983, this={dio_aligned_buf:{buf_write_offset:2633, buf_padding_size:1463, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111299814, truncate_used_ts:40614876}, log_block_size:67108864, dir_fd:182, io_fd:127}, aligned_buf_len=4096, aligned_block_offset=34537472, buf_len=122, total_write_size_=13472, total_write_size_after_dio_=430080, ob_pwrite_used_ts=20718, count=102, each_pwrite_cost=203) [2024-10-16 14:58:09.926691] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:09.927478] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:09.928097] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=23] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}) [2024-10-16 14:58:09.937661] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061889937471000, start_id=1729061889937471000, end_id=1729061889937471001) [2024-10-16 14:58:09.942988] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=15] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:09.946653] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26142][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB25CC6BC-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:58:09.947645] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1948, avg_group_log_size=216) [2024-10-16 14:58:09.947664] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1948) [2024-10-16 14:58:09.947699] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:09.947710] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:09.947953] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1948) [2024-10-16 14:58:09.948169] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=8, stat_interval=5000000, avg (count/cost)=22, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:58:09.948177] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=8] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=8, stat_interval=5000000, avg (count/cost)=1002, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:58:09.948180] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=8, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:58:09.948184] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=8, stat_interval=5000000, avg (count/cost)=-1048, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:58:09.948187] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=8, stat_interval=5000000, avg (count/cost)=-23, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:58:09.949813] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4302, mem_used=10398720) [2024-10-16 14:58:09.956742] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15F0-0-0] [lt=12] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:09.966389] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:09.968058] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54004-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:09.968079] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54004-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:09.968351] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=208, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:09.968374] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:09.968411] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:09.968685] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=22] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:09.968701] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:09.982216] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=160, replace_time=0, replace_start_pos=2768216, replace_num=31457) [2024-10-16 14:58:09.982250] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=160, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:09.983836] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7995-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938556, log_task={header:{begin_lsn:{lsn:213058278095}, end_lsn:{lsn:213058278217}, log_id:220938556, min_scn:{val:1729061889667117003, v:0}, max_scn:{val:1729061889667117003, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058277973}, prev_proposal_id:88, committed_end_lsn:{lsn:213058278095}, data_checksum:1715074492, accum_checksum:662476324, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061889883563, freeze_ts:1729061889883563, submit_ts:1729061889883566, flushed_ts:1729061889883773, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:207}, fs_cb_begin_ts=1729061889983829, log_life_time=100266) [2024-10-16 14:58:09.986621] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:09.986795] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC737-0-0] [lt=18] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:09.986812] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [106955][T1_L0_G19][T1][YB4A0A000409-00061E998BCBC737-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:09.992148] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B935B-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=218, avg_log_submit_to_slide_time=451) [2024-10-16 14:58:09.992162] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:10.004426] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.017540] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=6] ====== traversal_flush timer task ====== [2024-10-16 14:58:10.017653] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DB1-0-0] [lt=13] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061889883689000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061905590307000, latest_log_ts:{val:1729061889039053001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:10.017721] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DB1-0-0] [lt=64] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:10.040231] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061890040136000, start_id=1729061890040136000, end_id=1729061890040136001) [2024-10-16 14:58:10.043663] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061890043661, bytes: 4446825916942, bw: 0.398306 MB/s, add_ts: 1000094, add_bytes: 417693 [2024-10-16 14:58:10.048158] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD55F-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:10.048171] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD55F-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=518, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:10.064632] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1004, min_version={val:1729061889571647001, v:0}, base_version={val:1729061889165641004, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061889571647001, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061890064616, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061889571647001, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061889916348, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061889571647002, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061890040045, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:10.065308] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=13] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:10.065683] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=36] get number(ret=0, service_type_=0, range=1, base_id=1729061890065546000, start_id=1729061890065546000, end_id=1729061890065546001) [2024-10-16 14:58:10.066577] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=17, start_service_tstamp_=1722494263701883, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:10.068242] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:10.068486] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B935C-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:10.068503] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B935C-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=507, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:10.072718] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061890071906000, start_id=1729061890071906000, end_id=1729061890071906001) [2024-10-16 14:58:10.072744] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:10.072762] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061890072758], gts:1729061890071906000, latest_srr:[mts=1729061890072758]}) [2024-10-16 14:58:10.072777] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=533, get_gts_with_stc_cnt=82, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=78, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:10.075958] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C463F-0-0] [lt=25] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=429, avg_log_submit_to_slide_time=617) [2024-10-16 14:58:10.082092] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061890081613000, start_id=1729061890081613000, end_id=1729061890081613001) [2024-10-16 14:58:10.082156] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:10.082170] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:10.083212] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7997-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:10.084878] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061890083810000, start_id=1729061890083810000, end_id=1729061890083810001) [2024-10-16 14:58:10.086698] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.091532] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061886088435, last_first_register_time_us=1722494157071428, register_time_us=1729061886088435, curr_time_us=1729061890091530) [2024-10-16 14:58:10.091570] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061890091550, last_submit_register_req_time_us=1729061890091550, register_parent_reason=3) [2024-10-16 14:58:10.091579] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:58:10.094656] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:58:10.095416] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=15] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:10.095446] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=30] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:10.095469] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=18] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:10.095581] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=11] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061897008855000, latest_log_ts:{val:1729061880424064001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:10.095640] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=56] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:10.095654] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:58:10.095664] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061880424064001, v:0}) [2024-10-16 14:58:10.095671] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:10.095675] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:10.095686] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:10.095698] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:58:10.095709] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:10.095718] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:10.096710] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=24, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:10.096725] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=4937, avg_group_log_size=448) [2024-10-16 14:58:10.096740] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:10.096918] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=283, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:10.096938] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=22, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:10.096947] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=11, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:10.097139] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C114F-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=346, avg_log_submit_to_slide_time=607) [2024-10-16 14:58:10.100536] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:10.101308] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:10.104042] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Cache wash clean map node details(ret=0, clean_node_count=19, clean_time=0, clean_start_pos=1950334, clean_num=62914) [2024-10-16 14:58:10.104485] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.127041] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D89-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:10.127075] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D89-0-0] [lt=20] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061889667117004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061889667117004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:10.128693] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:10.128706] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:10.128781] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D89-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061890127941000, start_id=1729061890127941000, end_id=1729061890127941001) [2024-10-16 14:58:10.131320] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D89-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061890131088000, start_id=1729061890131088000, end_id=1729061890131088001) [2024-10-16 14:58:10.137806] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061890137331000, start_id=1729061890137331000, end_id=1729061890137331001) [2024-10-16 14:58:10.138628] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=5046, mem_used=10398720) [2024-10-16 14:58:10.147923] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=204, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:10.147939] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=45, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:10.149540] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=247, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:10.149558] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=230, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:10.149586] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:10.149605] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=695, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:10.149610] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:10.150247] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:10.160948] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=9] current monitor number(seq_=-1) [2024-10-16 14:58:10.167458] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D73-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061890167051000, start_id=1729061890167051000, end_id=1729061890167051001) [2024-10-16 14:58:10.167486] INFO [STORAGE.TRANS] handle_request (ob_timestamp_service.cpp:175) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D73-0-0] [lt=24] handle gts request statistics(total_rt=2761, total_cnt=448, avg_rt=6.149220489977728299e+00, avg_cnt=4.479999999999999716e+01) [2024-10-16 14:58:10.167491] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D74-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061890167051000, start_id=1729061890167051000, end_id=1729061890167051001) [2024-10-16 14:58:10.167534] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D72-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061890167051000, start_id=1729061890167051000, end_id=1729061890167051001) [2024-10-16 14:58:10.167730] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D76-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061890167051000, start_id=1729061890167051000, end_id=1729061890167051001) [2024-10-16 14:58:10.168306] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54008-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:10.183899] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=9] Cache replace map node details(ret=0, replace_node_count=187, replace_time=0, replace_start_pos=2799673, replace_num=31457) [2024-10-16 14:58:10.183917] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=187, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:10.184026] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get_max_applied_scn(scn={val:1729061890083810000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618806516}, palf_committed_end_scn_:{val:1729061890083810000, v:0}, last_check_scn_:{val:1729061890083810000, v:0}, max_applied_cb_scn_:{val:1729061890083810000, v:0}}) [2024-10-16 14:58:10.184047] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:58:10.185538] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:10.186748] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.204548] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.206241] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:10.209067] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5048-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:10.209084] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5048-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:10.212471] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5883-0-0] [lt=14] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:58:10.212618] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B592E5883-0-0] [lt=18][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:10.240279] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061890239411000, start_id=1729061890239411000, end_id=1729061890239411001) [2024-10-16 14:58:10.240343] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:10.240367] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:10.246927] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD14-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:10.247778] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:10.247807] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:10.248307] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD566-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=20, avg_log_submit_to_flush_time=265, avg_log_submit_to_slide_time=500) [2024-10-16 14:58:10.265529] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=49] get number(ret=0, service_type_=0, range=1, base_id=1729061890264587000, start_id=1729061890264587000, end_id=1729061890264587001) [2024-10-16 14:58:10.268695] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD567-0-0] [lt=26] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=238, avg_log_submit_to_slide_time=512) [2024-10-16 14:58:10.275543] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:10.275561] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:10.279263] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769ABA2-0-0] [lt=12] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12287, add_timestamp:1729061890279252}) [2024-10-16 14:58:10.279277] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6E-0-0] [lt=9] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12287, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:10.279407] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6E-0-0] [lt=22] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12287, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061890279395, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061890279395, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061890279395, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:10.279431] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6E-0-0] [lt=23] success to process renew task(task={tenant_id:1, table_id:12287, add_timestamp:1729061890279252}, locations=[{table_id:12287, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061890279395, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061890279395, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061890279395, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:10.281627] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061890081613000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=200009, in_cluster_service=true, cluster_version={val:1729061889881537000, v:0}, min_cluster_version={val:1729061889881537000, v:0}, max_cluster_version={val:1729061889881537000, v:0}, get_cluster_version_err=0, cluster_version_delta=400085, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244495, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=499351425, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:10.282435] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061890281562000, start_id=1729061890281562000, end_id=1729061890281562001) [2024-10-16 14:58:10.282582] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 237.75; ) [2024-10-16 14:58:10.282596] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:10.282639] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 133.00; ) [2024-10-16 14:58:10.282645] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:10.282654] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:10.282987] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=14] PNIO [ratelimit] time: 1729061890282986, bytes: 0, bw: 0.000000 MB/s, add_ts: 1011999, add_bytes: 0 [2024-10-16 14:58:10.284061] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769ABA2-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:10.285126] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=50] get number(ret=0, service_type_=0, range=1, base_id=1729061890283818000, start_id=1729061890283818000, end_id=1729061890283818001) [2024-10-16 14:58:10.286257] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=12] ls blacklist refresh finish(cost_time=7633) [2024-10-16 14:58:10.286805] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.292478] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:10.296674] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=45] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:58:10.296687] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:58:10.296690] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:10.297301] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061905590307000, latest_log_ts:{val:1729061889039053001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:10.297625] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4512) [2024-10-16 14:58:10.297650] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4936) [2024-10-16 14:58:10.298516] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45D1A-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:10.298527] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45D1A-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=710, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:10.304326] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:10.304664] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.305341] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851D9-0-0] [lt=23] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:10.314303] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22442][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=14][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:10.314356] WDIAG read_multi_row (ob_table_access_helper.h:238) [22442][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=14][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:10.315086] INFO delete_row (ob_table_access_helper.h:312) [22442][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1003, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061890314362) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061890314362) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:58:10.315121] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=11] ====== check clog disk timer task ====== [2024-10-16 14:58:10.315131] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:10.315139] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DFD-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510358183533, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410657, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:10.315145] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=373759872, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:10.330737] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1419, mem_used=4159488) [2024-10-16 14:58:10.336456] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=11] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:10.337927] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061890336474000, start_id=1729061890336474000, end_id=1729061890336474001) [2024-10-16 14:58:10.348076] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:58:10.348105] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1948) [2024-10-16 14:58:10.349489] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:58:10.349505] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=12582912, ctx.total_memstore_used_=46137344, ctx.total_memstore_hold_=46137344, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:58:10.349525] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:10.375850] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=33] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=303, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:10.375882] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=30] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:10.375904] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:58:10.376079] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4468-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:10.376090] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4468-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=667, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:10.381346] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:10.381366] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=220) [2024-10-16 14:58:10.383186] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89C-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:10.383210] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89C-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061890167051002, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061890167051002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:10.383643] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:10.384145] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061890083810000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=300328, in_cluster_service=true, cluster_version={val:1729061890083810000, v:0}, min_cluster_version={val:1729061890083810000, v:0}, max_cluster_version={val:1729061890083810000, v:0}, get_cluster_version_err=0, cluster_version_delta=300328, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244741, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=264780309, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:10.384499] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA89C-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061890383531000, start_id=1729061890383531000, end_id=1729061890383531001) [2024-10-16 14:58:10.385489] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A27AFA89C-0-0] [lt=26] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.061103057861328125e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=303, Throttle Trigger(MB)=181, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.065247953378918737e+03) [2024-10-16 14:58:10.385632] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=193, replace_time=0, replace_start_pos=2831130, replace_num=31457) [2024-10-16 14:58:10.385645] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=193, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:10.385700] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:58:10.385711] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54432140, tenant_id=1003) [2024-10-16 14:58:10.385723] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:10.386854] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.387586] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89C-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061890386881000, start_id=1729061890386881000, end_id=1729061890386881001) [2024-10-16 14:58:10.387673] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89C-0-0] [lt=20] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10323, avg_group_log_size=543) [2024-10-16 14:58:10.387684] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89C-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10324) [2024-10-16 14:58:10.387929] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=220, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:10.387946] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10323) [2024-10-16 14:58:10.387960] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=80, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:10.387972] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10324) [2024-10-16 14:58:10.387989] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:10.388242] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A16-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:10.388249] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A16-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=760, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:10.388253] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A16-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=357, avg_log_submit_to_slide_time=729) [2024-10-16 14:58:10.388309] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:10.392095] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E8B-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:10.392114] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E8B-0-0] [lt=11][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178743, log_task={header:{begin_lsn:{lsn:14538523058}, end_lsn:{lsn:14538523180}, log_id:115178743, min_scn:{val:1729061890167051001, v:0}, max_scn:{val:1729061890167051001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538522945}, prev_proposal_id:23, committed_end_lsn:{lsn:14538523058}, data_checksum:1715074492, accum_checksum:1460281172, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061890292013, freeze_ts:1729061890292013, submit_ts:1729061890292018, flushed_ts:1729061890292500, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:482}, fs_cb_begin_ts=1729061890392108, log_life_time=100095) [2024-10-16 14:58:10.392301] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=180, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:10.392320] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:10.392357] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:10.395978] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:10.399100] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=15] blacklist_loop exec finished(cost_time=15, is_enabled=true, send_cnt=0) [2024-10-16 14:58:10.404732] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.440520] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061890440441000, start_id=1729061890440441000, end_id=1729061890440441001) [2024-10-16 14:58:10.465693] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061890465211000, start_id=1729061890465211000, end_id=1729061890465211001) [2024-10-16 14:58:10.468774] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:10.475643] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=2217, avg_group_log_size=246) [2024-10-16 14:58:10.475658] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2217) [2024-10-16 14:58:10.475971] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:10.476181] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C464B-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=383, avg_log_submit_to_slide_time=561) [2024-10-16 14:58:10.476295] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=47, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:58:10.476310] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=15] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=1103, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:58:10.476355] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=44] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:58:10.476361] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=6] [APPLY STAT CB EXECUTE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=-614, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:58:10.476366] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=5] [APPLY STAT CB TOTAL TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=537, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:58:10.482643] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061890482269000, start_id=1729061890482269000, end_id=1729061890482269001) [2024-10-16 14:58:10.483519] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE799F-0-0] [lt=55] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:10.484176] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=29] no need to update min/max version(new_version={val:1729061890083810000, v:0}, min_version={val:1729061890083810000, v:0}, max_version={val:1729061890083810000, v:0}) [2024-10-16 14:58:10.486916] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.489938] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C125F-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061890489622000, start_id=1729061890489622000, end_id=1729061890489622001) [2024-10-16 14:58:10.496450] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9C-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:10.496470] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9C-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:10.496877] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4512) [2024-10-16 14:58:10.501694] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:10.504620] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:10.504756] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.505428] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851DD-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1333) [2024-10-16 14:58:10.534724] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] ====== check clog disk timer task ====== [2024-10-16 14:58:10.534745] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:58:10.534768] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444690029, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:10.538210] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] submit log success(service_type_=0, last_id=1729061907008855000, limited_id=1729061907008855000) [2024-10-16 14:58:10.538221] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] submit log with lock success(service_type_=0, last_id=1729061907008855000, limited_id=1729061907008855000) [2024-10-16 14:58:10.538226] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061890536200000, start_id=1729061890536200000, end_id=1729061890536200001) [2024-10-16 14:58:10.538803] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=2] update id meta(ret=0, service_type=0, limited_id=1729061907008855000, latest_scn={val:1729061890536200001, v:0}, *this={tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061907008855000, latest_log_ts:{val:1729061890536200001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:58:10.538838] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=35] handle submit callback(service_type_=0, success=true, used_ts=630, log_ts={val:1729061890536200001, v:0}, limited_id=1729061907008855000) [2024-10-16 14:58:10.538845] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=7] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:58:10.538850] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:58:10.538899] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3DF6-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061890538340000, start_id=1729061890538340000, end_id=1729061890538340001) [2024-10-16 14:58:10.539032] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=5] get_max_applied_scn(scn={val:1729061890336474000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390238283}, palf_committed_end_scn_:{val:1729061890536200002, v:0}, last_check_scn_:{val:1729061890536200002, v:0}, max_applied_cb_scn_:{val:1729061890336474000, v:0}}) [2024-10-16 14:58:10.539052] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:58:10.548271] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:58:10.548290] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:10.548496] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD56F-0-0] [lt=18] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=830, avg_log_submit_to_slide_time=986) [2024-10-16 14:58:10.564921] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=21] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061890071906002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=493008, in_cluster_service=true, cluster_version={val:1729061890071906001, v:0}, min_cluster_version={val:1729061890071906001, v:0}, max_cluster_version={val:1729061890071906001, v:0}, get_cluster_version_err=0, cluster_version_delta=493008, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228521, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=435956435, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:10.568810] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.569146] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=29] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.569622] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.569679] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.573117] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061890572872000, start_id=1729061890572872000, end_id=1729061890572872001) [2024-10-16 14:58:10.573302] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061890573294], gts:1729061890572872000, latest_srr:[mts=1729061890573294]}) [2024-10-16 14:58:10.573743] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601518, offset=9657756}, end_cursor:ObLogCursor{file_id=192, log_id=47601519, offset=9658040}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd217c050, len:612, log_data_len:284, seq:47601518, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:10.575471] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=17][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:10.576005] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22067][T1002_TeRec][T1002][YB4A0A000407-00061E9B225FA4C0-0-0] [lt=21] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=0, MAX_GAP=900000000000, new_sync_scn={val:1729061889472206000, v:0}, new_replayable_scn={val:1729061889472206000, v:0}, new_readable_scn={val:1729061889472206000, v:0}, old_tenant_info={tenant_id:1002, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061888465975000, v:0}, replayable_scn:{val:1729061888465975000, v:0}, standby_scn:{val:1729061888465975000, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"NOARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:58:10.576244] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4C0-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061890576054000, start_id=1729061890576054000, end_id=1729061890576054001) [2024-10-16 14:58:10.576654] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2339) [2024-10-16 14:58:10.577499] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601519, offset=9658040}, end_cursor:ObLogCursor{file_id=192, log_id=47601520, offset=9658314}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20bc050, len:4424, log_data_len:274, seq:47601519, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:10.580390] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=26] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601520, offset=9658314}, end_cursor:ObLogCursor{file_id=192, log_id=47601521, offset=9658588}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2030050, len:4150, log_data_len:274, seq:47601520, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:10.580665] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601521, offset=9658588}, end_cursor:ObLogCursor{file_id=192, log_id=47601522, offset=9658862}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2100050, len:3876, log_data_len:274, seq:47601521, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:10.580736] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561569, miss_cnt:1, cache_read_size:152590718922, hit_ratio:"9.999999928347037548e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:10.580774] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599155, miss_cnt:9, cache_read_size:149777948606, hit_ratio:"9.999999345926258165e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:10.583510] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79A1-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:10.583705] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:10.583726] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:10.583943] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55F3-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:10.583952] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55F3-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27679, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:10.583974] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:10.583980] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=5] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:10.585127] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061890583481000, start_id=1729061890583481000, end_id=1729061890583481001) [2024-10-16 14:58:10.586993] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.587476] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=185, replace_time=0, replace_start_pos=2862587, replace_num=31457) [2024-10-16 14:58:10.587511] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=185, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:10.590153] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257866, offset=39062385}, end_cursor:ObLogCursor{file_id=197, log_id=49257867, offset=39062668}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20b8050, len:1167, log_data_len:283, seq:49257866, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:10.592206] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E8F-0-0] [lt=31] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:10.592292] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB38B-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061890591550000, start_id=1729061890591550000, end_id=1729061890591550001) [2024-10-16 14:58:10.592833] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650502, miss_cnt:32, cache_read_size:8050452631, hit_ratio:"9.999994892302115357e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:10.593418] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=29] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257867, offset=39062668}, end_cursor:ObLogCursor{file_id=197, log_id=49257868, offset=39062950}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72fe050, len:884, log_data_len:282, seq:49257867, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:10.595508] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257868, offset=39062950}, end_cursor:ObLogCursor{file_id=197, log_id=49257869, offset=39063232}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2104050, len:602, log_data_len:282, seq:49257868, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:10.595782] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257869, offset=39063232}, end_cursor:ObLogCursor{file_id=197, log_id=49257870, offset=39063514}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2070050, len:4416, log_data_len:282, seq:49257869, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:10.595829] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657242, miss_cnt:17, cache_read_size:8050200124, hit_ratio:"9.999997286826735055e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:10.598057] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:10.598074] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:10.598688] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}, {id:1003}, {id:1002}, {id:1001}]) [2024-10-16 14:58:10.604775] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.605498] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851DF-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:10.605720] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=191, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:10.605733] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:58:10.605744] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:10.605797] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:10.605995] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55F4-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:10.606005] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D55F4-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=499, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:10.609505] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=12] new mysql sessid created(conn.sessid_=3221742036, support_ssl=false) [2024-10-16 14:58:10.609557] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=26] sm conn init succ(conn.sessid_=3221742036, sess.client_addr_="10.0.4.7:53482") [2024-10-16 14:58:10.609570] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=9] accept one succ(*s={this:0x2b0e2917e030, session_id:3221742036, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:203}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:10.609672] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:290) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=16] succeed to dispatch to tenant mysql queue(tenant_id=1) [2024-10-16 14:58:10.609681] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:295) [21317][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=9] mysql login queue(mysql_queue->queue_.size()=0) [2024-10-16 14:58:10.610333] INFO [SERVER] process (obmp_connect.cpp:502) [21581][T1_MysqlQueueTh][T1][Y0-00061E9B30EA2CA6-0-0] [lt=10] MySQL LOGIN(direct_client_ip="10.0.4.7", client_ip=10.0.4.7, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=%, sessid=3221742036, proxy_sessid=13882479996272574487, client_sessid=2107375639, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=4062023, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2024-10-16 14:58:10.610523] INFO [SERVER] process (obmp_init_db.cpp:53) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CA-0-0] [lt=3] init db(db_name=oceanbase) [2024-10-16 14:58:10.611989] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5947, mem_used=12478464) [2024-10-16 14:58:10.613031] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=12] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1001}) [2024-10-16 14:58:10.613046] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=15] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1002", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613053] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613057] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613060] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613754] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=4] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1}) [2024-10-16 14:58:10.613782] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=14] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:10.613791] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:10.613799] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:10.613810] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=10] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:10.613839] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=2] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1002}) [2024-10-16 14:58:10.613845] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"ocp", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613850] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=5] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613856] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=7] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613860] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613871] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1003}) [2024-10-16 14:58:10.613880] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=9] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"META$1004", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613883] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613886] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613889] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613895] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"BIGINT":1004}) [2024-10-16 14:58:10.613901] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"bddb", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613904] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"RANDOM", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613910] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=6] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"FULL{1}@myoceanbase1, FULL{1}@myoceanbase2, FULL{1}@myoceanbase3", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.613913] INFO [SERVER] inner_get_next_row (ob_agent_virtual_table.cpp:270) [21654][T1_L0_G0][T1][YB4A0A000407-00061E9B34FA85CE-0-0] [lt=3] mysql compat mode agent do not do convert(ret=0, ret="OB_SUCCESS", input={"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}) [2024-10-16 14:58:10.639775] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061890167051003, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=472717, in_cluster_service=true, cluster_version={val:1729061890071906001, v:0}, min_cluster_version={val:1729061890071906001, v:0}, max_cluster_version={val:1729061890071906001, v:0}, get_cluster_version_err=0, cluster_version_delta=567862, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244442, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=561969607, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:10.640724] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=38] get number(ret=0, service_type_=0, range=1, base_id=1729061890639178000, start_id=1729061890639178000, end_id=1729061890639178001) [2024-10-16 14:58:10.641464] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061890167051002, v:0}, new_max_version={val:1729061890167051002, v:0}, last_min_version={val:1729061890071906001, v:0}, last_max_version={val:1729061890071906001, v:0}, total_time_us=1641, delta_us=474411) [2024-10-16 14:58:10.641485] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061890167051002, v:0}, min_version={val:1729061890167051002, v:0}, max_version={val:1729061890167051002, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:58:10.641856] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [21446][T1_PalfGC][T1][Y0-0000000000000000-0-0] [lt=31] BlockGCTimerTask success(ret=0, cost_time_us=8, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):2048, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):342, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:58:10.665858] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061890665227000, start_id=1729061890665227000, end_id=1729061890665227001) [2024-10-16 14:58:10.667996] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000409-00061E9A3969BAEC-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061890667309000, start_id=1729061890667309000, end_id=1729061890667309001) [2024-10-16 14:58:10.668043] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A3969BAEF-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061890667309000, start_id=1729061890667309000, end_id=1729061890667309001) [2024-10-16 14:58:10.668096] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D79-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061890667309000, start_id=1729061890667309000, end_id=1729061890667309001) [2024-10-16 14:58:10.668104] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A3969BAED-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061890667309000, start_id=1729061890667309000, end_id=1729061890667309001) [2024-10-16 14:58:10.668246] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000409-00061E9A3969BAF0-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061890667309000, start_id=1729061890667309000, end_id=1729061890667309001) [2024-10-16 14:58:10.675785] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:10.675813] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:10.675840] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:10.675858] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:10.676107] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2339) [2024-10-16 14:58:10.676374] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4470-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:10.676421] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4470-0-0] [lt=46] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=630, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:10.677064] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:10.682763] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061890682248000, start_id=1729061890682248000, end_id=1729061890682248001) [2024-10-16 14:58:10.687046] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.692240] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E91-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:10.698253] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:58:10.698264] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:58:10.698268] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:10.698868] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061907008855000, latest_log_ts:{val:1729061890536200001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:10.701652] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD020-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061890692880, keepalive_ts:-1}) [2024-10-16 14:58:10.701667] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD020-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061890692880, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:10.701675] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:58:10.701685] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD021-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061890692925, keepalive_ts:-1}) [2024-10-16 14:58:10.701689] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD021-0-0] [lt=4] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061890692925, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:10.704905] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:10.705179] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.710696] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22141][T1002_TntShared][T1002][Y0-0000000000000000-0-0] [lt=22] clear dtl interm result cost(us)(clear_cost=913, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:58:10.720873] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3DFF-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510358589266, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410658, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:10.721759] WDIAG get_refactored (ob_hashtable.h:1102) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E03-0-0] [lt=15][errcode=-4201] hashtable not init [2024-10-16 14:58:10.721771] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E03-0-0] [lt=11][errcode=-4016] get cursor info failed(cursor_id=1121, get_sessid()=3221565766) [2024-10-16 14:58:10.721842] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E04-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061890721761000, start_id=1729061890721761000, end_id=1729061890721761001) [2024-10-16 14:58:10.730456] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:10.736908] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:10.736933] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=16] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=220) [2024-10-16 14:58:10.738187] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=42] get number(ret=0, service_type_=0, range=1, base_id=1729061890738058000, start_id=1729061890738058000, end_id=1729061890738058001) [2024-10-16 14:58:10.739549] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15F4-0-0] [lt=15] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:10.748116] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:10.748131] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:10.761700] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:10.761727] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=14] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:10.761735] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:10.773393] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:58:10.782473] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get_max_applied_scn(scn={val:1729061890682248000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114265034993}, palf_committed_end_scn_:{val:1729061890778626001, v:0}, last_check_scn_:{val:1729061890778626001, v:0}, max_applied_cb_scn_:{val:1729061890682248000, v:0}}) [2024-10-16 14:58:10.782493] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:58:10.783083] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=12] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:10.783609] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79A5-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938564, log_task={header:{begin_lsn:{lsn:213058279062}, end_lsn:{lsn:213058279184}, log_id:220938564, min_scn:{val:1729061890667309000, v:0}, max_scn:{val:1729061890667309000, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058278940}, prev_proposal_id:88, committed_end_lsn:{lsn:213058279062}, data_checksum:1715074492, accum_checksum:2297038018, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061890683509, freeze_ts:1729061890683509, submit_ts:1729061890683512, flushed_ts:1729061890683697, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:185}, fs_cb_begin_ts=1729061890783595, log_life_time=100086) [2024-10-16 14:58:10.783655] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:10.785042] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061890784983000, start_id=1729061890784983000, end_id=1729061890784983001) [2024-10-16 14:58:10.787080] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.788586] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.788611] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.789032] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=10] Cache replace map node details(ret=0, replace_node_count=144, replace_time=0, replace_start_pos=2894044, replace_num=31457) [2024-10-16 14:58:10.789048] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=144, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:10.789103] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.789117] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.789132] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.789149] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.789160] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.789174] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.789196] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.789467] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.789599] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:10.790251] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E0-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:10.790272] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E0-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061890667309000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061890667309000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:10.791360] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9E0-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061890791309000, start_id=1729061890791309000, end_id=1729061890791309001) [2024-10-16 14:58:10.792751] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B936B-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:10.792761] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B936B-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10542, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:10.794639] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E0-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061890794465000, start_id=1729061890794465000, end_id=1729061890794465001) [2024-10-16 14:58:10.794695] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E0-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9973, avg_group_log_size=664) [2024-10-16 14:58:10.794704] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E0-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9973) [2024-10-16 14:58:10.794925] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=288, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:10.794936] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9973) [2024-10-16 14:58:10.794945] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=80, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:10.795030] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9973) [2024-10-16 14:58:10.795045] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:10.795303] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD328-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:10.795310] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD328-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=726, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:10.795313] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD328-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=466, avg_log_submit_to_slide_time=694) [2024-10-16 14:58:10.795350] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:10.805216] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.818975] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:10.834951] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1003}, {id:1001}, {id:1}]) [2024-10-16 14:58:10.837700] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6D-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:10.837719] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6D-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061890667309001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061890667309001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:10.839620] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C6D-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061890838622000, start_id=1729061890838622000, end_id=1729061890838622001) [2024-10-16 14:58:10.840731] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061890839878000, start_id=1729061890839878000, end_id=1729061890839878001) [2024-10-16 14:58:10.844954] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C6D-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061890844335000, start_id=1729061890844335000, end_id=1729061890844335001) [2024-10-16 14:58:10.862704] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6708, mem_used=12478464) [2024-10-16 14:58:10.866049] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061890865467000, start_id=1729061890865467000, end_id=1729061890865467001) [2024-10-16 14:58:10.876113] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=218, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:10.876132] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=284, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:10.876142] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:10.876154] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:10.883407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=39] get number(ret=0, service_type_=0, range=1, base_id=1729061890883403000, start_id=1729061890883403000, end_id=1729061890883403001) [2024-10-16 14:58:10.887138] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.892012] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:58:10.892200] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4D-0-0] [lt=11] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061890572872002, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061888465975000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900098662000, latest_log_ts:{val:1729061883567257001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:10.892388] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4D-0-0] [lt=124] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061890572872002, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061888465975000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:10.892476] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4D-0-0] [lt=51] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061890667309002, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061888465975000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:10.892553] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4D-0-0] [lt=44] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061890668371001, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061888465975000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:10.892586] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4D-0-0] [lt=33] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:58:10.903256] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:10.905195] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:10.905284] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.913873] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:10.913890] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:58:10.913899] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:10.924557] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:10.925175] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:10.925222] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061887261758000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:58:10.926757] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=13] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:10.927566] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:10.937527] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] no need to update min/max version(new_version={val:1729061890536200002, v:0}, min_version={val:1729061890536200002, v:0}, max_version={val:1729061890536200002, v:0}) [2024-10-16 14:58:10.943754] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=19] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:10.947774] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF4-0-0] [lt=23] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061890915457, keepalive_ts:-1}) [2024-10-16 14:58:10.947790] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF4-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061890915457, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:10.948199] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:10.948217] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:10.948241] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:10.948251] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:10.949875] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4339, mem_used=10398720) [2024-10-16 14:58:10.950098] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:10.950114] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:10.950124] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=9] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=155) [2024-10-16 14:58:10.968664] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54018-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:10.968905] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=199, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:10.968929] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:10.968961] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:10.969137] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B936E-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:10.969201] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:10.969208] INFO [LIB] stat (utility.h:1165) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:10.986853] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061890986421000, start_id=1729061890986421000, end_id=1729061890986421001) [2024-10-16 14:58:10.987307] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:10.988417] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=13] ====== tenant freeze timer task ====== [2024-10-16 14:58:10.990656] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=192, replace_time=0, replace_start_pos=2925501, replace_num=31457) [2024-10-16 14:58:10.990672] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=192, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:10.992415] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E97-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=300, avg_log_submit_to_slide_time=22701) [2024-10-16 14:58:10.992463] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1089, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:11.005347] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.012433] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D56-0-0] [lt=12] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061891003623, keepalive_ts:-1}) [2024-10-16 14:58:11.012458] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D56-0-0] [lt=24] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061891003623, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:11.012480] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D57-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061891003676, keepalive_ts:-1}) [2024-10-16 14:58:11.012484] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D57-0-0] [lt=4] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061891003676, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:11.024483] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=14] LogLoopThread round_cost_time(us)(round_cost_time=9) [2024-10-16 14:58:11.024950] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061891024526000, start_id=1729061891024526000, end_id=1729061891024526001) [2024-10-16 14:58:11.038810] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061891037904000, start_id=1729061891037904000, end_id=1729061891037904001) [2024-10-16 14:58:11.041110] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061891040592000, start_id=1729061891040592000, end_id=1729061891040592001) [2024-10-16 14:58:11.043215] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:11.043230] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=11] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151364800, tenant_id=1) [2024-10-16 14:58:11.043244] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:11.043765] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=16] PNIO [ratelimit] time: 1729061891043765, bytes: 4446826335042, bw: 0.398690 MB/s, add_ts: 1000104, add_bytes: 418100 [2024-10-16 14:58:11.047799] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:58:11.047818] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=16] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:11.047830] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=292043889, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:58:11.048777] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:11.048788] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=538, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:11.048810] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:58:11.066013] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061891065802000, start_id=1729061891065802000, end_id=1729061891065802001) [2024-10-16 14:58:11.068836] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:11.069122] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9370-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:11.069134] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9370-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=463, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:11.073497] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061891073192000, start_id=1729061891073192000, end_id=1729061891073192001) [2024-10-16 14:58:11.073520] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061891073491], gts:1729061891073192000, latest_srr:[mts=1729061891073491]}) [2024-10-16 14:58:11.073532] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:11.077309] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4894-0-0] [lt=5] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061891044997, keepalive_ts:-1}) [2024-10-16 14:58:11.077323] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4894-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061891044997, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:11.077416] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C465E-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=37, avg_log_submit_to_flush_time=887, avg_log_submit_to_slide_time=1098) [2024-10-16 14:58:11.084234] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061891083811000, start_id=1729061891083811000, end_id=1729061891083811001) [2024-10-16 14:58:11.084295] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:11.084308] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:11.084401] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79AB-0-0] [lt=21] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:11.087307] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:11.087324] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:11.087330] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.096420] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=6] ====== traversal_flush timer task ====== [2024-10-16 14:58:11.096591] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4B-0-0] [lt=11] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061890771717002, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061888070867004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900721265000, latest_log_ts:{val:1729061884067775001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:11.096773] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4B-0-0] [lt=99] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061890771717002, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061888070867004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:11.096911] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4B-0-0] [lt=63] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061890667309005, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061888070867004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:11.097047] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4B-0-0] [lt=61] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061890667309003, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061888070867004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:11.097113] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4B-0-0] [lt=65] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:58:11.105505] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:11.105550] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.113502] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=99] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:11.114245] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=20] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:11.114316] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=14] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061890214512000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:58:11.127830] INFO [SQL.PC] need_check_schema_version (ob_plan_cache_value.cpp:2237) [22064][T1002_TenantInf][T1001][YB4A0A000407-00061E9B3B59B7EC-0-0] [lt=3] need check schema(new_schema_version=1722494878780592, cached_tenant_schema_version=1722494878780592, is_contain_synonym()=false, contain_sys_name_table_=true, is_contain_tmp_tbl()=false, is_contain_sys_pl_object()=false, pc_ctx.sql_ctx_.session_info_->get_has_temp_table_flag()=false, need_check=true, constructed_sql_=select ORA_ROWSCN, * from __all_tenant_info where tenant_id = ?) [2024-10-16 14:58:11.128815] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:11.128827] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:11.133657] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8A-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:11.133685] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8A-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061890667309005, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061890667309005, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:11.134726] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D8A-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061891134050000, start_id=1729061891134050000, end_id=1729061891134050001) [2024-10-16 14:58:11.137106] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D8A-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061891136103000, start_id=1729061891136103000, end_id=1729061891136103001) [2024-10-16 14:58:11.138080] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:11.138721] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5107, mem_used=10398720) [2024-10-16 14:58:11.148559] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=235, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:11.148576] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=47, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:11.148872] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=46, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:58:11.148898] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=26] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=817, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:58:11.148904] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=6] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:58:11.148907] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=-480, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:58:11.148911] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=15, stat_interval=5000000, avg (count/cost)=383, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:58:11.150305] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:58:11.168538] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D7D-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061891167704000, start_id=1729061891167704000, end_id=1729061891167704001) [2024-10-16 14:58:11.168855] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5401C-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:11.168935] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D7C-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061891168765000, start_id=1729061891168765000, end_id=1729061891168765001) [2024-10-16 14:58:11.168940] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D80-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061891168765000, start_id=1729061891168765000, end_id=1729061891168765001) [2024-10-16 14:58:11.168990] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D7E-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061891168765000, start_id=1729061891168765000, end_id=1729061891168765001) [2024-10-16 14:58:11.170617] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=7] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:11.170634] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=11] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:11.170642] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=7] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:58:11.170647] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=5] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:58:11.170651] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:58:11.170658] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=5] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16731) [2024-10-16 14:58:11.176410] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15F2-0-0] [lt=15] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:11.178794] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15EE-0-0] [lt=12] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:11.186966] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061891185491000, start_id=1729061891185491000, end_id=1729061891185491001) [2024-10-16 14:58:11.187399] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.192173] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=191, replace_time=0, replace_start_pos=2956958, replace_num=31457) [2024-10-16 14:58:11.192208] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=191, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:11.196756] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4] set tenant work area memory(tenant_id=1001, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:58:11.196811] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=14] set tenant work area memory(tenant_id=1002, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:58:11.196852] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7] set tenant work area memory(tenant_id=1003, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:58:11.196893] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5] set tenant work area memory(tenant_id=1004, wa_pctg=5, limit=157286400, ret=0) [2024-10-16 14:58:11.197033] EDIAG [SHARE.CONFIG] check_500_tenant_hold (ob_server_config.cpp:438) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7][errcode=0] the hold memory of tenant_500 is over the reserved memory(hold=1082318848, reserved=1073741824) BACKTRACE:0x1a5796b1 0x6847325 0x6a4dfba 0x6a4d9a4 0x6a4d8e0 0x6a4d707 0x1479497f 0x11dea49b 0x6809ea1 0x1a61b361 0x1a6176e7 0x2b0dadc8cea5 0x2b0dadf9ef3d [2024-10-16 14:58:11.197051] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:32) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=17] === Run print tenant memory usage task === [2024-10-16 14:58:11.197083] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:102) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4] ====== tenants memory info ====== === TENANTS MEMORY INFO === divisive_memory_used= 220,811,264 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,082,318,848 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 19,554,304 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 17,457,152 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 1 now= 1,729,061,891,196,675 active_memstore_used= 247,463,936 total_memstore_used= 245,366,784 total_memstore_hold= 245,366,784 memstore_freeze_trigger_limit= 151,364,800 memstore_limit= 858,993,440 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 1,986,498,560 max_mem_memstore_can_get_now= 756,824,034 memstore_alloc_pos= 20,002,635,776 memstore_frozen_pos= 19,755,171,840 memstore_reclaimed_pos= 19,757,268,992 [TENANT_MEMORY] tenant_id= 1,001 now= 1,729,061,891,196,675 active_memstore_used= 50,331,648 total_memstore_used= 46,137,344 total_memstore_hold= 46,137,344 memstore_freeze_trigger_limit= 54,012,700 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 911,101,952 max_mem_memstore_can_get_now= 270,063,594 memstore_alloc_pos= 32,719,765,504 memstore_frozen_pos= 32,669,433,856 memstore_reclaimed_pos= 32,673,628,160 [TENANT_MEMORY] tenant_id= 1,002 now= 1,729,061,891,196,675 active_memstore_used= 46,137,344 total_memstore_used= 46,137,344 total_memstore_hold= 46,137,344 memstore_freeze_trigger_limit= 28,230,020 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 963,407,872 max_mem_memstore_can_get_now= 141,150,194 memstore_alloc_pos= 465,953,619,968 memstore_frozen_pos= 465,907,482,624 memstore_reclaimed_pos= 465,907,482,624 [TENANT_MEMORY] tenant_id= 1,003 now= 1,729,061,891,196,675 active_memstore_used= 60,817,408 total_memstore_used= 60,817,408 total_memstore_hold= 60,817,408 memstore_freeze_trigger_limit= 54,432,140 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 929,976,320 max_mem_memstore_can_get_now= 272,160,758 memstore_alloc_pos= 34,273,755,136 memstore_frozen_pos= 34,212,937,728 memstore_reclaimed_pos= 34,212,937,728 [TENANT_MEMORY] tenant_id= 1,004 now= 1,729,061,891,196,675 active_memstore_used= 39,845,888 total_memstore_used= 39,845,888 total_memstore_hold= 39,845,888 memstore_freeze_trigger_limit= 27,418,600 memstore_limit= 429,496,720 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 958,939,136 max_mem_memstore_can_get_now= 137,093,088 memstore_alloc_pos= 1,602,224,128 memstore_frozen_pos= 1,562,378,240 memstore_reclaimed_pos= 1,562,378,240 [2024-10-16 14:58:11.197140] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:114) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=10] [CHUNK_MGR] limit= 12,884,901,888 hold= 7,743,684,608 total_hold= 8,633,974,784 used= 6,869,254,144 freelists_hold= 874,430,464 total_maps= 388,959 total_unmaps= 385,796 large_maps= 301 large_unmaps= 0 huge_maps= 200 huge_unmaps= 187 memalign=0 resident_size= 8,394,457,088 virtual_memory_used= 9,715,818,496 [CHUNK_MGR] 2 MB_CACHE: hold= 134,217,728 free= 64 pushes= 6,196,549,638 pops= 6,196,549,574 maps= 388,458 unmaps= 385,609 [CHUNK_MGR] 4 MB_CACHE: hold= 167,239,680 free= 73 pushes= 6,821,164 pops= 6,821,091 maps= 172 unmaps= 0 [CHUNK_MGR] 6 MB_CACHE: hold= 193,880,064 free= 46 pushes= 6,920 pops= 6,874 maps= 59 unmaps= 0 [CHUNK_MGR] 8 MB_CACHE: hold= 25,247,744 free= 4 pushes= 16,248,883 pops= 16,248,879 maps= 11 unmaps= 0 [CHUNK_MGR] 10 MB_CACHE: hold= 109,318,144 free= 13 pushes= 123 pops= 110 maps= 30 unmaps= 0 [CHUNK_MGR] 12 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [CHUNK_MGR] 14 MB_CACHE: hold= 26,435,584 free= 2 pushes= 4 pops= 2 maps= 9 unmaps= 0 [CHUNK_MGR] 16 MB_CACHE: hold= 33,316,864 free= 2 pushes= 4 pops= 2 maps= 2 unmaps= 0 [CHUNK_MGR] 18 MB_CACHE: hold= 184,774,656 free= 11 pushes= 993 pops= 982 maps= 18 unmaps= 0 [CHUNK_MGR] 20 MB_CACHE: hold= 0 free= 0 pushes= 0 pops= 0 maps= 0 unmaps= 0 [2024-10-16 14:58:11.197176] INFO print (ob_malloc_time_monitor.cpp:39) [21074][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8] [MALLOC_TIME_MONITOR] show the distribution of ob_malloc's cost_time [MALLOC_TIME_MONITOR] [ 0, 10): delta_total_cost_time= -573577969, delta_count= -2748721986, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 10, 100): delta_total_cost_time= -26270900, delta_count= -1676851, avg_cost_time= 15 [MALLOC_TIME_MONITOR] [ 100, 1000): delta_total_cost_time= -389582, delta_count= -2058, avg_cost_time= 189 [MALLOC_TIME_MONITOR] [ 1000, 10000): delta_total_cost_time= -274276, delta_count= -97, avg_cost_time= 2827 [MALLOC_TIME_MONITOR] [ 10000, 100000): delta_total_cost_time= -291366, delta_count= -15, avg_cost_time= 19424 [MALLOC_TIME_MONITOR] [ 100000, 1000000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 1000000, 9223372036854775807): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [2024-10-16 14:58:11.198595] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=42] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=24, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:11.198619] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=13, avg_log_batch_cnt=1, total_group_log_size=4340, avg_group_log_size=333) [2024-10-16 14:58:11.198642] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=20] [PALF STAT APPEND COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:11.198810] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=217, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:11.198828] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=130, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:11.198872] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=13, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:11.199168] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45D25-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=13, avg_log_gen_to_freeze_time=23, avg_log_gen_to_submit_time=53, avg_log_submit_to_flush_time=406, avg_log_submit_to_slide_time=697) [2024-10-16 14:58:11.205611] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.209993] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5049-0-0] [lt=8] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:11.210010] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD5049-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:11.241957] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061891240567000, start_id=1729061891240567000, end_id=1729061891240567001) [2024-10-16 14:58:11.242055] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:11.242085] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:11.243549] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061891243514000, start_id=1729061891243514000, end_id=1729061891243514001) [2024-10-16 14:58:11.245411] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:58:11.245433] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=19] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:58:11.245441] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=7] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:58:11.245448] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:11.247757] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] Dumping All Cache Objs(alloc_obj_list.count()=86, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}, {obj_id:78893917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:78893918, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893919, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893920, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}]) [2024-10-16 14:58:11.247964] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=204] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:11.248351] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:11.248368] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:11.248701] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=268, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:11.248725] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=340, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:11.248740] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:11.248967] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE401E0-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=311, avg_log_submit_to_slide_time=531) [2024-10-16 14:58:11.249014] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD584-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:11.249022] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD584-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=893, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:11.256059] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:58:11.256897] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061891256173000, start_id=1729061891256173000, end_id=1729061891256173001) [2024-10-16 14:58:11.266024] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061891265686000, start_id=1729061891265686000, end_id=1729061891265686001) [2024-10-16 14:58:11.269168] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B9374-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=236, avg_log_submit_to_slide_time=1194) [2024-10-16 14:58:11.276973] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:11.276989] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:11.284318] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061891283883000, start_id=1729061891283883000, end_id=1729061891283883001) [2024-10-16 14:58:11.284777] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:11.286210] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 6144.00, iops: 3.99, rt: 328.00; ) [2024-10-16 14:58:11.286234] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:11.286275] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 199.00; ) [2024-10-16 14:58:11.286283] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:11.286295] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:11.287270] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061891287269, bytes: 0, bw: 0.000000 MB/s, add_ts: 1004283, add_bytes: 0 [2024-10-16 14:58:11.287461] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.288883] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=9] replay status is not follower(min_unreplayed_lsn={lsn:45338084276}, base_lsn={lsn:45161508864}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:58:11.288935] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=52] get_replay_process success(id={id:1}, replayed_log_size=176575412, unreplayed_log_size=0) [2024-10-16 14:58:11.288953] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22408][T1003_ReplayPro][T1003][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1003, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=168, last_replayed_log_size(MB)=168, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:58:11.293307] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:11.304946] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=35] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:79, free_obj_hold_cnt:126, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:58:11.305670] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.305824] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:11.305924] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851ED-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:11.306389] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:11.313264] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:58:11.313378] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DB0-0-0] [lt=11] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061899029444000, latest_log_ts:{val:1729061882455772001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:11.313442] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DB0-0-0] [lt=61] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:11.319183] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E05-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510359187578, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410659, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:11.329055] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1045-0-0] [lt=15] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:58:11.330813] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1504, mem_used=4159488) [2024-10-16 14:58:11.336533] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:11.348721] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:11.348727] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:11.377301] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=226, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:11.377336] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=31] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=694, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:11.377355] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:11.377703] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4485-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:11.377712] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4485-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1181, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:11.379388] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABC-0-0] [lt=16] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:58:11.379403] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABC-0-0] [lt=11] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=17) [2024-10-16 14:58:11.380198] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E0B-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061891378892000, start_id=1729061891378892000, end_id=1729061891378892001) [2024-10-16 14:58:11.383664] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WRS] update tenant weak read server version(tenant_id=1003, server_version={version:{val:1729061891283883000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061891383630}, version_delta=99780) [2024-10-16 14:58:11.384542] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:11.387900] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [25332][T1_L0_G0][T1][YB4A0A000408-00061E99841D34BC-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510359256293, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410660, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:11.388100] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061891383723000, start_id=1729061891383723000, end_id=1729061891383723001) [2024-10-16 14:58:11.388172] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9670, avg_group_log_size=644) [2024-10-16 14:58:11.388189] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9832) [2024-10-16 14:58:11.388277] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.388423] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=383, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:11.388450] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=27] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9670) [2024-10-16 14:58:11.388461] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=128, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:11.388481] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9832) [2024-10-16 14:58:11.388495] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:11.388780] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A25-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:11.388793] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A25-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=955, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:11.388797] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A25-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=4, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=717, avg_log_submit_to_slide_time=921) [2024-10-16 14:58:11.390691] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89D-0-0] [lt=20] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:11.390710] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [41376][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89D-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061891174034001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061891174034001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:11.393788] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] Cache replace map node details(ret=0, replace_node_count=143, replace_time=0, replace_start_pos=2988415, replace_num=31457) [2024-10-16 14:58:11.393803] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=143, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:11.393987] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E9F-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:11.394005] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2E9F-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178753, log_task={header:{begin_lsn:{lsn:14538524278}, end_lsn:{lsn:14538524400}, log_id:115178753, min_scn:{val:1729061891168765001, v:0}, max_scn:{val:1729061891168765001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538524156}, prev_proposal_id:23, committed_end_lsn:{lsn:14538524278}, data_checksum:1715074492, accum_checksum:1677527228, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061891292586, freeze_ts:1729061891292586, submit_ts:1729061891292590, flushed_ts:1729061891293344, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:754}, fs_cb_begin_ts=1729061891393999, log_life_time=101413) [2024-10-16 14:58:11.394243] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:11.394256] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=102, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:11.394298] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:11.395996] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:11.396497] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA89D-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061891396059000, start_id=1729061891396059000, end_id=1729061891396059001) [2024-10-16 14:58:11.398829] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4341) [2024-10-16 14:58:11.398854] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4341) [2024-10-16 14:58:11.399026] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89D-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061891398162000, start_id=1729061891398162000, end_id=1729061891398162001) [2024-10-16 14:58:11.399109] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C115E-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:11.399116] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C115E-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=657, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:11.399585] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:11.405738] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.425577] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=17] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:11.428737] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=7] LogLoopThread round_cost_time(us)(round_cost_time=9) [2024-10-16 14:58:11.441537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061891441343000, start_id=1729061891441343000, end_id=1729061891441343001) [2024-10-16 14:58:11.442240] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get_max_applied_scn(scn={val:1729061891073192002, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959743586}, palf_committed_end_scn_:{val:1729061891073192003, v:0}, last_check_scn_:{val:1729061891073192003, v:0}, max_applied_cb_scn_:{val:1729061891073192002, v:0}}) [2024-10-16 14:58:11.442266] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=20] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:58:11.442321] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=40] get_max_applied_scn(scn={val:1729061891073192002, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023143774}, palf_committed_end_scn_:{val:1729061891073192003, v:0}, last_check_scn_:{val:1729061891073192003, v:0}, max_applied_cb_scn_:{val:1729061891073192002, v:0}}) [2024-10-16 14:58:11.442330] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:58:11.442354] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:58:11.442361] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=5] get_min_unreplayed_log_info(lsn={lsn:213058280038}, scn={val:1729061891168765003, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061891384816, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058280038}, next_to_submit_scn_:{val:1729061891168765003, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058279916}, end_lsn:{lsn:213058280038}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061891168765002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061891168765002, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:386188701, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061891168765002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561578, miss_cnt:1, cache_read_size:152590720020, hit_ratio:"9.999999928347041989e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:58:11.443423] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3AD1D0-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.731511650085449219e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=865, Throttle Trigger(MB)=519, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.167297611430484494e+01) [2024-10-16 14:58:11.443536] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061891443490000, start_id=1729061891443490000, end_id=1729061891443490001) [2024-10-16 14:58:11.466177] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061891465857000, start_id=1729061891465857000, end_id=1729061891465857001) [2024-10-16 14:58:11.469317] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:11.477023] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:11.477038] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:11.477386] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=46] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:58:11.477396] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=9] try gc retain ctx [2024-10-16 14:58:11.477399] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=2] try to retry start_working [2024-10-16 14:58:11.477583] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:11.477753] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4669-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=16, avg_log_submit_to_flush_time=963, avg_log_submit_to_slide_time=1241) [2024-10-16 14:58:11.478007] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061899029444000, latest_log_ts:{val:1729061882455772001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:11.484558] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061891482912000, start_id=1729061891482912000, end_id=1729061891482912001) [2024-10-16 14:58:11.484764] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79B3-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:11.488338] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.488757] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:58:11.488766] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=7] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:58:11.488772] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:58:11.488777] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:11.490980] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:58:11.491118] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=136] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:11.493652] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:58:11.493662] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:58:11.493674] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=70981247, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:11.497497] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9D-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:11.497509] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9D-0-0] [lt=11] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:11.498162] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1271-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061891497673000, start_id=1729061891497673000, end_id=1729061891497673001) [2024-10-16 14:58:11.505798] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.506154] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=47] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1190080512, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:11.506278] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851F1-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:11.509060] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=8] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:11.509070] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:58:11.509081] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:58:11.509075] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED6-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:11.509086] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:58:11.509090] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:58:11.509088] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED6-0-0] [lt=11] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:11.509092] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:58:11.509095] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:58:11.509097] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:58:11.509100] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:58:11.509102] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:58:11.509105] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:58:11.517987] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061881516345, end_time=1729061891517986, dag_status={}) [2024-10-16 14:58:11.518000] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:11.518008] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:11.518012] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:11.518015] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:11.518019] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:11.518036] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=17] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:11.518045] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:11.518048] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:11.518052] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:11.518055] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518061] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518065] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518069] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518073] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518076] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518080] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518084] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518088] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518092] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518095] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518100] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518104] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518107] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518111] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518115] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518119] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518123] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518127] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518131] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518135] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518139] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518142] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518146] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518150] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518154] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518161] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518165] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518169] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518173] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518176] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518180] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518184] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518188] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518192] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518195] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518199] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518203] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518207] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518215] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518219] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518222] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518226] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518230] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518234] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518238] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518241] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518245] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518249] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518252] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518256] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518260] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518263] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518267] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518271] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518274] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518278] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518282] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:11.518286] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:58:11.518289] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:58:11.518293] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:58:11.518295] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:58:11.518298] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:58:11.518300] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:58:11.518303] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:58:11.518305] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:58:11.518307] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:58:11.518310] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:58:11.518313] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22134][T1002_DagSchedu][T1002][Y0-0000000000000000-0-0] [lt=2] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:58:11.548956] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=25] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:11.549482] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE401EA-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:11.558661] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:11.566622] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:17493514547}], scn_array=[cnt:1, {val:1729061891073192004, v:0}], curr_size=122, accum_size=610, time_cost=18036) [2024-10-16 14:58:11.566679] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=2137, avg_log_submit_to_slide_time=2361) [2024-10-16 14:58:11.569149] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.569622] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.569639] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.573877] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061891573537000, start_id=1729061891573537000, end_id=1729061891573537001) [2024-10-16 14:58:11.573892] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] ts response task statistics(alloc_count=261365634, free_count=130682816) [2024-10-16 14:58:11.574620] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.581938] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601522, offset=9658862}, end_cursor:ObLogCursor{file_id=192, log_id=47601523, offset=9659146}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2024050, len:3602, log_data_len:284, seq:47601522, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:11.583959] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79B5-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:11.583984] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79B5-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:11.583990] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79B5-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=55371, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:11.584015] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561579, miss_cnt:1, cache_read_size:152590720142, hit_ratio:"9.999999928347041989e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:11.584055] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:11.584061] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:11.585642] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4C1-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061891583558000, start_id=1729061891583558000, end_id=1729061891583558001) [2024-10-16 14:58:11.585809] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=673, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:11.585838] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=457, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:11.586942] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601523, offset=9659146}, end_cursor:ObLogCursor{file_id=192, log_id=47601524, offset=9659420}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2008050, len:3318, log_data_len:274, seq:47601523, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:11.588381] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.590022] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061891589181000, start_id=1729061891589181000, end_id=1729061891589181001) [2024-10-16 14:58:11.591454] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601524, offset=9659420}, end_cursor:ObLogCursor{file_id=192, log_id=47601525, offset=9659694}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20d4050, len:3044, log_data_len:274, seq:47601524, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:11.591858] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601525, offset=9659694}, end_cursor:ObLogCursor{file_id=192, log_id=47601526, offset=9659968}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2238050, len:2770, log_data_len:274, seq:47601525, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:11.591942] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599164, miss_cnt:9, cache_read_size:149777949704, hit_ratio:"9.999999345926301464e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:11.595278] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=129, replace_time=0, replace_start_pos=3019872, replace_num=31457) [2024-10-16 14:58:11.595289] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=129, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:11.597961] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:11.597972] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:11.598083] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EA3-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:11.598099] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EA3-0-0] [lt=9][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178755, log_task={header:{begin_lsn:{lsn:14538524522}, end_lsn:{lsn:14538524644}, log_id:115178755, min_scn:{val:1729061891168765003, v:0}, max_scn:{val:1729061891168765003, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538524400}, prev_proposal_id:23, committed_end_lsn:{lsn:14538524522}, data_checksum:1715074492, accum_checksum:3840221726, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061891492617, freeze_ts:1729061891492617, submit_ts:1729061891492621, flushed_ts:1729061891493927, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1306}, fs_cb_begin_ts=1729061891598093, log_life_time=105476) [2024-10-16 14:58:11.598143] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650511, miss_cnt:32, cache_read_size:8050453729, hit_ratio:"9.999994892302849214e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:11.598749] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1003}, {id:1001}, {id:1}]) [2024-10-16 14:58:11.598871] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=4341) [2024-10-16 14:58:11.599175] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257870, offset=39063514}, end_cursor:ObLogCursor{file_id=197, log_id=49257871, offset=39063797}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db73a2050, len:4134, log_data_len:283, seq:49257870, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:11.601352] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB38C-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061891599439000, start_id=1729061891599439000, end_id=1729061891599439001) [2024-10-16 14:58:11.602796] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257871, offset=39063797}, end_cursor:ObLogCursor{file_id=197, log_id=49257872, offset=39064079}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd210c050, len:3851, log_data_len:282, seq:49257871, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:11.602947] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6F-0-0] [lt=11] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12274, vtable_type={tenant_id:1, type:4}) [2024-10-16 14:58:11.603072] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6F-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12274, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061891603061, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061891603061, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061891603061, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:11.603091] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E6F-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:12274, add_timestamp:1729061891602914}, locations=[{table_id:12274, partition_id:0, partition_cnt:3, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061891603061, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:3, replica_locations:[{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061891603061, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:3, replica_locations:[{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061891603061, sql_renew_time:0, is_mark_fail:false}]) [2024-10-16 14:58:11.604438] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB29C-0-0] [lt=10][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:58:11.606078] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851F3-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:11.606098] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851F3-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:11.606104] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851F3-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=11641, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:11.606199] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.606495] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=214, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:11.606506] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:11.606517] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=321, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:11.606542] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:11.608798] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257872, offset=39064079}, end_cursor:ObLogCursor{file_id=197, log_id=49257873, offset=39064361}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21cc050, len:3569, log_data_len:282, seq:49257872, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:11.609278] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257873, offset=39064361}, end_cursor:ObLogCursor{file_id=197, log_id=49257874, offset=39064643}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2258050, len:3287, log_data_len:282, seq:49257873, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:11.609411] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=53] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657251, miss_cnt:17, cache_read_size:8050201222, hit_ratio:"9.999997286827124743e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:11.609672] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:77) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B257FB29C-0-0] [lt=13][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008) [2024-10-16 14:58:11.612336] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5964, mem_used=12478464) [2024-10-16 14:58:11.642569] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061891641870000, start_id=1729061891641870000, end_id=1729061891641870001) [2024-10-16 14:58:11.643749] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061891642929000, start_id=1729061891642929000, end_id=1729061891642929001) [2024-10-16 14:58:11.644108] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1002, new_min_version={val:1729061891073192003, v:0}, new_max_version={val:1729061891073192003, v:0}, last_min_version={val:1729061891073192001, v:0}, last_max_version={val:1729061891073192001, v:0}, total_time_us=2422, delta_us=570914) [2024-10-16 14:58:11.644135] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=26] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1002, current_version={val:1729061891073192003, v:0}, min_version={val:1729061891073192003, v:0}, max_version={val:1729061891073192003, v:0}, in_service=true, leader_epoch=246) [2024-10-16 14:58:11.663832] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26147][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB18CC6EF-0-0] [lt=57] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:58:11.666313] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061891664565000, start_id=1729061891664565000, end_id=1729061891664565001) [2024-10-16 14:58:11.668995] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D81-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061891668540000, start_id=1729061891668540000, end_id=1729061891668540001) [2024-10-16 14:58:11.669136] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54026-0-0] [lt=16][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173450, log_task={header:{begin_lsn:{lsn:14352411001}, end_lsn:{lsn:14352411123}, log_id:115173450, min_scn:{val:1729061891174034003, v:0}, max_scn:{val:1729061891174034003, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352410879}, prev_proposal_id:17, committed_end_lsn:{lsn:14352411001}, data_checksum:940009244, accum_checksum:341238489, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061891569054, freeze_ts:1729061891569054, submit_ts:1729061891569059, flushed_ts:1729061891572825, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:3766}, fs_cb_begin_ts=1729061891669130, log_life_time=100076) [2024-10-16 14:58:11.670366] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D82-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061891669593000, start_id=1729061891669593000, end_id=1729061891669593001) [2024-10-16 14:58:11.670452] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D85-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061891669593000, start_id=1729061891669593000, end_id=1729061891669593001) [2024-10-16 14:58:11.670525] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D83-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061891669593000, start_id=1729061891669593000, end_id=1729061891669593001) [2024-10-16 14:58:11.670555] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0D84-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061891669593000, start_id=1729061891669593000, end_id=1729061891669593001) [2024-10-16 14:58:11.677142] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:11.677160] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:11.677158] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:11.677182] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:11.677190] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:11.677412] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1342) [2024-10-16 14:58:11.677449] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:11.677761] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F448F-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:11.677783] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F448F-0-0] [lt=22] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1303, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:11.684614] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061891684358000, start_id=1729061891684358000, end_id=1729061891684358001) [2024-10-16 14:58:11.684675] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79B7-0-0] [lt=5][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938573, log_task={header:{begin_lsn:{lsn:213058280160}, end_lsn:{lsn:213058280282}, log_id:220938573, min_scn:{val:1729061891168765004, v:0}, max_scn:{val:1729061891168765004, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058280038}, prev_proposal_id:88, committed_end_lsn:{lsn:213058280160}, data_checksum:1715074492, accum_checksum:1563043874, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061891583939, freeze_ts:1729061891583939, submit_ts:1729061891583943, flushed_ts:1729061891586622, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:2679}, fs_cb_begin_ts=1729061891684669, log_life_time=100730) [2024-10-16 14:58:11.688440] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.692734] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EA5-0-0] [lt=29] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:11.704909] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=8] gc stale ls task succ [2024-10-16 14:58:11.706236] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.706495] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:11.710385] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:79, free_obj_hold_cnt:126, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:58:11.730553] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=16] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:11.748682] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:11.748698] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:11.761743] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] gc stale ls task succ [2024-10-16 14:58:11.761769] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=14] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:11.761776] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:11.773453] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:58:11.777164] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] tx gc loop thread is running(MTL_ID()=1002) [2024-10-16 14:58:11.777176] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] try gc retain ctx [2024-10-16 14:58:11.777179] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:11.777787] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061889472206000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900098662000, latest_log_ts:{val:1729061883567257001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:11.778538] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=45] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061889472206000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:11.779343] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=56] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061889472206000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:11.780037] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=41] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1002, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061889472206000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:11.783205] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=11] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:11.784491] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:11.788108] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.788109] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.788122] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.788132] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.788453] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.788488] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15F6-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:11.788585] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15F8-0-0] [lt=8] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:11.788595] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.789005] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.789026] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.789038] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.789391] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1001, min_version={val:1729061891583558000, v:0}, base_version={val:1729061891383723000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061891589181000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061891789379, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061891583558000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061891657700, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061891589181000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061891714900, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:11.789638] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.790250] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061891789592000, start_id=1729061891789592000, end_id=1729061891789592001) [2024-10-16 14:58:11.790347] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.790369] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:11.794195] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=15] BlockGCTimerTask success(ret=0, cost_time_us=9, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1002", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1633, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:58:11.796350] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E1-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:11.796369] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E1-0-0] [lt=10] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061891762576000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061891762576000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:11.796784] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=153, replace_time=0, replace_start_pos=3051329, replace_num=31457) [2024-10-16 14:58:11.796801] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=153, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:11.797362] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9E1-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061891797060000, start_id=1729061891797060000, end_id=1729061891797060001) [2024-10-16 14:58:11.799893] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E1-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061891799182000, start_id=1729061891799182000, end_id=1729061891799182001) [2024-10-16 14:58:11.799960] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E1-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9444, avg_group_log_size=629) [2024-10-16 14:58:11.799976] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E1-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9444) [2024-10-16 14:58:11.800159] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=252, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:11.800175] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9444) [2024-10-16 14:58:11.800185] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=334, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:11.800200] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9444) [2024-10-16 14:58:11.800211] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:11.800390] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD337-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:11.800398] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD337-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=1043, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:11.800407] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD337-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=763, avg_log_submit_to_slide_time=1013) [2024-10-16 14:58:11.800456] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:11.806296] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.812027] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4815-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061891779679, keepalive_ts:-1}) [2024-10-16 14:58:11.812045] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4815-0-0] [lt=17] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061891779679, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:11.830004] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:11.830022] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=14] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:58:11.830038] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=52730056, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:11.835059] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1002}, {id:1003}]) [2024-10-16 14:58:11.843019] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061891842515000, start_id=1729061891842515000, end_id=1729061891842515001) [2024-10-16 14:58:11.844077] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061891842515000, start_id=1729061891842515000, end_id=1729061891842515001) [2024-10-16 14:58:11.847425] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6E-0-0] [lt=8] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:11.847450] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6E-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061891669593001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061891669593001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:11.848513] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C6E-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061891847608000, start_id=1729061891847608000, end_id=1729061891847608001) [2024-10-16 14:58:11.851510] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C6E-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061891850697000, start_id=1729061891850697000, end_id=1729061891850697001) [2024-10-16 14:58:11.862815] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6781, mem_used=12478464) [2024-10-16 14:58:11.866418] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061891865453000, start_id=1729061891865453000, end_id=1729061891865453001) [2024-10-16 14:58:11.866986] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=43, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:58:11.866998] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=904, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:58:11.867001] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:58:11.867005] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=-691, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:58:11.867008] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=45, stat_interval=5000000, avg (count/cost)=256, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:58:11.874644] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:11.875412] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:11.875475] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=19] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061884443293000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:58:11.876864] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26143][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB26CC6BC-0-0] [lt=21] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=true) [2024-10-16 14:58:11.877479] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=329, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:11.877494] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=743, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:11.877486] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:11.877510] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:11.884771] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061891884177000, start_id=1729061891884177000, end_id=1729061891884177001) [2024-10-16 14:58:11.888476] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.892877] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EA9-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:11.892893] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EA9-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=60195, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:11.892897] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EA9-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178758, log_task={header:{begin_lsn:{lsn:14538524888}, end_lsn:{lsn:14538525010}, log_id:115178758, min_scn:{val:1729061891669593001, v:0}, max_scn:{val:1729061891669593001, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538524766}, prev_proposal_id:23, committed_end_lsn:{lsn:14538524888}, data_checksum:1715074492, accum_checksum:3189888404, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061891792740, freeze_ts:1729061891792740, submit_ts:1729061891792744, flushed_ts:1729061891792975, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:231}, fs_cb_begin_ts=1729061891892871, log_life_time=100131) [2024-10-16 14:58:11.906389] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.906843] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1194295296, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:11.921431] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=6] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:11.921453] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=219) [2024-10-16 14:58:11.923285] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DDF-0-0] [lt=10] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:58:11.924184] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=38] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:11.924974] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=19] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:11.925031] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=15] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061891854891000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:58:11.926839] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:11.927667] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:11.929851] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=14] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):114, sequence:1}) [2024-10-16 14:58:11.944782] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=16] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:11.948756] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=14, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:11.948780] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:11.949998] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4379, mem_used=10398720) [2024-10-16 14:58:11.969271] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5402C-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:11.969306] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5402C-0-0] [lt=24] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:58:11.969565] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=281, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:11.969586] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=384, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:11.969636] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:11.969939] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:11.969950] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:11.988845] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:11.990469] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061891988943000, start_id=1729061891988943000, end_id=1729061891988943001) [2024-10-16 14:58:11.991310] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061891990645000, start_id=1729061891990645000, end_id=1729061891990645001) [2024-10-16 14:58:11.998260] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=120, replace_time=0, replace_start_pos=3082786, replace_num=31457) [2024-10-16 14:58:11.998299] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] replace map num details(ret=0, replace_node_count=120, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:11.999446] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D3E-0-0] [lt=64] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:11.999495] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000409-00061E9A11378D3E-0-0] [lt=45] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:12.006432] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.024733] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A15-0-0] [lt=3] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061892015948, keepalive_ts:-1}) [2024-10-16 14:58:12.024758] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A15-0-0] [lt=25] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061892015948, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:12.042729] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061892041877000, start_id=1729061892041877000, end_id=1729061892041877001) [2024-10-16 14:58:12.044086] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061892043935000, start_id=1729061892043935000, end_id=1729061892043935001) [2024-10-16 14:58:12.044152] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061892044150, bytes: 4446826789266, bw: 0.433015 MB/s, add_ts: 1000385, add_bytes: 454224 [2024-10-16 14:58:12.048789] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2036, avg_group_log_size=185) [2024-10-16 14:58:12.048811] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2036) [2024-10-16 14:58:12.048983] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1914) [2024-10-16 14:58:12.049393] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD59C-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:12.049406] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD59C-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1623, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:12.052202] INFO [CLOG] do_thread_task_ (ob_remote_fetch_log_worker.cpp:249) [21544][T1_RFLWorker][T1][YB4A0A000407-00061E996BC960C1-0-0] [lt=13] ObRemoteFetchWorker is running(thread_index=0) [2024-10-16 14:58:12.053542] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:58:12.053592] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22148][T1002_DupTbLeas][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1002 and flag like "%DUPLICATE%", tenant_id=1002, exec_tenant_id=1001, need_member_list=false) [2024-10-16 14:58:12.062155] INFO [CLOG] is_in_sync_ (palf_handle_impl.cpp:5230) [22059][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=8] is_in_sync(ret=0, palf_id=1002, is_in_sync=true, is_use_cache=true, remote_check_is_log_sync=false, leader_max_scn={val:18446744073709551615, v:3}, local_max_scn={val:1729061891669593003, v:0}) [2024-10-16 14:58:12.066637] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061892065975000, start_id=1729061892065975000, end_id=1729061892065975001) [2024-10-16 14:58:12.069488] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:12.069753] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD59D-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:12.069771] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD59D-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11278, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:12.074217] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061892073503000, start_id=1729061892073503000, end_id=1729061892073503001) [2024-10-16 14:58:12.074240] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:12.074262] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061892074258], gts:1729061892073503000, latest_srr:[mts=1729061892074258]}) [2024-10-16 14:58:12.077839] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C467B-0-0] [lt=27] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=619, avg_log_submit_to_slide_time=1034) [2024-10-16 14:58:12.084257] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79BF-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:12.085162] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061892084992000, start_id=1729061892084992000, end_id=1729061892084992001) [2024-10-16 14:58:12.085222] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:12.085244] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:12.088878] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.092999] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EAD-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=528, avg_log_submit_to_slide_time=54337) [2024-10-16 14:58:12.093040] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:12.106493] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.107094] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:12.107622] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061888104778, last_first_register_time_us=1722494150998907, register_time_us=1729061888104778, curr_time_us=1729061892107621) [2024-10-16 14:58:12.107651] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=14] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061892107637, last_submit_register_req_time_us=1729061892107637, register_parent_reason=3) [2024-10-16 14:58:12.107659] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:58:12.109955] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Cache wash clean map node details(ret=0, clean_node_count=29, clean_time=0, clean_start_pos=2013248, clean_num=62914) [2024-10-16 14:58:12.111626] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:12.128784] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:12.128804] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:12.138777] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5283, mem_used=10398720) [2024-10-16 14:58:12.139759] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8B-0-0] [lt=21] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:12.139788] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8B-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061891669593004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061891669593004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:12.140838] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D8B-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061892139861000, start_id=1729061892139861000, end_id=1729061892139861001) [2024-10-16 14:58:12.143313] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D8B-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061892143150000, start_id=1729061892143150000, end_id=1729061892143150001) [2024-10-16 14:58:12.149153] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1258, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:12.149172] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=35, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:12.150369] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:58:12.161111] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=16] current monitor number(seq_=-1) [2024-10-16 14:58:12.169375] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54030-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:12.169640] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D88-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061892169125000, start_id=1729061892169125000, end_id=1729061892169125001) [2024-10-16 14:58:12.169691] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D87-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061892169125000, start_id=1729061892169125000, end_id=1729061892169125001) [2024-10-16 14:58:12.169802] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D8A-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061892169125000, start_id=1729061892169125000, end_id=1729061892169125001) [2024-10-16 14:58:12.170093] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D86-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061892169125000, start_id=1729061892169125000, end_id=1729061892169125001) [2024-10-16 14:58:12.189004] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.190659] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061892188137000, start_id=1729061892188137000, end_id=1729061892188137001) [2024-10-16 14:58:12.198838] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=10] ====== traversal_flush timer task ====== [2024-10-16 14:58:12.198969] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D8B-0-0] [lt=17] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061907008855000, latest_log_ts:{val:1729061890536200001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:12.199026] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D8B-0-0] [lt=54] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:12.199184] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=21, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:12.199197] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3545, avg_group_log_size=354) [2024-10-16 14:58:12.199205] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:12.199384] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=234, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:12.199397] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=34, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:12.199411] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:12.199896] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=8] Cache replace map node details(ret=0, replace_node_count=60, replace_time=0, replace_start_pos=3114243, replace_num=31457) [2024-10-16 14:58:12.199912] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=60, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:12.206552] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.210818] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504A-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:12.210836] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504A-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:12.215622] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=7] Garbage Collector is running(seq_=656702, gc_interval=10000000) [2024-10-16 14:58:12.215651] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=14] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:12.215665] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=9] gc_check_member_list_ cost time(ret=0, time_us=28) [2024-10-16 14:58:12.215673] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:58:12.216362] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=6] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:12.216378] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=13] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:12.216387] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=8] execute_gc cost time(ret=0, time_us=9) [2024-10-16 14:58:12.216391] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22415][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:58:12.220344] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22788][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=17] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e3fdfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:12.242885] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061892241581000, start_id=1729061892241581000, end_id=1729061892241581001) [2024-10-16 14:58:12.242952] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=22, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:12.242967] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:12.246673] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061892245167000, start_id=1729061892245167000, end_id=1729061892245167001) [2024-10-16 14:58:12.247178] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=3] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=246, start_service_tstamp_=1722494263598349, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:12.247946] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=368, avg_log_submit_to_slide_time=1139) [2024-10-16 14:58:12.248919] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:12.248942] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:12.249124] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2079, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:12.249142] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=87, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:12.249174] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:12.249361] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5A1-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:12.249368] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5A1-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2362, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:12.249398] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5A2-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=20, avg_log_submit_to_flush_time=1309, avg_log_submit_to_slide_time=1595) [2024-10-16 14:58:12.266621] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061892265873000, start_id=1729061892265873000, end_id=1729061892265873001) [2024-10-16 14:58:12.269798] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5A3-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=705, avg_log_submit_to_slide_time=10534) [2024-10-16 14:58:12.277497] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=51] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:12.277512] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:12.285402] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061892284966000, start_id=1729061892284966000, end_id=1729061892284966001) [2024-10-16 14:58:12.289044] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.291309] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.98, rt: 192.75; ) [2024-10-16 14:58:12.291328] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:12.291373] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.98, rt: 756.75; ) [2024-10-16 14:58:12.291382] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:12.291393] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:12.293255] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:12.293838] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061892293837, bytes: 0, bw: 0.000000 MB/s, add_ts: 1006568, add_bytes: 0 [2024-10-16 14:58:12.299839] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107484][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D41-0-0] [lt=17] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:12.299865] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107484][T1003_L0_G19][T1003][YB4A0A000409-00061E9A11278D41-0-0] [lt=22] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:12.306615] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.306696] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE851FF-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:12.310217] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:12.315217] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:12.315235] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:12.315252] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=373770448, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:12.322297] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E07-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510360190691, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410661, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:12.324377] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [21909][T1001_SSTableGC][T1001][YB4A0A000407-00061E99777CB788-0-0] [lt=11] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60476}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015249374835000}, old_min_reserved_snapshot=1729060088741781000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060078735149000) [2024-10-16 14:58:12.330902] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1571, mem_used=4159488) [2024-10-16 14:58:12.334996] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [21909][T1001_SSTableGC][T1001][YB4A0A000407-00061E99777CB788-0-0] [lt=18] SSTableGCTask(cost_ts=10772) [2024-10-16 14:58:12.336609] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=14] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:12.349251] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:12.349310] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1914) [2024-10-16 14:58:12.350495] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:58:12.350513] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=12582912, ctx.total_memstore_used_=46137344, ctx.total_memstore_hold_=46137344, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:58:12.350532] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:12.353271] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EB-0-0] [lt=36] [LS_LOCATION]dump tenant ls location caches(tenant_id=1, tenant_ls_locations=[{cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061890006575, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:12.353295] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=12] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:12.353296] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EB-0-0] [lt=24] [LS_LOCATION]dump tenant ls location caches(tenant_id=1001, tenant_ls_locations=[{cache_key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061890007412, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:12.353310] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:12.353318] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:58:12.353321] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:58:12.353324] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:58:12.353327] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22799][T1004_ObTimer][T1004][Y0-0000000000000000-0-0] [lt=2] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=17121) [2024-10-16 14:58:12.353306] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EB-0-0] [lt=9] [LS_LOCATION]dump tenant ls location caches(tenant_id=1002, tenant_ls_locations=[{cache_key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061890008136, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061890008135, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061890008137, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}]}, {cache_key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061890008133, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:12.353333] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EB-0-0] [lt=27] [LS_LOCATION]dump tenant ls location caches(tenant_id=1003, tenant_ls_locations=[{cache_key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061890008761, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:12.353342] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [21249][DumpLSLoc][T0][YB4A0A000407-00061E9AB1ACC6EB-0-0] [lt=8] [LS_LOCATION]dump tenant ls location caches(tenant_id=1004, tenant_ls_locations=[{cache_key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, renew_time:1729061890009494, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, renew_time:1729061890009497, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, renew_time:1729061890009498, replica_locations:[{server:"10.0.4.7:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}]}, {cache_key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, renew_time:1729061890009496, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}, {server:"10.0.4.8:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.0.4.9:2890", role:2, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}]) [2024-10-16 14:58:12.377850] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=261, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:12.377875] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=43, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:12.377897] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:12.378106] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4683-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:12.378116] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4683-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1018, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:12.382126] INFO [RPC.OBRPC] do_client_loop (ob_net_keepalive.cpp:653) [21314][KeepAliveClient][T0][Y0-0000000000000000-0-0] [lt=2] dump dest keepalive data states, addr: "10.0.4.7:2890", last_write_ts_=1729061892181826, last_read_ts_=1729061891981585, in_black_=0 [2024-10-16 14:58:12.382143] INFO [RPC.OBRPC] do_client_loop (ob_net_keepalive.cpp:653) [21314][KeepAliveClient][T0][Y0-0000000000000000-0-0] [lt=12] dump dest keepalive data states, addr: "10.0.4.8:2890", last_write_ts_=1729061892181826, last_read_ts_=1729061891981585, in_black_=0 [2024-10-16 14:58:12.382150] INFO [RPC.OBRPC] do_client_loop (ob_net_keepalive.cpp:653) [21314][KeepAliveClient][T0][Y0-0000000000000000-0-0] [lt=4] dump dest keepalive data states, addr: "10.0.4.9:2890", last_write_ts_=1729061892181826, last_read_ts_=1729061891981585, in_black_=0 [2024-10-16 14:58:12.384782] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:12.384851] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:12.389106] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.390927] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061892388973000, start_id=1729061892388973000, end_id=1729061892388973001) [2024-10-16 14:58:12.391012] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9324, avg_group_log_size=666) [2024-10-16 14:58:12.391022] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9324) [2024-10-16 14:58:12.391362] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:12.391380] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=13] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54432140, tenant_id=1003) [2024-10-16 14:58:12.391392] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:12.391468] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=338, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:12.391482] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9324) [2024-10-16 14:58:12.391495] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=322, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:12.391515] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9324) [2024-10-16 14:58:12.391532] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:12.391628] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F96D7-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:12.391637] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F96D7-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=1254, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:12.391641] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F96D7-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=789, avg_log_submit_to_slide_time=1222) [2024-10-16 14:58:12.391693] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=16] [APPLY STAT CB APPEND COST TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=45, this=0x2b0e07718280, extra_info="") [2024-10-16 14:58:12.391699] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=5] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=932, this=0x2b0e077184b0, extra_info="") [2024-10-16 14:58:12.391702] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e077186e0, extra_info="") [2024-10-16 14:58:12.391705] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=-493, this=0x2b0e07718910, extra_info="") [2024-10-16 14:58:12.391708] INFO [LIB] stat (utility.h:1165) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=42, stat_interval=5000000, avg (count/cost)=484, this=0x2b0e07718b40, extra_info="") [2024-10-16 14:58:12.396058] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=9] gc stale ls task succ [2024-10-16 14:58:12.397105] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=11] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:58:12.399500] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3545) [2024-10-16 14:58:12.399528] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3545) [2024-10-16 14:58:12.399761] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45D31-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:12.399771] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45D31-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1234, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:12.399973] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=7] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=3145700, replace_num=31457) [2024-10-16 14:58:12.399986] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=10) [2024-10-16 14:58:12.401366] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89E-0-0] [lt=21] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:12.401391] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89E-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061892176519000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061892176519000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:12.402401] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA89E-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061892401728000, start_id=1729061892401728000, end_id=1729061892401728001) [2024-10-16 14:58:12.405052] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89E-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061892404908000, start_id=1729061892404908000, end_id=1729061892404908001) [2024-10-16 14:58:12.405645] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=3] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:12.406568] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:12.406678] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.429856] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183786542, max_lsn:114618828957, max_scn:{val:1729061892404908000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45685691230}, reuse_lsn:{lsn:114618828957}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114576885917}}, last_submit_lsn:{lsn:114618828202}, last_submit_end_lsn:{lsn:114618828957}, last_submit_log_id:183786542, last_submit_log_pid:33705, max_flushed_lsn:{lsn:114618828202}, max_flushed_end_lsn:{lsn:114618828957}, max_flushed_log_pid:33705, committed_end_lsn:{lsn:114618828957}, last_slide_log_id:183786542, last_slide_scn:{val:1729061892404908000, v:0}, last_slide_lsn:{lsn:114618828202}, last_slide_end_lsn:{lsn:114618828957}, last_slide_log_pid:33705, last_slide_log_accum_checksum:2278270575, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0df99e2530}, [StateMgr]={this:0x2b0df99e6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, leader:"10.0.4.7:2890", leader_epoch:6, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183786543, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261592457, palf_role_change_cb:0x2b0df99ed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, reconfig_barrier:{prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_end_lsn:{lsn:45686897415}, prev_mode_pid:33705}, persistent_config_version:{proposal_id:33705, config_seq:14}, ms_ack_list:0, resend_config_version:{proposal_id:33705, config_seq:14}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494258289572, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494258289572, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0df99e3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261792660, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729029056668344001, v:0}, min_block_id:1702, min_block_min_scn:{val:1729022447183332001, v:0}, base_lsn_for_block_gc:{lsn:114547838976}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33705}, log_config_meta:{version:3, proposal_id:33705, prev:{version:1, config:{config_version:{proposal_id:33704, config_seq:13}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33705, config_seq:14}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33704, prev_lsn:{lsn:45686897293}, prev_mode_pid:33705}, log_snapshot_meta:{version:1, base_lsn:{lsn:114547838976}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33705, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:233218048}, readable_log_tail:{lsn:233218048}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114682048512}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/meta", dir_fd:146, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:35201024, block_header_serialize_buf_:0x2b0df99e89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114618828957}, readable_log_tail:{lsn:114618828957}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114614943744}, min_scn:{val:1729061506448191001, v:0}, curr_block_id:1708, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1001/1/log", dir_fd:148, min_block_id:1702, max_block_id:1709, curr_writable_block_id:1708}, logical_block_size_:67104768, curr_block_writable_size_:63219555, block_header_serialize_buf_:0x2b0df99ea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261592575, this:0x2b0df99e6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33705, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33704, majority_max_lsn:{lsn:45686897415}, saved_end_lsn:{lsn:45686897415}, last_submit_prepare_req_time_us:1722494261592686, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0df99e69b0}) [2024-10-16 14:58:12.430000] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=141] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061892405421, last_flushed_end_lsn:{lsn:114618828957}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061892405537, last_flushed_end_lsn:{lsn:114618828957}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061892405571, last_flushed_end_lsn:{lsn:114618828957}}]) [2024-10-16 14:58:12.430011] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183786543, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061892405421, last_flushed_end_lsn:{lsn:114618828957}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061892405537, last_flushed_end_lsn:{lsn:114618828957}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061892405571, last_flushed_end_lsn:{lsn:114618828957}}]) [2024-10-16 14:58:12.443153] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061892442363000, start_id=1729061892442363000, end_id=1729061892442363001) [2024-10-16 14:58:12.446554] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3AD1DF-0-0] [lt=15] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.731511650085449219e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=865, Throttle Trigger(MB)=519, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.167297611430484494e+01) [2024-10-16 14:58:12.446746] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061892445748000, start_id=1729061892445748000, end_id=1729061892445748001) [2024-10-16 14:58:12.465971] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=16] no need to update min/max version(new_version={val:1729061891669593004, v:0}, min_version={val:1729061891669593004, v:0}, max_version={val:1729061891669593004, v:0}) [2024-10-16 14:58:12.470387] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:12.477618] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:12.477630] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:58:12.477908] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:12.485491] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] submit log success(service_type_=0, last_id=1729061909029444000, limited_id=1729061909029444000) [2024-10-16 14:58:12.485501] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] submit log with lock success(service_type_=0, last_id=1729061909029444000, limited_id=1729061909029444000) [2024-10-16 14:58:12.485506] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061892483852000, start_id=1729061892483852000, end_id=1729061892483852001) [2024-10-16 14:58:12.485861] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061909029444000, latest_scn={val:1729061892483852001, v:0}, *this={tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061909029444000, latest_log_ts:{val:1729061892483852001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:58:12.485906] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=44] handle submit callback(service_type_=0, success=true, used_ts=415, log_ts={val:1729061892483852001, v:0}, limited_id=1729061909029444000) [2024-10-16 14:58:12.485941] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=34] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:58:12.485948] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:58:12.486289] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061892485594000, start_id=1729061892485594000, end_id=1729061892485594001) [2024-10-16 14:58:12.489166] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.493193] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EB5-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:58:12.493388] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=254, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:12.493405] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=167, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:12.493439] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:12.498293] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9E-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:12.498313] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9E-0-0] [lt=19] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:12.505370] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1283-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061892505173000, start_id=1729061892505173000, end_id=1729061892505173001) [2024-10-16 14:58:12.506692] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85203-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1098) [2024-10-16 14:58:12.506757] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.510552] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:12.512733] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:12.517624] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DD-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061892508678, keepalive_ts:-1}) [2024-10-16 14:58:12.517638] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DD-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061892508678, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:12.520098] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=43] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:12.520727] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:12.520773] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=12] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061888741781000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:58:12.527151] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=34] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a81850, tx_id:{txid:199664647}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061892526163, active_ts:1729061892526163, commit_ts:-1, finish_ts:1729061892526163, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1729061922526158, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061892521915, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:58:12.527972] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:373) [22806][T1004_ArcSrv][T1003][YB4A0A000407-00061E9972A960C1-0-0] [lt=38] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x2b0e27404030}, tx={this:0x2b0ed9a6bc70, tx_id:{txid:199664648}, state:7, addr:"10.0.4.7:2890", tenant_id:1003, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1729061892527221, active_ts:1729061892527221, commit_ts:-1, finish_ts:1729061892527221, timeout_us:29999996, lock_timeout_us:-1, expire_ts:1729061922527217, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061892521915, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}) [2024-10-16 14:58:12.534831] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=11] ====== check clog disk timer task ====== [2024-10-16 14:58:12.534852] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4385) [2024-10-16 14:58:12.534879] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1444699649, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:12.545947] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=4] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=911101952, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=162639872, wash_ratio=6, tenant_id=1001) [2024-10-16 14:58:12.546004] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=25] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:12.546013] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:12.546048] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=963407872, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=110333952, wash_ratio=6, tenant_id=1002) [2024-10-16 14:58:12.546082] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:12.546087] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:58:12.546116] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=929976320, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=143765504, wash_ratio=6, tenant_id=1003) [2024-10-16 14:58:12.546157] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:12.546163] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011878, drift_size_=0, pre_drift_size=1498290, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:12.546213] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=8] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=958939136, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=114802688, wash_ratio=6, tenant_id=1004) [2024-10-16 14:58:12.546246] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:12.546251] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=0, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:12.549609] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5AB-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:12.569159] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.569543] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.569564] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.569580] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.574607] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061892573877000, start_id=1729061892573877000, end_id=1729061892573877001) [2024-10-16 14:58:12.574640] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=19] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061892574634], gts:1729061892573877000, latest_srr:[mts=1729061892574634]}) [2024-10-16 14:58:12.579765] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:954) [22405][T1003_DupTbLeas][T1003][Y0-0000000000000000-0-0] [lt=12][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2024-10-16 14:58:12.579839] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:673) [22405][T1003_DupTbLeas][T1003][Y0-0000000000000000-0-0] [lt=13][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1003 and flag like "%DUPLICATE%", tenant_id=1003, exec_tenant_id=1, need_member_list=false) [2024-10-16 14:58:12.580879] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4689-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=295, avg_log_submit_to_slide_time=641) [2024-10-16 14:58:12.584588] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79C9-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:12.584672] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79C9-0-0] [lt=78] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:12.585012] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44AB-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:12.585023] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44AB-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=18642, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:12.585027] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561590, miss_cnt:1, cache_read_size:152590721484, hit_ratio:"9.999999928347047540e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:12.585080] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=46, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:12.585089] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=8] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=21, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:12.585724] INFO [CLOG] push_log_replay_task (ob_replay_status.cpp:1147) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=5] submit pre barrier log success(log_type=17, ls_id={id:1002}, lsn={lsn:14538525920}, scn={val:1729061892169125004, v:0}, is_pre_barrier=true, is_post_barrier=false, log_size=24) [2024-10-16 14:58:12.585727] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:957) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] wait for barrier(ret=-4023, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:33, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061892585683, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538525976}, next_to_submit_scn_:{val:1729061892169125005, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538525864}, end_lsn:{lsn:14538525976}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061892169125004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061892169125004, v:0}, data_checksum:3911848264, flag:0}}, init_mode_version:0, accumulate_checksum:3088894137, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061892169125004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650522, miss_cnt:32, cache_read_size:8050455061, hit_ratio:"9.999994892303745164e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}, replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14538525920}, scn_:{val:1729061892169125004, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061892576483, first_handle_ts_:1729061892585726, replay_cost_:-1, retry_cost_:-1, read_log_buf_:0x2b0f77c35430, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}) [2024-10-16 14:58:12.585843] INFO [STORAGE] inner_update_reserved_snapshot (ob_ls_reserved_snapshot_mgr.cpp:288) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] success to update reserved snapshot(ret=0, ls_id={id:1002}, min_reserved_snapshot_=1729060091854891000) [2024-10-16 14:58:12.585852] INFO [CLOG] do_replay_task_ (ob_log_replay_service.cpp:975) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=8] pre barrier log replay succ(replay_task={ls_id_:{id:1002}, log_type:17, log_type:"RESERVED_SNAPSHOT", lsn_:{lsn:14538525920}, scn_:{val:1729061892169125004, v:0}, is_pre_barrier_:true, is_post_barrier_:false, read_log_size_:24, replay_hint_:1729061892576483, first_handle_ts_:1729061892585833, replay_cost_:18, retry_cost_:-1, read_log_buf_:0x2b0f77c35440, decompression_buf_:null, has_decompressed_:false, decompressed_log_size_:0, base_header_len_:16}, replay_status={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061892585683, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538525976}, next_to_submit_scn_:{val:1729061892169125005, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:112, curr_read_buf_start_pos:0, curr_read_buf_end_pos:112, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538525864}, end_lsn:{lsn:14538525976}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061892169125004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:24, scn_:{val:1729061892169125004, v:0}, data_checksum:3911848264, flag:0}}, init_mode_version:0, accumulate_checksum:3088894137, curr_entry_is_padding:0, padding_entry_size:56, padding_entry_scn:{val:1729061892169125004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650522, miss_cnt:32, cache_read_size:8050455061, hit_ratio:"9.999994892303745164e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}, replay_queue_idx=3, ret=0) [2024-10-16 14:58:12.587046] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:84) [22451][T1003_ObXAHbWor][T1003][Y0-0000000000000000-0-0] [lt=8] XA scheduler heartbeat task statistics(avg_time=1) [2024-10-16 14:58:12.589225] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.590989] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061892589776000, start_id=1729061892589776000, end_id=1729061892589776001) [2024-10-16 14:58:12.592670] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601526, offset=9659968}, end_cursor:ObLogCursor{file_id=192, log_id=47601527, offset=9660252}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20d0050, len:2496, log_data_len:284, seq:47601526, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:12.594498] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4C2-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061892594227000, start_id=1729061892594227000, end_id=1729061892594227001) [2024-10-16 14:58:12.595472] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601527, offset=9660252}, end_cursor:ObLogCursor{file_id=192, log_id=47601528, offset=9660526}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20c8050, len:2212, log_data_len:274, seq:47601527, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:12.598071] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:12.598086] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:12.598535] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601528, offset=9660526}, end_cursor:ObLogCursor{file_id=192, log_id=47601529, offset=9660800}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2130050, len:1938, log_data_len:274, seq:47601528, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:12.598686] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601529, offset=9660800}, end_cursor:ObLogCursor{file_id=192, log_id=47601530, offset=9661074}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd22a4050, len:1664, log_data_len:274, seq:47601529, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:12.598743] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599174, miss_cnt:9, cache_read_size:149777950924, hit_ratio:"9.999999345926349203e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:12.598806] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1002}, {id:1003}, {id:1}]) [2024-10-16 14:58:12.599384] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3545) [2024-10-16 14:58:12.600016] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=9) [2024-10-16 14:58:12.602737] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCBAD-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061892602149000, start_id=1729061892602149000, end_id=1729061892602149001) [2024-10-16 14:58:12.603115] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=201, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:12.603133] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:12.606681] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85205-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=9, avg_log_batch_cnt=0, total_group_log_size=1098, avg_group_log_size=122) [2024-10-16 14:58:12.606818] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.606895] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=247, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:12.606906] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1098) [2024-10-16 14:58:12.606916] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:12.606943] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:12.607646] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C468B-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:12.607654] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C468B-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10537, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:12.607868] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] LogLoopThread round_cost_time(us)(round_cost_time=23) [2024-10-16 14:58:12.610710] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257874, offset=39064643}, end_cursor:ObLogCursor{file_id=197, log_id=49257875, offset=39064926}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21a8050, len:3005, log_data_len:283, seq:49257874, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:12.612439] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5993, mem_used=12478464) [2024-10-16 14:58:12.612950] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB38D-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061892610567000, start_id=1729061892610567000, end_id=1729061892610567001) [2024-10-16 14:58:12.613862] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257875, offset=39064926}, end_cursor:ObLogCursor{file_id=197, log_id=49257876, offset=39065208}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21e4050, len:2722, log_data_len:282, seq:49257875, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:12.615806] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257876, offset=39065208}, end_cursor:ObLogCursor{file_id=197, log_id=49257877, offset=39065490}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2168050, len:2440, log_data_len:282, seq:49257876, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:12.615946] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=13] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257877, offset=39065490}, end_cursor:ObLogCursor{file_id=197, log_id=49257878, offset=39065772}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2094050, len:2158, log_data_len:282, seq:49257877, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:12.615986] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=50] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650523, miss_cnt:32, cache_read_size:8050455183, hit_ratio:"9.999994892303827321e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:12.615997] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=49] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657262, miss_cnt:17, cache_read_size:8050202564, hit_ratio:"9.999997286827601028e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:12.643235] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061892642879000, start_id=1729061892642879000, end_id=1729061892642879001) [2024-10-16 14:58:12.646890] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061892646010000, start_id=1729061892646010000, end_id=1729061892646010001) [2024-10-16 14:58:12.648007] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061892245167001, v:0}, new_max_version={val:1729061892245167001, v:0}, last_min_version={val:1729061892245167000, v:0}, last_max_version={val:1729061892245167000, v:0}, total_time_us=1903, delta_us=402838) [2024-10-16 14:58:12.648025] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061892245167001, v:0}, min_version={val:1729061892245167001, v:0}, max_version={val:1729061892245167001, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:58:12.649470] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:12.649691] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5AE-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=436, avg_log_submit_to_slide_time=586) [2024-10-16 14:58:12.658212] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [55755][T1004_PsCacheEv][T1004][Y0-0000000000000000-0-0] [lt=23] ps cache evict(tenant_id=1004, stmt_id_map_.size()=132, stmt_info_map_.size()=132, op.get_used_size()=3123604, get_mem_high()=24159150, expired_stmt_ids.count()=0, closed_stmt_ids.count()=132) [2024-10-16 14:58:12.666940] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061892666102000, start_id=1729061892666102000, end_id=1729061892666102001) [2024-10-16 14:58:12.670115] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0D8E-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061892669247000, start_id=1729061892669247000, end_id=1729061892669247001) [2024-10-16 14:58:12.670146] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D8C-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061892669247000, start_id=1729061892669247000, end_id=1729061892669247001) [2024-10-16 14:58:12.670452] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D8B-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061892670306000, start_id=1729061892670306000, end_id=1729061892670306001) [2024-10-16 14:58:12.670482] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D8F-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061892670306000, start_id=1729061892670306000, end_id=1729061892670306001) [2024-10-16 14:58:12.677233] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:12.678092] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=6] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2024-10-16 14:58:12.678109] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1003, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:58:12.678116] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=115, cache_node_num=115) [2024-10-16 14:58:12.678121] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:12.680155] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:12.680173] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:12.680201] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:12.680215] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1211) [2024-10-16 14:58:12.680223] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=115, alloc_obj_list=[{obj_id:786568, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:786611, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202069}, {obj_id:786614, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:321191}, {obj_id:786631, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188589}, {obj_id:786817, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:787201, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:787202, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:787203, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201552}, {obj_id:787208, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:787219, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176914}, {obj_id:787228, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:787238, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175672}, {obj_id:787350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:787353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:787354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:787355, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175682}, {obj_id:787358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181318}, {obj_id:787359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:787361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:787362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219750}, {obj_id:787365, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254046}, {obj_id:787367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203773}, {obj_id:787368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:787371, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:787372, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:787373, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:787376, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:787383, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189112}, {obj_id:787384, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178436}, {obj_id:787385, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:787386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:787387, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:787388, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:787389, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:787390, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:787391, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:787392, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:787393, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:787394, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:787395, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:787396, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:787397, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:787398, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176437}, {obj_id:787399, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203301}, {obj_id:787400, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197344}, {obj_id:787401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:787402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:787403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:787404, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:787405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:787406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:787407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:787408, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:787409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:787410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:787411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:787412, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:787413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:787414, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:787415, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:787416, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:787417, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:787418, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:787419, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:787420, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:787421, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189833}, {obj_id:787422, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:787423, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:787424, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:787425, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:787426, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:787427, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:787428, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:787432, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:787433, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:787434, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:787435, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:787438, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:787439, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:787440, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228540}, {obj_id:787523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:787524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:787525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230700}, {obj_id:787526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:787527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:787529, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:787530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:787531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:787532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:787533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:787534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:787535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:787536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:787537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:787538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:787539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:787540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:787541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:787542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:271165}, {obj_id:787543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:787544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348193}, {obj_id:787545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:386920}, {obj_id:787546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:787547, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787548, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:787549, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:787550, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:775067, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:778969, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:779106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:781597, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:785413, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:786010, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}]) [2024-10-16 14:58:12.680439] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22336][T1003_PlanCache][T1003][Y0-0000000000000000-0-0] [lt=214] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:12.680433] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:12.680467] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:12.680852] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44AD-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:12.680864] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44AD-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=602, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:12.685682] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061892685248000, start_id=1729061892685248000, end_id=1729061892685248001) [2024-10-16 14:58:12.688540] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2024-10-16 14:58:12.689294] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.689441] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22394][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061892688435000, start_id=1729061892688435000, end_id=1729061892688435001) [2024-10-16 14:58:12.693496] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EB9-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:12.693524] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EB9-0-0] [lt=21] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1454) [2024-10-16 14:58:12.704937] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:58:12.708640] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:12.708654] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:12.708660] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.710840] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:12.730659] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:12.738280] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22441][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=18] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:12.749220] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:12.749234] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:12.761769] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=2] gc stale ls task succ [2024-10-16 14:58:12.761798] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=16] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:12.761809] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=9] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:12.773515] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:58:12.780448] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:12.780468] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=15] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:12.780479] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21594][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=8] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=57) [2024-10-16 14:58:12.783244] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=14] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:12.785303] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1816, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:12.787876] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.787889] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.787898] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.788612] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.788961] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.788973] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.788984] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.789036] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.789261] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.789278] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.789363] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.789647] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:12.791058] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061892789787000, start_id=1729061892789787000, end_id=1729061892789787001) [2024-10-16 14:58:12.800091] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=8) [2024-10-16 14:58:12.802099] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E2-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:12.802126] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E2-0-0] [lt=18] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061892676616000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061892676616000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:12.803184] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9E2-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061892802742000, start_id=1729061892802742000, end_id=1729061892802742001) [2024-10-16 14:58:12.805558] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E2-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061892804859000, start_id=1729061892804859000, end_id=1729061892804859001) [2024-10-16 14:58:12.805627] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E2-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10700, avg_group_log_size=594) [2024-10-16 14:58:12.805638] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E2-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10700) [2024-10-16 14:58:12.805861] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=203, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:12.805875] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10700) [2024-10-16 14:58:12.805887] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=33, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:12.805897] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10700) [2024-10-16 14:58:12.805908] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:12.806146] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD349-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:12.806154] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD349-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=519, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:12.806158] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD349-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=18, avg_log_gen_to_submit_time=46, avg_log_submit_to_flush_time=264, avg_log_submit_to_slide_time=472) [2024-10-16 14:58:12.806194] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=19] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:12.808686] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.819155] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22464][T1003_ArcSender][T1003][YB4A0A000407-00061E9970A960C1-0-0] [lt=15][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:12.835185] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=26] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1003}, {id:1002}, {id:1}]) [2024-10-16 14:58:12.843375] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061892842896000, start_id=1729061892842896000, end_id=1729061892842896001) [2024-10-16 14:58:12.844139] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=20] get_min_unreplayed_log_info(lsn={lsn:207589586849}, scn={val:1729061892676616002, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061892807124, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589586849}, next_to_submit_scn_:{val:1729061892676616002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589586727}, end_lsn:{lsn:207589586849}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061892676616001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061892676616001, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:2888361612, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061892676616001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599177, miss_cnt:9, cache_read_size:149777951290, hit_ratio:"9.999999345926363636e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:58:12.846190] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=28] no need to update min/max version(new_version={val:1729061892245167001, v:0}, min_version={val:1729061892245167001, v:0}, max_version={val:1729061892245167001, v:0}) [2024-10-16 14:58:12.854011] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6F-0-0] [lt=23] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:12.854040] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C6F-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061892670306000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061892670306000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:12.855073] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C6F-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061892854476000, start_id=1729061892854476000, end_id=1729061892854476001) [2024-10-16 14:58:12.858074] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C6F-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061892857520000, start_id=1729061892857520000, end_id=1729061892857520001) [2024-10-16 14:58:12.862918] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6840, mem_used=12478464) [2024-10-16 14:58:12.866987] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061892865987000, start_id=1729061892865987000, end_id=1729061892865987001) [2024-10-16 14:58:12.880463] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:12.880505] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=241, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:12.880515] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=46, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:12.880523] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:12.885770] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061892884842000, start_id=1729061892884842000, end_id=1729061892884842001) [2024-10-16 14:58:12.889432] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.893984] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40213-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:12.893996] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40213-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=8374, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:12.908975] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21404][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=15] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:58:12.909133] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.911112] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:12.912771] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21687][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=19] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:58:12.914212] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:12.915915] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=3] ====== tenant freeze timer task ====== [2024-10-16 14:58:12.915930] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27418600, tenant_id=1004) [2024-10-16 14:58:12.915942] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:12.916045] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061882913794, end_time=1729061892916044, dag_status={}) [2024-10-16 14:58:12.916061] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:12.916068] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:12.916072] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:12.916079] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:12.916082] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:12.916088] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:12.916092] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:12.916098] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:12.916101] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:12.916105] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916118] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=14] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916125] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916129] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916136] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916141] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916148] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916152] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916159] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916163] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916169] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916174] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916181] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916188] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916192] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916198] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916202] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916209] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916213] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916219] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916223] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916230] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916234] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916240] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916244] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916251] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916254] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916261] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916265] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916271] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916275] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916279] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916285] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916289] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916300] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916304] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916310] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916314] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916321] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916325] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916331] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916335] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916339] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916343] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916350] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916353] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916360] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916364] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916370] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916374] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916380] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916384] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916390] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916396] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916407] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916411] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916417] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:12.916421] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:58:12.916425] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:58:12.916428] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:58:12.916431] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:58:12.916436] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:58:12.916439] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:58:12.916441] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:58:12.916443] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:58:12.916449] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:58:12.916457] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:58:12.916460] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [21522][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:58:12.921810] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22501][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=24] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:58:12.926684] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [22286][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=25] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2024-10-16 14:58:12.926885] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=15] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:12.927764] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:12.939054] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107484][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE201B-0-0] [lt=17] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1003, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:12.939075] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107484][T1003_L0_G19][T1003][YB4A0A000408-00061E9B44BE201B-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:12.946239] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1, min_version={val:1729061892646010000, v:0}, base_version={val:1729061892245167001, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061892646010000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061892946227, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061892646010000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061892844175, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061892646010001, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061892892233, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:12.947010] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061892946737000, start_id=1729061892946737000, end_id=1729061892946737001) [2024-10-16 14:58:12.948719] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF5-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061892916435, keepalive_ts:-1}) [2024-10-16 14:58:12.948732] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF5-0-0] [lt=12] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061892916435, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:12.949321] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:12.949334] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:12.950104] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4412, mem_used=10398720) [2024-10-16 14:58:12.950145] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=17] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:12.958735] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15F2-0-0] [lt=14] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:12.966208] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2024-10-16 14:58:12.966223] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=12] start lib cache evict(tenant_id=1, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:58:12.966229] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=79691776, mem_limit=107374180, cache_obj_num=288, cache_node_num=287) [2024-10-16 14:58:12.966234] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:12.966617] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:12.966631] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:12.966639] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22459][T1003_ObTimer][T1003][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=165) [2024-10-16 14:58:12.968499] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=288, alloc_obj_list=[{obj_id:16369692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181256}, {obj_id:17407335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182026}, {obj_id:17407336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210205}, {obj_id:17407343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190643}, {obj_id:17407362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189988}, {obj_id:17407363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210629}, {obj_id:17407364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191677}, {obj_id:17407375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174506}, {obj_id:17407378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:17407383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:17407389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:17407390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:17407392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:17407394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209129}, {obj_id:17407395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179195}, {obj_id:17407397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17407405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:17407407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183969}, {obj_id:17407410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194844}, {obj_id:17407411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184412}, {obj_id:17407451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199372}, {obj_id:17407468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276297}, {obj_id:17407471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205840}, {obj_id:17407474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181907}, {obj_id:17407477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158481}, {obj_id:17407482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225467}, {obj_id:17407483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:17407486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196678}, {obj_id:17407487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:17407502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185174}, {obj_id:17407507, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181267}, {obj_id:17407529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191854}, {obj_id:17407530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:17407531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173248}, {obj_id:17407532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:17407545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207068}, {obj_id:17407555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184105}, {obj_id:17407557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218270}, {obj_id:17407603, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:17407604, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:17407605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177452}, {obj_id:17407606, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:17407608, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217443}, {obj_id:17407609, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217095}, {obj_id:17407722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:17407738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407739, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174676}, {obj_id:17407803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407804, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174500}, {obj_id:17407950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178861}, {obj_id:17407951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184875}, {obj_id:17408015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:17408040, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214233}, {obj_id:17408219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17408220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178864}, {obj_id:17408238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:17408239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174059}, {obj_id:17408240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187871}, {obj_id:17408241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186392}, {obj_id:17408445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217987}, {obj_id:17408446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218800}, {obj_id:17409663, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267322}, {obj_id:17409664, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:17409665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:17410135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156389}, {obj_id:17410147, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186441}, {obj_id:17410148, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188042}, {obj_id:17410149, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183571}, {obj_id:17410150, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230701}, {obj_id:17410151, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175024}, {obj_id:17410152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189217}, {obj_id:17410153, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174256}, {obj_id:17410154, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185349}, {obj_id:17410155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:17410156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176312}, {obj_id:17410157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:17410158, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182316}, {obj_id:17410159, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175535}, {obj_id:17410160, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184740}, {obj_id:17410161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204149}, {obj_id:17410162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173216}, {obj_id:17410163, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177327}, {obj_id:17410164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17410165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205683}, {obj_id:17410166, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17410167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201486}, {obj_id:17410168, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:17410169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175661}, {obj_id:17410170, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:17410171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175768}, {obj_id:17410172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:17410173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178172}, {obj_id:17410174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229932}, {obj_id:17410175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176324}, {obj_id:17410176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201403}, {obj_id:17410177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175590}, {obj_id:17410178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:17410179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176989}, {obj_id:17410180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:17410181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187859}, {obj_id:17410182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177679}, {obj_id:17410183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:17410185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185516}, {obj_id:17410186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174979}, {obj_id:17410187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:17313231, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:408794}, {obj_id:16335202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:17287122, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:17287127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:17287131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:17287944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173792}, {obj_id:17288351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174025}, {obj_id:17288353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175950}, {obj_id:17288360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:17288381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181074}, {obj_id:17288438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:17288673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180917}, {obj_id:17288680, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:17288681, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185957}, {obj_id:17288682, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181368}, {obj_id:17288683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190006}, {obj_id:17288684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190104}, {obj_id:17288685, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175140}, {obj_id:17288686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196544}, {obj_id:17288687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:17288688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195246}, {obj_id:17288690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193495}, {obj_id:17288691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197828}, {obj_id:17288692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196504}, {obj_id:17288693, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:17288694, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:17288696, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175585}, {obj_id:17288697, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:17288698, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188174}, {obj_id:17288700, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173943}, {obj_id:17288701, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176833}, {obj_id:17288702, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173944}, {obj_id:17288703, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173980}, {obj_id:17288704, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173947}, {obj_id:17288705, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175166}, {obj_id:17288706, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173755}, {obj_id:17288707, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173748}, {obj_id:17288709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173751}, {obj_id:17288710, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:17288711, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17288712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:17288713, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191159}, {obj_id:17288714, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173678}, {obj_id:17288715, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187104}, {obj_id:17288716, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199007}, {obj_id:17288717, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206447}, {obj_id:17288718, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186331}, {obj_id:17288719, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179238}, {obj_id:17288720, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:17288721, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188747}, {obj_id:17288722, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184995}, {obj_id:17288723, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178239}, {obj_id:17288724, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182742}, {obj_id:17288725, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:17288726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:17288727, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:17288729, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199753}, {obj_id:17288730, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182560}, {obj_id:17288731, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187786}, {obj_id:17288732, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:17288733, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:17288734, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170912}, {obj_id:17288735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:17288738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205342}, {obj_id:17288740, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201256}, {obj_id:17288741, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:17288743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209581}, {obj_id:17288744, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227325}, {obj_id:17288749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:17288750, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188691}, {obj_id:17288751, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193453}, {obj_id:17288752, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192391}, {obj_id:17288753, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:17288754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:17288755, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:17288756, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171488}, {obj_id:17288758, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185850}, {obj_id:17288763, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280687}, {obj_id:17288764, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187947}, {obj_id:17288767, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172272}, {obj_id:17288768, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191460}, {obj_id:17288769, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186390}, {obj_id:17288770, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189058}, {obj_id:17288781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156412}, {obj_id:17288782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180795}, {obj_id:17288783, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179832}, {obj_id:17288784, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268338}, {obj_id:17288785, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:277422}, {obj_id:17288786, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240645}, {obj_id:17288787, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187423}, {obj_id:17288788, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195852}, {obj_id:17288789, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173184}, {obj_id:17288790, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208448}, {obj_id:17288791, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188363}, {obj_id:17288792, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319051}, {obj_id:17288793, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181464}, {obj_id:17288794, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220160}, {obj_id:17288795, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182215}, {obj_id:17288796, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243173}, {obj_id:17288797, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313405}, {obj_id:17288798, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:17288799, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:17288800, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:17288801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175564}, {obj_id:17288803, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:17288805, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179841}, {obj_id:17288806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172448}, {obj_id:17288807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172096}, {obj_id:17288894, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176374}, {obj_id:17288896, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177089}, {obj_id:17288904, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176459}, {obj_id:17288905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176342}, {obj_id:17288906, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183257}, {obj_id:17288907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180064}, {obj_id:17288909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181647}, {obj_id:17288910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275068}, {obj_id:17288911, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:281610}, {obj_id:17288912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184218}, {obj_id:17288913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194685}, {obj_id:17288914, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286475}, {obj_id:17288915, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:17288916, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:17288918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175794}, {obj_id:17288919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179663}, {obj_id:17288920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270138}, {obj_id:17288932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179629}, {obj_id:17289619, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:383525}, {obj_id:17289620, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263288}, {obj_id:14536849, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:17289709, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188153}, {obj_id:17289735, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172587}, {obj_id:17289736, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289737, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:17289738, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:17341434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17341435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176336}, {obj_id:17294455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172560}, {obj_id:17294456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157126}, {obj_id:17294457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:17294458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185849}, {obj_id:17294459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175263}, {obj_id:17294460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189863}, {obj_id:17294462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:465499}, {obj_id:17294467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:353397}, {obj_id:17294470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:886497}, {obj_id:17294471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195542}, {obj_id:17294472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190218}, {obj_id:17294473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189161}, {obj_id:17294474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193651}, {obj_id:17294475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192400}, {obj_id:13266173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:17199009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:17199095, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174722}, {obj_id:17053903, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:17056224, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175146}, {obj_id:16368312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:17056627, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208657}]) [2024-10-16 14:58:12.968847] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21467][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=345] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:12.970573] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54040-0-0] [lt=3] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:12.970595] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54040-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:12.970833] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=215, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:12.970851] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:12.970888] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:12.971154] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=49, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:12.971169] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=52, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:12.986264] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [21940][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=29] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}, {1002,1,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2024-10-16 14:58:12.989494] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:12.991393] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061892991008000, start_id=1729061892991008000, end_id=1729061892991008001) [2024-10-16 14:58:12.991807] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=11] ====== tenant freeze timer task ====== [2024-10-16 14:58:12.992843] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] ====== checkpoint timer task ====== [2024-10-16 14:58:12.993542] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] succ to get rec scn(*this={this:0x2b0e3be04030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261661057002, v:0}, max_applied_start_working_ts_:{val:1722494261661057002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494653930610000, v:0}, last_push_gc_task_ts_:1722495950787128, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:12.993568] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e438c6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728928802480820, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e438c6080, snapshot_version:{val:1728928802480820, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:12.993586] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:12.993652] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=8] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061909029444000, latest_log_ts:{val:1729061892483852001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:12.993699] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=44] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e3aec41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e3aed20d0, memtable_head:1163, memtable_tail:1164, t3m:0x2b0e2c31e030, tables:[null, null, null, null, null, null, null, null, null, null, null, 0x2b0e438a2080, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729055536809296, tx_data_table:0x2b0e3aed3c50, ls_tablet_svr:0x2b0e3aec4190}, min_rec_scn={val:1729057302398073002, v:0}, ls_id_={id:1}) [2024-10-16 14:58:12.993717] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1721066408226309000, v:0}) [2024-10-16 14:58:12.993724] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061892483852001, v:0}) [2024-10-16 14:58:12.993728] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:12.993731] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:12.993738] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729057302398073002, v:0}, checkpoint_scn_in_ls_meta={val:1729057302398073002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:12.993747] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114212315136}, new_base_lsn={lsn:114212315136}, lsn={lsn:114212315136}) [2024-10-16 14:58:12.993755] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114212315136}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:12.993760] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22456][T1003_TxCkpt][T1003][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:13.000170] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=7) [2024-10-16 14:58:13.006935] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8520D-0-0] [lt=11][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497731, log_task={header:{begin_lsn:{lsn:207589586849}, end_lsn:{lsn:207589586971}, log_id:218497731, min_scn:{val:1729061892676616002, v:0}, max_scn:{val:1729061892676616002, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589586727}, prev_proposal_id:81, committed_end_lsn:{lsn:207589586849}, data_checksum:940009244, accum_checksum:1331971564, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061892906863, freeze_ts:1729061892906863, submit_ts:1729061892906867, flushed_ts:1729061892907795, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:928}, fs_cb_begin_ts=1729061893006929, log_life_time=100066) [2024-10-16 14:58:13.009171] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.029859] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21881][T1001_LockWaitM][T1001][Y0-0000000000000000-0-0] [lt=14] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:13.043302] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=75] get number(ret=0, service_type_=0, range=1, base_id=1729061893042680000, start_id=1729061893042680000, end_id=1729061893042680001) [2024-10-16 14:58:13.045461] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=5] ====== tenant freeze timer task ====== [2024-10-16 14:58:13.045481] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=16] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151364800, tenant_id=1) [2024-10-16 14:58:13.045493] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:13.047176] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061893047175, bytes: 4446827184042, bw: 0.375352 MB/s, add_ts: 1003025, add_bytes: 394776 [2024-10-16 14:58:13.047901] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ====== [2024-10-16 14:58:13.047915] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:13.047925] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=292050392, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:58:13.049371] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:13.049393] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:13.049605] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:13.049849] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5B9-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:13.049858] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5B9-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=562, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:13.062824] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21575][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:13.062899] WDIAG read_multi_row (ob_table_access_helper.h:238) [21575][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:13.063755] INFO delete_row (ob_table_access_helper.h:312) [21575][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=63] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061893062962) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061893062962) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:58:13.067303] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061893067083000, start_id=1729061893067083000, end_id=1729061893067083001) [2024-10-16 14:58:13.070773] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:13.071036] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5BB-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:13.071051] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5BB-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=510, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:13.074969] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061893074505000, start_id=1729061893074505000, end_id=1729061893074505001) [2024-10-16 14:58:13.074994] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=16] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=651, get_gts_with_stc_cnt=27, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=27, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:13.075003] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:13.075016] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5] gts statistics(tenant_id=1002, gts_rpc_cnt=10, get_gts_cache_cnt=521, get_gts_with_stc_cnt=11, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:13.075022] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=4] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061893075013], gts:1729061893074505000, latest_srr:[mts=1729061893075013]}) [2024-10-16 14:58:13.078318] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [21603][T1_MediumLoop][T1][YB4A0A000407-00061E99739F3878-0-0] [lt=15] MediumCheckTask(cost_ts=7) [2024-10-16 14:58:13.080872] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4699-0-0] [lt=25] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=305, avg_log_submit_to_slide_time=596) [2024-10-16 14:58:13.084897] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79D1-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1816) [2024-10-16 14:58:13.087362] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061893086247000, start_id=1729061893086247000, end_id=1729061893086247001) [2024-10-16 14:58:13.087425] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:13.087441] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:13.089551] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.093662] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EC1-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=247, avg_log_submit_to_slide_time=9527) [2024-10-16 14:58:13.093703] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:13.104472] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=5] LogLoopThread round_cost_time(us)(round_cost_time=17) [2024-10-16 14:58:13.104499] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061889090974, last_first_register_time_us=1722494157071428, register_time_us=1729061889090974, curr_time_us=1729061893104498) [2024-10-16 14:58:13.104536] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061893104510, last_submit_register_req_time_us=1729061893104510, register_parent_reason=3) [2024-10-16 14:58:13.104544] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:13.108183] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=11] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061889105859, last_first_register_time_us=1722494150998907, register_time_us=1729061889105859, curr_time_us=1729061893108182) [2024-10-16 14:58:13.108211] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061893108199, last_submit_register_req_time_us=1729061893108199, register_parent_reason=3) [2024-10-16 14:58:13.108218] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=6] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:13.109217] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.111410] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:13.125046] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:392) [21123][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=15] log compressor cycles once. (ret=0, cost_time=0, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=97898364928) [2024-10-16 14:58:13.129596] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:13.129627] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=30][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:13.138834] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5377, mem_used=10398720) [2024-10-16 14:58:13.146026] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8C-0-0] [lt=22] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:13.146057] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8C-0-0] [lt=22] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061892670306004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061892670306004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:13.147364] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061893146613000, start_id=1729061893146613000, end_id=1729061893146613001) [2024-10-16 14:58:13.147584] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D8C-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061893146613000, start_id=1729061893146613000, end_id=1729061893146613001) [2024-10-16 14:58:13.150426] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:13.150453] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=266, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:13.150473] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=61, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:13.151214] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D8C-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061893151045000, start_id=1729061893151045000, end_id=1729061893151045001) [2024-10-16 14:58:13.161256] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=18] current monitor number(seq_=-1) [2024-10-16 14:58:13.170486] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D90-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061893169586000, start_id=1729061893169586000, end_id=1729061893169586001) [2024-10-16 14:58:13.170557] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D91-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061893169586000, start_id=1729061893169586000, end_id=1729061893169586001) [2024-10-16 14:58:13.170630] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54044-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:13.170648] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D94-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061893169586000, start_id=1729061893169586000, end_id=1729061893169586001) [2024-10-16 14:58:13.170861] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D92-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061893170706000, start_id=1729061893170706000, end_id=1729061893170706001) [2024-10-16 14:58:13.189617] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.191716] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061893190815000, start_id=1729061893190815000, end_id=1729061893190815001) [2024-10-16 14:58:13.200197] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=6) [2024-10-16 14:58:13.209279] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.211570] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504B-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:13.211585] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504B-0-0] [lt=13] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:13.211789] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21665][T1_L0_G0][T1][YB4A0A000408-00061E9B592E5884-0-0] [lt=10] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:58:13.211895] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21665][T1_L0_G0][T1][YB4A0A000408-00061E9B592E5884-0-0] [lt=23][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:13.243457] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061893243143000, start_id=1729061893243143000, end_id=1729061893243143001) [2024-10-16 14:58:13.243524] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=23, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:13.243539] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:13.246656] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD15-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:13.249483] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:13.249501] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:13.249956] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=246, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:13.249971] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=62, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:13.249988] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:13.250138] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5BF-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:13.250144] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5BF-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=616, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:13.252788] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=14, avg_log_submit_to_flush_time=654, avg_log_submit_to_slide_time=895) [2024-10-16 14:58:13.267407] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061893266359000, start_id=1729061893266359000, end_id=1729061893266359001) [2024-10-16 14:58:13.271035] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B939D-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=260, avg_log_submit_to_slide_time=494) [2024-10-16 14:58:13.280525] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:13.280543] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:13.284420] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769ABA3-0-0] [lt=12][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:13.285639] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:396) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=13] start to print blacklist info [2024-10-16 14:58:13.285789] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=9] ls blacklist refresh finish(cost_time=7112) [2024-10-16 14:58:13.287301] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061893285588000, start_id=1729061893285588000, end_id=1729061893285588001) [2024-10-16 14:58:13.289678] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.293324] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 143.25; ) [2024-10-16 14:58:13.293344] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:13.293386] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 125.50; ) [2024-10-16 14:58:13.293395] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:13.293403] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:13.294609] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:58:13.300044] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=24, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:13.300060] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3667, avg_group_log_size=333) [2024-10-16 14:58:13.300079] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:13.300247] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=270, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:13.300265] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=169, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:13.300283] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:13.300519] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1171-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=399, avg_log_submit_to_slide_time=603) [2024-10-16 14:58:13.301455] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1729061893301454, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007617, add_bytes: 0 [2024-10-16 14:58:13.307063] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85213-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:13.307554] INFO [CLOG] handle_submit_task_ (ob_log_replay_service.cpp:1264) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] succ to submit log task to replay service(to_submit_lsn={lsn:207589587393}, to_submit_scn={val:1729061893184531001, v:0}, replay_status={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:1, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061893307526, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:207589587459}, next_to_submit_scn_:{val:1729061893184531002, v:0}, base_lsn_:{lsn:57777205248}, base_scn_:{val:1722494112498402003, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e8a405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:207589587337}, end_lsn:{lsn:207589587459}, read_buf:{buf_len_:3747840, buf_:0x2b0e8a405000}, block_size:67104768, log_storage_:0x2b0e0db39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061893184531001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061893184531001, v:0}, data_checksum:2124447694, flag:1}}, init_mode_version:0, accumulate_checksum:97953945, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061893184531001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599182, miss_cnt:9, cache_read_size:149777951900, hit_ratio:"9.999999345926386951e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}}}}) [2024-10-16 14:58:13.309306] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.311701] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1196392448, global_cache_size=1573060608, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:13.317554] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:13.325457] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E09-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361193850, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410662, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.331007] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=1664, mem_used=4159488) [2024-10-16 14:58:13.331807] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1046-0-0] [lt=12] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:58:13.336682] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=11] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:13.347761] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] no need to update min/max version(new_version={val:1729061892946737000, v:0}, min_version={val:1729061892946737000, v:0}, max_version={val:1729061892946737000, v:0}) [2024-10-16 14:58:13.347779] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:13.349859] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:13.349909] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:13.360377] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [22251][T1002_SSTableGC][T1002][YB4A0A000407-00061E997A3CB6A6-0-0] [lt=14] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60379}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015250305282004}, old_min_reserved_snapshot=1729060090214512000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060080208141000) [2024-10-16 14:58:13.380945] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABD-0-0] [lt=17] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:58:13.380964] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21656][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABD-0-0] [lt=15] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=20) [2024-10-16 14:58:13.381465] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46A1-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:13.381474] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46A1-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=701, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:13.381517] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44C2-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:13.385323] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=40] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:13.388039] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [22251][T1002_SSTableGC][T1002][YB4A0A000407-00061E997A3CB6A6-0-0] [lt=22] SSTableGCTask(cost_ts=27866) [2024-10-16 14:58:13.388111] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=683, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:13.388123] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=603, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:13.389743] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.391761] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061893391757000, start_id=1729061893391757000, end_id=1729061893391757001) [2024-10-16 14:58:13.391847] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9324, avg_group_log_size=666) [2024-10-16 14:58:13.391866] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9324) [2024-10-16 14:58:13.392104] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=321, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:13.392115] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9324) [2024-10-16 14:58:13.392125] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=150, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:13.392145] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9324) [2024-10-16 14:58:13.392164] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=12, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:13.392551] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A41-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:13.392560] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A41-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=876, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:13.392565] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A41-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=542, avg_log_submit_to_slide_time=845) [2024-10-16 14:58:13.396074] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:58:13.396569] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [21573][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=18] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:13.400253] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=5) [2024-10-16 14:58:13.400357] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3080) [2024-10-16 14:58:13.400385] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3080) [2024-10-16 14:58:13.400616] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1172-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:13.400625] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C1172-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=643, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:13.402751] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=6] query dest_info_ succ(dest_info=[], archive_dest_changed=false) [2024-10-16 14:58:13.402765] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22153][T1002_CdcSrv][T1002][Y0-0000000000000000-0-0] [lt=12] finish to resize log external storage handler(current_ts=1729061893402764, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:58:13.407783] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:13.408011] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89F-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:13.408036] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA89F-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061893184531002, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061893184531002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:13.408617] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2055) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=14] start schedule all tablet merge(merge_version=1729015204132784000, medium_ls_tablet_iter_={ls_idx:-1, ls_ids:[{id:1}, {id:1001}, {id:1002}, {id:1003}], tablet_idx:0, tablet_ids_.count():0, schedule_tablet_cnt:0, max_batch_tablet_cnt:50000}) [2024-10-16 14:58:13.408821] INFO [STORAGE] get_adaptive_merge_reason (ob_partition_merge_policy.cpp:1751) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=18] Check tablet adaptive merge reason(ret=0, ls_id={id:1}, tablet_id={id:60379}, reason=0, tablet_analyzer={tablet_stat:{ls_id:1, tablet_id:60379, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, total_tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_small_tenant:false, boost_factor:"1.000000000000000000e+00"}, crazy_medium_flag=false) [2024-10-16 14:58:13.409186] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA89F-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061893409071000, start_id=1729061893409071000, end_id=1729061893409071001) [2024-10-16 14:58:13.409366] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.413686] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21933][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA89F-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061893413550000, start_id=1729061893413550000, end_id=1729061893413550001) [2024-10-16 14:58:13.414506] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:13.430370] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=27] batch clear tablet stat in ls(ret=0, ls_id={id:1}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:58:13.430456] INFO [SHARE] batch_update_unequal_report_scn_tablet (ob_tablet_meta_table_compaction_operator.cpp:190) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=14] start to update unequal tablet id array(ret=0, ret="OB_SUCCESS", ls_id={id:1}, major_frozen_scn=1729015204132784000, input_tablet_id_array_cnt=681) [2024-10-16 14:58:13.441908] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2091) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=9] try to update report scn as ls leader(tmp_ret=0, ls_id={id:1}) [2024-10-16 14:58:13.443589] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061893442516000, start_id=1729061893442516000, end_id=1729061893442516001) [2024-10-16 14:58:13.444631] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] get_max_applied_scn(scn={val:1722494599852226000, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:80, palf_committed_end_lsn_:{lsn:57814812323}, palf_committed_end_scn_:{val:1722494599852226000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494599852226000, v:0}}) [2024-10-16 14:58:13.448506] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3AD1EB-0-0] [lt=11] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.731511650085449219e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=865, Throttle Trigger(MB)=519, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.167297611430484494e+01) [2024-10-16 14:58:13.448647] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061893446991000, start_id=1729061893446991000, end_id=1729061893446991001) [2024-10-16 14:58:13.451878] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=14] batch clear tablet stat in ls(ret=0, ls_id={id:1001}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:58:13.451959] INFO [SHARE] batch_update_unequal_report_scn_tablet (ob_tablet_meta_table_compaction_operator.cpp:190) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=15] start to update unequal tablet id array(ret=0, ret="OB_SUCCESS", ls_id={id:1001}, major_frozen_scn=1729015204132784000, input_tablet_id_array_cnt=328) [2024-10-16 14:58:13.460912] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2091) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=10] try to update report scn as ls leader(tmp_ret=0, ls_id={id:1001}) [2024-10-16 14:58:13.463465] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2177) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:66390247825}], scn_array=[cnt:1, {val:1729061893446991000, v:0}], curr_size=587, accum_size=587, time_cost=14352) [2024-10-16 14:58:13.466567] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=15] batch clear tablet stat in ls(ret=0, ls_id={id:1002}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:58:13.466583] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2091) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=13] try to update report scn as ls leader(tmp_ret=-4722, ls_id={id:1002}) [2024-10-16 14:58:13.469439] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061893468031000, start_id=1729061893468031000, end_id=1729061893468031001) [2024-10-16 14:58:13.472241] INFO [STORAGE] batch_clear_tablet_stat (ob_tenant_tablet_stat_mgr.cpp:944) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=5] batch clear tablet stat in ls(ret=0, ls_id={id:1003}, tablet_cnt=0, clear_cnt=0) [2024-10-16 14:58:13.472260] INFO [STORAGE.COMPACTION] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:2091) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=15] try to update report scn as ls leader(tmp_ret=-4722, ls_id={id:1003}) [2024-10-16 14:58:13.472267] INFO [STORAGE.COMPACTION] after_schedule_tenant_medium (ob_tenant_tablet_scheduler.cpp:2161) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=4] finish schedule all tablet merge(merge_version=1729015204132784000, schedule_stats_={schedule_dag_cnt:0, submit_clog_cnt:0, finish_cnt:1713, wait_rs_validate_cnt:0}, tenant_merge_finish=true, merged_version_=1729015204132784000, is_scan_all_tablet_finish=true, time_guard=) [2024-10-16 14:58:13.472279] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:103) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3827-0-0] [lt=12] MediumLoopTask(cost_ts=63699) [2024-10-16 14:58:13.472290] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22808][T1004_MediumLoo][T1004][YB4A0A000407-00061E99740F3828-0-0] [lt=4] MediumCheckTask(cost_ts=5) [2024-10-16 14:58:13.480622] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:13.480638] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:13.480922] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:13.485344] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1938) [2024-10-16 14:58:13.487506] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061893486704000, start_id=1729061893486704000, end_id=1729061893486704001) [2024-10-16 14:58:13.489788] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.493750] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:13.493766] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:58:13.493777] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=70999896, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:13.493875] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EC9-0-0] [lt=21] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:58:13.494099] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=197, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:13.494112] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=85, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:13.494157] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:13.499168] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9F-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:13.499184] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2A9F-0-0] [lt=14] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:13.507246] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85217-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:13.509393] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.509423] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:13.509433] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:58:13.509443] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:58:13.509448] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:58:13.509451] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:58:13.509454] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:58:13.509446] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED7-0-0] [lt=5] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:13.509457] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:58:13.509459] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:58:13.509462] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:58:13.509459] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED7-0-0] [lt=11] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:13.509464] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:58:13.509467] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=0] tenant got versions failed(tmp_ret=-4201) [2024-10-16 14:58:13.513887] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1295-0-0] [lt=15] submit log success(service_type_=0, last_id=1729061910098662000, limited_id=1729061910098662000) [2024-10-16 14:58:13.513897] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1295-0-0] [lt=11] submit log with lock success(service_type_=0, last_id=1729061910098662000, limited_id=1729061910098662000) [2024-10-16 14:58:13.513902] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1295-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061893511066000, start_id=1729061893511066000, end_id=1729061893511066001) [2024-10-16 14:58:13.514292] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:58:13.514476] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1192198144, global_cache_size=1573060608, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:13.515007] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=7] succ to get rec scn(*this={this:0x2b0e08204030, ls_id:{id:1}, tenant_id:1001, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261473967003, v:0}, max_applied_start_working_ts_:{val:1722494261473967003, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494659617814000, v:0}, last_push_gc_task_ts_:1722495945165102, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:936}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:13.515034] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=26] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e09b8c080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1728151203985392, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e09b8c080, snapshot_version:{val:1728151203985392, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:13.515058] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:13.515123] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729061701145953001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061905590307000, latest_log_ts:{val:1729061889039053001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:13.515179] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=54] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e076c41b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e076d20d0, memtable_head:1125, memtable_tail:1126, t3m:0x2b0df6d1e030, tables:[null, null, null, null, null, 0x2b0e09b7a080, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729056570066732, tx_data_table:0x2b0e076d3c50, ls_tablet_svr:0x2b0e076c4190}, min_rec_scn={val:1729059828922812000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:13.515197] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448804509783000, v:0}) [2024-10-16 14:58:13.515204] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061889039053001, v:0}) [2024-10-16 14:58:13.515208] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:1729056578237664001, v:0}) [2024-10-16 14:58:13.515211] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:13.515222] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729056578237664001, v:0}, checkpoint_scn_in_ls_meta={val:1729056578237664001, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2024-10-16 14:58:13.515235] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:114547838976}, new_base_lsn={lsn:114547838976}, lsn={lsn:114547838976}) [2024-10-16 14:58:13.515246] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:114547838976}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:13.515251] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21896][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:13.516796] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] Garbage Collector is running(seq_=656498, gc_interval=10000000) [2024-10-16 14:58:13.516827] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=14] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:13.516838] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:58:13.517114] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:58:13.521664] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=29] update id meta(ret=0, service_type=0, limited_id=1729061910098662000, latest_scn={val:1729061893511066001, v:0}, *this={tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061890572872000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061910098662000, latest_log_ts:{val:1729061893511066001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:58:13.521705] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=40] handle submit callback(service_type_=0, success=true, used_ts=7819, log_ts={val:1729061893511066001, v:0}, limited_id=1729061910098662000) [2024-10-16 14:58:13.521713] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:58:13.521717] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:58:13.526952] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:58:13.526971] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=13] gc_check_member_list_ cost time(ret=0, time_us=10161) [2024-10-16 14:58:13.526982] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=1) [2024-10-16 14:58:13.527761] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:13.528402] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=11] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:13.528977] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:13.529558] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:13.529570] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:13.529578] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:13.529584] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:13.529589] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:13.529594] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=25) [2024-10-16 14:58:13.529598] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22720][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:58:13.550429] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:13.568761] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=12] get_max_applied_scn(scn={val:1722494433651538000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:22, palf_committed_end_lsn_:{lsn:6488080380}, palf_committed_end_scn_:{val:1722494433651538000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433651538000, v:0}}) [2024-10-16 14:58:13.568789] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=20] get_min_unreplayed_log_info(lsn={lsn:14538527196}, scn={val:1729061893169586004, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061893494437, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30759676, ret_code_:0}}, next_to_submit_lsn_:{lsn:14538527196}, next_to_submit_scn_:{val:1729061893169586004, v:0}, base_lsn_:{lsn:6374952960}, base_scn_:{val:1722448822792156001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0ea6405000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14538527074}, end_lsn:{lsn:14538527196}, read_buf:{buf_len_:3747840, buf_:0x2b0ea6405000}, block_size:67104768, log_storage_:0x2b0e43cc7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061893169586003, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061893169586003, v:0}, data_checksum:3348837161, flag:0}}, init_mode_version:0, accumulate_checksum:2302013769, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061893169586003, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650532, miss_cnt:32, cache_read_size:8050456281, hit_ratio:"9.999994892304561178e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}}}}) [2024-10-16 14:58:13.568833] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=42] get_max_applied_scn(scn={val:1722494600220969001, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:16, palf_committed_end_lsn_:{lsn:6302436797}, palf_committed_end_scn_:{val:1722494600220969001, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494600220969001, v:0}}) [2024-10-16 14:58:13.569532] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.569551] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.569564] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.569635] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.570862] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5404C-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173469, log_task={header:{begin_lsn:{lsn:14352413310}, end_lsn:{lsn:14352413432}, log_id:115173469, min_scn:{val:1729061893177072002, v:0}, max_scn:{val:1729061893177072002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352413197}, prev_proposal_id:17, committed_end_lsn:{lsn:14352413310}, data_checksum:940009244, accum_checksum:1112545327, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061893470762, freeze_ts:1729061893470762, submit_ts:1729061893470766, flushed_ts:1729061893472548, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1782}, fs_cb_begin_ts=1729061893570854, log_life_time=100092) [2024-10-16 14:58:13.570909] INFO [PALF] read (log_cache.cpp:103) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:13.575348] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061893574836000, start_id=1729061893574836000, end_id=1729061893574836001) [2024-10-16 14:58:13.575556] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=18] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061893575546], gts:1729061893574836000, latest_srr:[mts=1729061893575546]}) [2024-10-16 14:58:13.582245] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [26148][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB19CC6EF-0-0] [lt=20] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:58:13.582594] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46A9-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=24, avg_log_submit_to_flush_time=1254, avg_log_submit_to_slide_time=1633) [2024-10-16 14:58:13.582622] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=5] [APPLY STAT CB APPEND COST TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=28, this=0x2b0e2616c280, extra_info="") [2024-10-16 14:58:13.582632] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=924, this=0x2b0e2616c4b0, extra_info="") [2024-10-16 14:58:13.582636] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e2616c6e0, extra_info="") [2024-10-16 14:58:13.582639] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=-924, this=0x2b0e2616c910, extra_info="") [2024-10-16 14:58:13.582643] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=23, stat_interval=5000000, avg (count/cost)=28, this=0x2b0e2616cb40, extra_info="") [2024-10-16 14:58:13.585138] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79DB-0-0] [lt=20] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:13.585170] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79DB-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:13.585174] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79DB-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15972, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:13.586343] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:13.586356] INFO [LIB] stat (utility.h:1165) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=63, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:13.589849] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.591903] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061893590927000, start_id=1729061893590927000, end_id=1729061893590927001) [2024-10-16 14:58:13.598326] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:13.598351] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:13.598559] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [21238][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=12] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=1446) [2024-10-16 14:58:13.598900] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1002}, {id:1003}]) [2024-10-16 14:58:13.599970] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601530, offset=9661074}, end_cursor:ObLogCursor{file_id=192, log_id=47601531, offset=9661358}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2190050, len:1390, log_data_len:284, seq:47601530, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:13.600211] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3080) [2024-10-16 14:58:13.600311] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=4) [2024-10-16 14:58:13.601777] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4C3-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061893599396000, start_id=1729061893599396000, end_id=1729061893599396001) [2024-10-16 14:58:13.602595] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601531, offset=9661358}, end_cursor:ObLogCursor{file_id=192, log_id=47601532, offset=9661632}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2078050, len:1106, log_data_len:274, seq:47601531, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:13.604469] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=29] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601532, offset=9661632}, end_cursor:ObLogCursor{file_id=192, log_id=47601533, offset=9661906}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20c0050, len:832, log_data_len:274, seq:47601532, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:13.604612] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601533, offset=9661906}, end_cursor:ObLogCursor{file_id=192, log_id=47601534, offset=9662180}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2338050, len:558, log_data_len:274, seq:47601533, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:13.604652] INFO [PALF] next (palf_iterator.h:170) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=57] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561600, miss_cnt:1, cache_read_size:152590752055, hit_ratio:"9.999999928347053091e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:13.604668] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599185, miss_cnt:9, cache_read_size:149777985778, hit_ratio:"9.999999345926401384e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:13.607282] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85219-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=34854, avg_group_log_size=3168) [2024-10-16 14:58:13.607496] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=187, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:13.607510] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=34854) [2024-10-16 14:58:13.607520] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=119, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:13.607553] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:13.607714] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46AB-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:13.607730] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46AB-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9544, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:13.609454] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.612509] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6008, mem_used=12478464) [2024-10-16 14:58:13.617375] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257878, offset=39065772}, end_cursor:ObLogCursor{file_id=197, log_id=49257879, offset=39066055}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0db72d2050, len:1876, log_data_len:283, seq:49257878, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:13.619406] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB38E-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061893618848000, start_id=1729061893618848000, end_id=1729061893618848001) [2024-10-16 14:58:13.620275] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257879, offset=39066055}, end_cursor:ObLogCursor{file_id=197, log_id=49257880, offset=39066337}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2064050, len:1593, log_data_len:282, seq:49257879, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:13.622180] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257880, offset=39066337}, end_cursor:ObLogCursor{file_id=197, log_id=49257881, offset=39066619}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21ac050, len:1311, log_data_len:282, seq:49257880, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:13.622326] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257881, offset=39066619}, end_cursor:ObLogCursor{file_id=197, log_id=49257882, offset=39066901}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2250050, len:1029, log_data_len:282, seq:49257881, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:13.622370] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650533, miss_cnt:32, cache_read_size:8050456403, hit_ratio:"9.999994892304642224e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:13.622381] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657272, miss_cnt:17, cache_read_size:8050203775, hit_ratio:"9.999997286828034015e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:13.624927] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EBF-0-0] [lt=22] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061893592465, keepalive_ts:-1}) [2024-10-16 14:58:13.624943] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EBF-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061893592465, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:13.624959] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EC0-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061893592502, keepalive_ts:-1}) [2024-10-16 14:58:13.624965] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A2BDA4EC0-0-0] [lt=5] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061893592502, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:13.642761] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=8] replay status is not follower(min_unreplayed_lsn={lsn:33216819320}, base_lsn={lsn:33149755392}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:58:13.642823] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=61] get_replay_process success(id={id:1}, replayed_log_size=67063928, unreplayed_log_size=0) [2024-10-16 14:58:13.642842] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21541][T1_ReplayProces][T1][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=63, last_replayed_log_size(MB)=63, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:58:13.643785] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061893643053000, start_id=1729061893643053000, end_id=1729061893643053001) [2024-10-16 14:58:13.648747] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061893648405000, start_id=1729061893648405000, end_id=1729061893648405001) [2024-10-16 14:58:13.649432] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061893446991000, v:0}, new_max_version={val:1729061893446991000, v:0}, last_min_version={val:1729061893146613000, v:0}, last_max_version={val:1729061893146613000, v:0}, total_time_us=1511, delta_us=202440) [2024-10-16 14:58:13.649453] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061893446991000, v:0}, min_version={val:1729061893446991000, v:0}, max_version={val:1729061893446991000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:58:13.649944] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:13.650127] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5CC-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=2248, avg_log_submit_to_slide_time=2507) [2024-10-16 14:58:13.655166] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCBB4-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061893654792000, start_id=1729061893654792000, end_id=1729061893654792001) [2024-10-16 14:58:13.655498] INFO [RPC] process (ob_batch_processor.cpp:106) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCBB4-0-0] [lt=5] batch rpc statistics(clog_batch_nodelay_cnt=0, clog_batch_cnt=0, trx_batch_cnt=1, sql_batch_cnt=0) [2024-10-16 14:58:13.655679] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=12, avg_log_batch_cnt=0, total_group_log_size=52192, avg_group_log_size=4349) [2024-10-16 14:58:13.655865] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=286, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:13.655882] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=168, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:13.656001] INFO [STORAGE.TRANS] handle_2pc_prepare_request_impl_ (ob_two_phase_downstream_committer.cpp:350) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=10] committer is under logging(ret=0, *this={this:0x2b0e0c214b50, ref:4, trans_id:{txid:392285936}, tenant_id:1002, is_exiting:false, trans_expired_time:1729062493520089, cluster_version:17180066304, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1729061893520089}{self_ls_id:{id:1001}, session_id:3221769447, part_trans_action:3, pending_write:0, 2pc_role:"LEAF", ctx_tx_data_:{ctx_mgr_:0x2b0e26cb6030, tx_data_guard_:{tx_data:{tx_id:{txid:392285936}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:1729061893074505005, v:0}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:36834333536395134, replay_completeness_:1, upstream_state:"PREPARE", collected:[], rec_log_ts:{val:1729061893074505005, v:0}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1729061893551403, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008951550040 max_table_version=1729008951550040 trans_version={val:1729061893574836001, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285936} ls_id=1001 row_callback[alloc:340, free:0, unsubmit:0] redo[fill:350,sync_succ:197, sync_fail:0] main_list_len=350 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,350,350,197,350,0,0,0)|]}} -{busy_cbs_cnt:1, oldest_busy_cb:{ObTxBaseLogCb:{base_ts:{val:1729061893574836001, v:0}, log_ts:{val:1729061893574836001, v:0}, lsn:{lsn:204023175285}, submit_ts:1729061893655428}, this:0x2b0e0c217aa8, is_inited_:true, trans_id:{txid:392285936}, ls_id:{id:1001}, ctx:0x2b0e0c214b50, tx_data_guard:{tx_data:NULL}, is_callbacked_:false, is_dynamic_:false, mds_range_:{range_array_.count():0, range_array_:[]}, cb_arg_array_:[{log_type:0x1, arg:null}, {log_type:0x40, arg:null}, {log_type:0x80, arg:null}], first_part_scn_:{val:18446744073709551615, v:3}, extra_cb_:null, need_free_extra_cb_:false, callbacks_.count():1}, -{info_log_submitted:1 state_log_submitting:1 state_log_submitted:1 }-, -{downstream_state:"INIT", upstream:{id:1002}, participants:[], redo_log_no:1, scheduler:"10.0.4.8:2890", prepare_version:{val:1729061893574836001, v:0}, trans_type:2, next_log_entry_no:2, max_applied_log_ts:{val:1729061893074505005, v:0}, max_appling_log_ts:{val:1729061893074505005, v:0}, max_applying_part_log_no:0, max_submitted_seq_no:{branch:0, seq:11614482}, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, redo_lsns:[{lsn:204023146383}], max_durable_lsn:{lsn:204023146383}, }- }-}) [2024-10-16 14:58:13.656183] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=10] rpc statistics(total_trans_msg_count=4, total_batch_msg_count=4) [2024-10-16 14:58:13.669617] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C129D-0-0] [lt=75] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1002, table_id:10009, add_timestamp:1729061893669609}) [2024-10-16 14:58:13.669688] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061893667972000, start_id=1729061893667972000, end_id=1729061893667972001) [2024-10-16 14:58:13.669816] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E70-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1002, table_id=10009, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061893669799, sql_renew_time:1729061893669799, is_mark_fail:false}]) [2024-10-16 14:58:13.669834] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E70-0-0] [lt=17] success to process renew task(task={tenant_id:1002, table_id:10009, add_timestamp:1729061893669609}, locations=[{table_id:10009, partition_id:10009, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061893669799, sql_renew_time:1729061893669799, is_mark_fail:false}]) [2024-10-16 14:58:13.670035] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C129D-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361538429, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893921, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.670885] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D96-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061893669870000, start_id=1729061893669870000, end_id=1729061893669870001) [2024-10-16 14:58:13.671065] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0D98-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061893670971000, start_id=1729061893670971000, end_id=1729061893670971001) [2024-10-16 14:58:13.671181] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D99-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061893670971000, start_id=1729061893670971000, end_id=1729061893670971001) [2024-10-16 14:58:13.671184] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D95-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061893670971000, start_id=1729061893670971000, end_id=1729061893670971001) [2024-10-16 14:58:13.674216] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C129F-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361542609, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893922, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.675558] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001) [2024-10-16 14:58:13.675572] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=12] start lib cache evict(tenant_id=1001, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:58:13.675579] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=48234496, mem_limit=53687090, cache_obj_num=108, cache_node_num=108) [2024-10-16 14:58:13.675584] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:13.675736] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12A1-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361544131, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893923, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.677283] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:13.677840] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=108, alloc_obj_list=[{obj_id:591431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:600420, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:609614, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198696}, {obj_id:613232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185385}, {obj_id:613237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175082}, {obj_id:613239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:613241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187125}, {obj_id:613390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175980}, {obj_id:613391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201829}, {obj_id:614564, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:614566, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177420}, {obj_id:614574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176381}, {obj_id:615304, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181105}, {obj_id:615540, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174753}, {obj_id:615782, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182712}, {obj_id:615795, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194974}, {obj_id:615797, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:615798, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201680}, {obj_id:615807, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:616042, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183593}, {obj_id:616043, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:616044, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196528}, {obj_id:616045, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:616046, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188640}, {obj_id:616047, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181340}, {obj_id:616048, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180776}, {obj_id:616049, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174096}, {obj_id:616050, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175618}, {obj_id:616051, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:616053, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197856}, {obj_id:616054, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189981}, {obj_id:616055, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190107}, {obj_id:616056, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194838}, {obj_id:616057, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190746}, {obj_id:616058, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188206}, {obj_id:616059, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174745}, {obj_id:616060, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198472}, {obj_id:616061, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178690}, {obj_id:616062, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185988}, {obj_id:616063, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178269}, {obj_id:616065, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186393}, {obj_id:616066, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189086}, {obj_id:616067, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175616}, {obj_id:616068, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196714}, {obj_id:616070, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:616071, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:616072, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:616077, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188298}, {obj_id:616078, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:616079, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:616080, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:616081, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:616082, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:616083, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176861}, {obj_id:616096, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189657}, {obj_id:616097, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194828}, {obj_id:616142, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:616143, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188127}, {obj_id:616144, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:616145, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:616146, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:616147, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:616151, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:616152, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:616153, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181039}, {obj_id:616154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199010}, {obj_id:616155, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197032}, {obj_id:616156, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181472}, {obj_id:616157, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:616158, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191508}, {obj_id:616161, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:616162, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:616166, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187972}, {obj_id:616172, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191432}, {obj_id:616224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174304}, {obj_id:616225, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173920}, {obj_id:616226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175509}, {obj_id:616227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616228, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174466}, {obj_id:616229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:616230, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173056}, {obj_id:616231, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:616232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173408}, {obj_id:616233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:616234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}, {obj_id:616235, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:616236, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:616237, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219500}, {obj_id:616238, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216983}, {obj_id:616239, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179170}, {obj_id:616240, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:616241, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173904}, {obj_id:616242, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176422}, {obj_id:616243, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:616244, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:616245, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:458909}, {obj_id:616246, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179693}, {obj_id:616247, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184392}, {obj_id:616248, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245222}, {obj_id:616249, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238893}, {obj_id:616250, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189111}, {obj_id:616251, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381435}, {obj_id:616252, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258251}, {obj_id:616253, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371175}, {obj_id:616254, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616255, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:616256, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:616257, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}]) [2024-10-16 14:58:13.677987] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [21747][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=144] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:13.680687] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:13.680707] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:13.680732] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=52657, avg_group_log_size=3761) [2024-10-16 14:58:13.680740] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=52657) [2024-10-16 14:58:13.681025] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1226) [2024-10-16 14:58:13.681088] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1226) [2024-10-16 14:58:13.681415] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44D3-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:13.681426] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44D3-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=1427, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:13.682777] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12A3-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361551163, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893924, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.687686] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061893686855000, start_id=1729061893686855000, end_id=1729061893686855001) [2024-10-16 14:58:13.689901] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.691705] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12A4-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361560098, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893925, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.694764] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2024-10-16 14:58:13.695116] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22A2-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361563509, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893926, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.695237] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2ECD-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:13.695252] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2ECD-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:13.695592] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22698][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061893694378000, start_id=1729061893694378000, end_id=1729061893694378001) [2024-10-16 14:58:13.700959] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12A7-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361569353, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893927, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.704536] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12A9-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361572930, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893928, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.704997] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:58:13.709484] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.711994] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12AB-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361580387, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893929, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.713592] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22A4-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361581986, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893930, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.714840] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1177518080, global_cache_size=1589706752, tenant_max_wash_size=13817446, tenant_min_wash_size=13817446, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:13.715120] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash memory, (tenant_id=1002, cache_size=95715328, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=13817446, max_wash_size=13817446, mem_usage=980185088, reserve_mem=107374182, wash_size=13817446) [2024-10-16 14:58:13.715245] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Wash time detail, (compute_wash_size_time=93, refresh_score_time=70, wash_time=202) [2024-10-16 14:58:13.718109] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=5] Cache wash clean map node details(ret=0, clean_node_count=15, clean_time=0, clean_start_pos=2076162, clean_num=62914) [2024-10-16 14:58:13.719928] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=30] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:13.727784] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12B0-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361596177, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893931, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.731300] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12B2-0-0] [lt=28][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361599693, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893932, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.731640] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:13.739495] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22220][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=13] report RowHolderMapper summary info(count=0, bkt_cnt=315) [2024-10-16 14:58:13.742700] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15F6-0-0] [lt=11] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:13.742818] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15F8-0-0] [lt=21] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:13.746884] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12B4-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361615278, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893933, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.749744] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] tx gc loop thread is running(MTL_ID()=1004) [2024-10-16 14:58:13.749759] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] try gc retain ctx [2024-10-16 14:58:13.749763] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:13.749777] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=3] get_max_applied_scn(scn={val:1729061893574836000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:17493518472}, palf_committed_end_scn_:{val:1729061893694378000, v:0}, last_check_scn_:{val:1729061893694378000, v:0}, max_applied_cb_scn_:{val:1729061893574836000, v:0}}) [2024-10-16 14:58:13.749797] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:7668488755}, scn={val:1722494261263129005, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:58:13.750408] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=45] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061900721265000, latest_log_ts:{val:1729061884067775001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:13.750505] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=40] get_max_applied_scn(scn={val:1729061893574836000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:17, palf_committed_end_lsn_:{lsn:13645457094}, palf_committed_end_scn_:{val:1729061893574836000, v:0}, last_check_scn_:{val:1729061893574836000, v:0}, max_applied_cb_scn_:{val:1729061893574836000, v:0}}) [2024-10-16 14:58:13.750518] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] get_min_unreplayed_log_info(lsn={lsn:6198988121}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:58:13.751122] INFO [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:747) [25362][T1_PsCacheEvict][T1][Y0-0000000000000000-0-0] [lt=16] ps cache evict(tenant_id=1, stmt_id_map_.size()=4, stmt_info_map_.size()=4, op.get_used_size()=8920, get_mem_high()=48318300, expired_stmt_ids.count()=0, closed_stmt_ids.count()=4) [2024-10-16 14:58:13.751125] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:13.751172] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:13.751180] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:13.751778] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:13.751809] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] get_min_unreplayed_log_info(lsn={lsn:14352413676}, scn={val:1729061893177072005, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061893671614, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:30065272, ret_code_:0}}, next_to_submit_lsn_:{lsn:14352413676}, next_to_submit_scn_:{val:1729061893177072005, v:0}, base_lsn_:{lsn:6240743424}, base_scn_:{val:1722448822511595002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0eada05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:14352413554}, end_lsn:{lsn:14352413676}, read_buf:{buf_len_:3747840, buf_:0x2b0eada05000}, block_size:67104768, log_storage_:0x2b0e43d39a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061893177072004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1729061893177072004, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:624398928, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1729061893177072004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657273, miss_cnt:17, cache_read_size:8050203897, hit_ratio:"9.999997286828077314e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}}}}) [2024-10-16 14:58:13.752456] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:13.759390] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12B6-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361627784, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893934, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.760850] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12B8-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361629245, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893935, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.761802] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:13.761832] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=12] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:13.761839] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=6] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:13.764074] INFO [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:262) [22249][T1002_ArcSender][T1002][YB4A0A000407-00061E996F9960C1-0-0] [lt=12] ObArchiveSender is running(thread_index=1) [2024-10-16 14:58:13.768452] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12BA-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361636846, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893936, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.773575] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=23] gc stale ls task succ [2024-10-16 14:58:13.775031] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12BC-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361643425, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893937, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.779554] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12BE-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361647948, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893938, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.783415] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=11] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:13.784865] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12C0-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361653260, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893939, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.785893] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=51925, hit_cnt=13, read_cnt=13, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:13.788612] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.788811] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.788830] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.788843] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.788851] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.788865] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.788874] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.788888] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.788899] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.788910] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.789564] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:13.789948] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.791840] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12C2-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361660228, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893940, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.791989] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061893790536000, start_id=1729061893790536000, end_id=1729061893790536001) [2024-10-16 14:58:13.800450] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=28] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=3) [2024-10-16 14:58:13.804257] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12C4-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361672650, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893941, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.805749] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12C6-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361674144, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893942, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.807019] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12C8-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361675415, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893943, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.808426] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E3-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:13.808452] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E3-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061893675221000, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061893675221000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:13.810023] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9E3-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061893809177000, start_id=1729061893809177000, end_id=1729061893809177001) [2024-10-16 14:58:13.810466] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12CA-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361678861, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893944, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.810506] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.811913] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79DD-0-0] [lt=7] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061893803141, keepalive_ts:-1}) [2024-10-16 14:58:13.811926] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B4A8E79DD-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061893803141, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:13.812519] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12CC-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361680915, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893945, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.814032] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12CE-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361682429, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893946, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.814504] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E3-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061893813621000, start_id=1729061893813621000, end_id=1729061893813621001) [2024-10-16 14:58:13.814572] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E3-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=9332, avg_group_log_size=666) [2024-10-16 14:58:13.814591] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E3-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9332) [2024-10-16 14:58:13.814899] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=258, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:13.814911] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9332) [2024-10-16 14:58:13.814922] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=147, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:13.814939] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=25] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9332) [2024-10-16 14:58:13.814946] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:13.815705] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12D0-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361684102, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893947, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.815788] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD357-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:13.815811] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD357-0-0] [lt=19] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=828, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:13.815816] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD357-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=32, avg_log_submit_to_flush_time=531, avg_log_submit_to_slide_time=795) [2024-10-16 14:58:13.815888] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:13.818979] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12D2-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361687376, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893948, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.824402] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DE-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061893792088, keepalive_ts:-1}) [2024-10-16 14:58:13.824422] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000409-00061E99FD6C47DE-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061893792088, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:13.826078] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12D4-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361694473, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893949, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.829101] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12D6-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361697497, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893950, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.830115] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:58:13.830132] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:58:13.830148] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52750088, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:13.835301] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1002}, {id:1001}, {id:1}]) [2024-10-16 14:58:13.837622] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12D8-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361706017, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893951, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.841222] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12DA-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361709616, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893952, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.843867] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061893843194000, start_id=1729061893843194000, end_id=1729061893843194001) [2024-10-16 14:58:13.845006] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [29808][T1002_L0_G0][T1002][YB4A0A000407-00061E9B2ECA223D-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361713401, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893953, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.848789] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061893848461000, start_id=1729061893848461000, end_id=1729061893848461001) [2024-10-16 14:58:13.850515] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12DD-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361718909, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893954, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.861232] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C70-0-0] [lt=23] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:13.861257] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C70-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061893670971000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061893670971000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:13.861378] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] new mysql sessid created(conn.sessid_=3221742642, support_ssl=false) [2024-10-16 14:58:13.861408] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=11] sm conn init succ(conn.sessid_=3221742642, sess.client_addr_="10.0.4.7:53484") [2024-10-16 14:58:13.861418] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=7] accept one succ(*s={this:0x2b0e5cdcc030, session_id:3221742642, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:13.861636] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=23][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdcc030, session_id:3221742642, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:13.861653] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=15] kill and revert session(conn.sessid_=3221742642, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:13.861659] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=5] sock ref clean, do destroy(*s={this:0x2b0e5cdcc030, session_id:3221742642, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:13.861666] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:223) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=6] mark session id unused(conn.sessid_=3221742642) [2024-10-16 14:58:13.861670] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21317][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221742642, proxy_sessid=0, tenant_id=0, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=false, ret=0, trace_id=Y0-0000000000000000-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:0], disconnect_state=0) [2024-10-16 14:58:13.862465] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C70-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061893861556000, start_id=1729061893861556000, end_id=1729061893861556001) [2024-10-16 14:58:13.863029] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6899, mem_used=14558208) [2024-10-16 14:58:13.863178] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12DE-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361731571, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893955, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.864697] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12E0-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361733092, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893956, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.866416] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C70-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061893865802000, start_id=1729061893865802000, end_id=1729061893865802001) [2024-10-16 14:58:13.869768] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061893868983000, start_id=1729061893868983000, end_id=1729061893868983001) [2024-10-16 14:58:13.881065] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=344, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:13.881093] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=70, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:13.881092] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=52657) [2024-10-16 14:58:13.881131] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:13.887717] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061893887143000, start_id=1729061893887143000, end_id=1729061893887143001) [2024-10-16 14:58:13.890010] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.894679] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93AA-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:13.894692] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93AA-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30264, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:13.904330] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12E5-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361772723, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893957, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.910576] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.918387] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1577222144, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:13.920044] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12E7-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361788437, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893958, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.921614] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12E9-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361789999, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893959, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.927136] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22A6-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361795530, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893960, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.927188] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=10] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:13.928040] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=42] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:13.928651] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:13.928697] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061887261758000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:58:13.928774] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:13.930073] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22615][T1004_PalfGC][T1004][Y0-0000000000000000-0-0] [lt=16] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):612, sequence:1}) [2024-10-16 14:58:13.931296] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12EB-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361799690, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893961, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.941862] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12ED-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361810255, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893962, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.948396] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12EF-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361816791, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893963, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.949956] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:13.949974] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:13.950015] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12F1-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361818411, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893964, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.950217] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4532, mem_used=10398720) [2024-10-16 14:58:13.951861] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108056][T1002_ComTimerT][T1002][Y0-0000000000000000-0-0] [lt=11] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:13.954964] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12F3-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361823359, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893965, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.956224] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12F5-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361824620, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893966, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.959331] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12F7-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361827727, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893967, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.960608] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12F9-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361828989, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893968, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.961810] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12FB-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361830207, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893969, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.969420] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12FD-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361837816, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893970, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.970717] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C12FF-0-0] [lt=37][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361839114, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893971, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.971141] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54054-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:13.971163] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54054-0-0] [lt=14] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:13.971395] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=210, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:13.971409] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=178, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:13.971464] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:13.971754] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:13.971764] INFO [LIB] stat (utility.h:1165) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=10] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:13.976051] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1301-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361844446, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893972, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.979165] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1303-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361847561, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893973, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.982337] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1305-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361850724, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893974, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.990097] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:13.990909] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1307-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361859303, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893975, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.992100] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061893992021000, start_id=1729061893992021000, end_id=1729061893992021001) [2024-10-16 14:58:13.993458] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=10] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:13.993476] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22800][T1004_TabletShe][T1004][Y0-0000000000000000-0-0] [lt=9] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=218) [2024-10-16 14:58:13.993930] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1309-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361862324, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893976, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.995522] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C130B-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361863918, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893977, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:13.996868] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C130D-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361865264, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893978, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.005740] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C130F-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361874134, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893979, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.007295] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1311-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361875690, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893980, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.008892] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1313-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361877287, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893981, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.009486] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=2) [2024-10-16 14:58:14.010627] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.017411] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1315-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361885805, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893982, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.020493] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1317-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361888888, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893983, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.021944] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1319-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361890340, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893984, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.033723] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C131B-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361902115, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893985, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.037315] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C131D-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361905709, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893986, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.043820] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061894042898000, start_id=1729061894042898000, end_id=1729061894042898001) [2024-10-16 14:58:14.045622] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C131F-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361914017, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893987, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.047306] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] ====== checkpoint timer task ====== [2024-10-16 14:58:14.048051] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x2b0e26c04030, ls_id:{id:1}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261705406001, v:0}, max_applied_start_working_ts_:{val:1722494261705406001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667464207000, v:0}, last_push_gc_task_ts_:1722495948674013, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:685}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.048079] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=28] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495948675495, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563a080, snapshot_version:{val:1722495948675495, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.048102] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:14.048268] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015248776112002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, }, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891573537000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061910098662000, latest_log_ts:{val:1729061893511066001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:14.048318] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=45] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891573537000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061910098662000, latest_log_ts:{val:1729061893511066001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015248776112002, v:0}, ls_id_={id:1}) [2024-10-16 14:58:14.048348] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=26] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490032781494005, v:0}) [2024-10-16 14:58:14.048355] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061893511066001, v:0}) [2024-10-16 14:58:14.048359] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=1, rec_log_ts={val:1729061599022421002, v:0}) [2024-10-16 14:58:14.048367] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015248776112002, v:0}, checkpoint_scn_in_ls_meta={val:1729015248776112002, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:14.048377] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17849868288}, new_base_lsn={lsn:17849868288}, lsn={lsn:17849868288}) [2024-10-16 14:58:14.048385] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17849868288}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:14.048907] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061894047138000, start_id=1729061894047138000, end_id=1729061894047138001) [2024-10-16 14:58:14.049088] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061894049087, bytes: 4446827710753, bw: 0.501352 MB/s, add_ts: 1001912, add_bytes: 526711 [2024-10-16 14:58:14.049108] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e26cb6030, ls_id:{id:1001}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147254557, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:495}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.049131] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=23] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563b350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1726531400066482, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563b350, snapshot_version:{val:1726531400066482, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.049166] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=33] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:58:14.049209] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045387747012001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, }, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891573537000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:14.049269] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=58] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891573537000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045387747012001, v:0}, ls_id_={id:1001}) [2024-10-16 14:58:14.049294] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=23] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097427155000, v:0}) [2024-10-16 14:58:14.049301] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045387747012001, v:0}, checkpoint_scn_in_ls_meta={val:1729045387747012001, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:58:14.049308] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:203595866112}, new_base_lsn={lsn:203595866112}, lsn={lsn:203595866112}) [2024-10-16 14:58:14.049315] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:203595866112}, ls->get_ls_id()={id:1001}) [2024-10-16 14:58:14.050200] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e27804030, ls_id:{id:1002}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433932036001, v:0}, max_applied_start_working_ts_:{val:1722494433932036001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147270518, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:488}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.050250] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=49] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563c620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563c620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.050263] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:58:14.050306] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729044692383096004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, }, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891573537000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:14.050353] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=45] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891573537000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729044692383096004, v:0}, ls_id_={id:1002}) [2024-10-16 14:58:14.050379] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=24] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097423917000, v:0}) [2024-10-16 14:58:14.050386] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729044692383096004, v:0}, checkpoint_scn_in_ls_meta={val:1729044692383096004, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:58:14.050393] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:212587905024}, new_base_lsn={lsn:212587905024}, lsn={lsn:212587905024}) [2024-10-16 14:58:14.050399] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:212587905024}, ls->get_ls_id()={id:1002}) [2024-10-16 14:58:14.051276] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succ to get rec scn(*this={this:0x2b0e278b6030, ls_id:{id:1003}, tenant_id:1002, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494599852226001, v:0}, max_applied_start_working_ts_:{val:1722494599852226001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722442300452214001, v:0}, last_push_gc_task_ts_:1722494147305891, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:485}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.051295] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=18] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e4563d8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e4563d8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.051306] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:58:14.051340] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729045459272503001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, }, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891573537000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:14.051400] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=57] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891573537000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729045459272503001, v:0}, ls_id_={id:1003}) [2024-10-16 14:58:14.051426] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=24] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722490097485464001, v:0}) [2024-10-16 14:58:14.051433] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729045459272503001, v:0}, checkpoint_scn_in_ls_meta={val:1729045459272503001, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:58:14.051440] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:207152418816}, new_base_lsn={lsn:207152418816}, lsn={lsn:207152418816}) [2024-10-16 14:58:14.051451] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:207152418816}, ls->get_ls_id()={id:1003}) [2024-10-16 14:58:14.051455] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22236][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:58:14.051875] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:14.051888] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2694, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:14.053555] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1321-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361921948, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893988, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.056969] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1323-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361925363, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893989, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.060385] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1325-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361928780, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893990, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.061111] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [22780][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=17] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2024-10-16 14:58:14.065494] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1327-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361933888, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893991, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.069857] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061894068676000, start_id=1729061894068676000, end_id=1729061894068676001) [2024-10-16 14:58:14.070544] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1329-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361938938, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893992, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.071517] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:14.071831] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93AD-0-0] [lt=9] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:14.071843] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93AD-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10480, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:14.075762] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894075060000, start_id=1729061894075060000, end_id=1729061894075060001) [2024-10-16 14:58:14.075784] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=15] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061894075755], gts:1729061894075060000, latest_srr:[mts=1729061894075755]}) [2024-10-16 14:58:14.075794] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:14.075996] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21292][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=1886, avg_group_log_size=157) [2024-10-16 14:58:14.076008] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21292][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1880) [2024-10-16 14:58:14.076015] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [21292][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=6] submit log success(service_type_=0, last_id=1729061910721265000, limited_id=1729061910721265000) [2024-10-16 14:58:14.076019] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [21292][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=4] submit log with lock success(service_type_=0, last_id=1729061910721265000, limited_id=1729061910721265000) [2024-10-16 14:58:14.076072] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C132B-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361944466, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893993, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.076483] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1886) [2024-10-16 14:58:14.076786] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:760) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] update id meta(ret=0, service_type=0, limited_id=1729061910721265000, latest_scn={val:1729061894075060001, v:0}, *this={tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061910721265000, latest_log_ts:{val:1729061894075060001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}) [2024-10-16 14:58:14.076821] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=34] handle submit callback(service_type_=0, success=true, used_ts=805, log_ts={val:1729061894075060001, v:0}, limited_id=1729061910721265000) [2024-10-16 14:58:14.076828] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:562) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=6] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2024-10-16 14:58:14.076833] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:600) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2024-10-16 14:58:14.077741] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C132D-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361946137, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893994, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.081739] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44DE-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=452, avg_log_submit_to_slide_time=1410) [2024-10-16 14:58:14.085453] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79E3-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=51925) [2024-10-16 14:58:14.085471] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79E3-0-0] [lt=11][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938600, log_task={header:{begin_lsn:{lsn:213058334389}, end_lsn:{lsn:213058334511}, log_id:220938600, min_scn:{val:1729061893670971002, v:0}, max_scn:{val:1729061893670971002, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058334267}, prev_proposal_id:88, committed_end_lsn:{lsn:213058334389}, data_checksum:1715074492, accum_checksum:186295196, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061893985449, freeze_ts:1729061893985449, submit_ts:1729061893985454, flushed_ts:1729061893985868, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:414}, fs_cb_begin_ts=1729061894085465, log_life_time=100016) [2024-10-16 14:58:14.085485] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C132F-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361953879, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893995, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.087132] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1331-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361955527, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893996, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.088073] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] Garbage Collector is running(seq_=656495, gc_interval=10000000) [2024-10-16 14:58:14.088100] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=11] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:14.088107] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] self is leader, skip it(ls->get_ls_id()={id:1001}) [2024-10-16 14:58:14.088336] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061894087834000, start_id=1729061894087834000, end_id=1729061894087834001) [2024-10-16 14:58:14.088404] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:14.088419] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:14.089567] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=2] GC check ls in member list, skip it(id={id:1002}, leader="10.0.4.8:2890") [2024-10-16 14:58:14.089890] INFO [CLOG] handle_rpc_response_ (ob_garbage_collector.cpp:274) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=15] GC check ls in member list, skip it(id={id:1003}, leader="10.0.4.9:2890") [2024-10-16 14:58:14.089910] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=14] gc_check_member_list_ cost time(ret=0, time_us=1826) [2024-10-16 14:58:14.089922] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] execute_gc cost time(ret=0, time_us=0) [2024-10-16 14:58:14.090142] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:14.090160] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:14.090166] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.090740] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:14.091435] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=20] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:14.092166] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=20] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:14.092864] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=11] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:14.092877] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=10] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:14.092886] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=8] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:14.092891] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=6] ls status is normal, skip(id={id:1002}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:14.092896] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] ls status is normal, skip(id={id:1003}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1002}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1003}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:14.092902] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=25) [2024-10-16 14:58:14.092906] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:58:14.092910] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:133) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-10-16 14:58:14.092921] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:140) [22157][T1002_GCCollect][T1002][Y0-0000000000000000-0-0] [lt=4] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-10-16 14:58:14.093816] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894093625000, start_id=1729061894093625000, end_id=1729061894093625001) [2024-10-16 14:58:14.094653] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40239-0-0] [lt=18] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=5, avg_log_submit_to_flush_time=327, avg_log_submit_to_slide_time=18502) [2024-10-16 14:58:14.094689] INFO [PALF] read (log_cache.cpp:103) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:14.095952] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1333-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361964336, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893997, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.106718] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1335-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361975104, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893998, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.108128] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1337-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361976523, cache_obj->added_lc()=false, cache_obj->get_object_id()=78893999, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.109794] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061890091550, last_first_register_time_us=1722494157071428, register_time_us=1729061890091550, curr_time_us=1729061894109793) [2024-10-16 14:58:14.109829] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=17] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061894109812, last_submit_register_req_time_us=1729061894109812, register_parent_reason=3) [2024-10-16 14:58:14.109839] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:58:14.110691] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.115148] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1339-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361983543, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894000, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.117028] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=64] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:14.117613] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=20] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:14.117658] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061890214512000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:58:14.118529] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C133B-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361986924, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894001, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.118684] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1577222144, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:14.119934] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C133D-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361988331, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894002, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.121277] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C133F-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361989674, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894003, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.122567] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1341-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510361990963, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894004, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.128679] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:14.128693] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:14.130215] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22399][T1003_TenantMet][T1003][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:833, free_obj_hold_cnt:48, allocator used:3390088, allocator total:3662848}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:100, free_obj_hold_cnt:104, allocator used:13380564, allocator total:13380564}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=17112916, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:35, allocator used:122880, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=933) [2024-10-16 14:58:14.132288] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1343-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362000683, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894005, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.138890] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5427, mem_used=10398720) [2024-10-16 14:58:14.139096] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1345-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362007484, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894006, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.146039] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1347-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362014434, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894007, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.150488] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=10] gc stale ls task succ [2024-10-16 14:58:14.153936] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8D-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:14.153969] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8D-0-0] [lt=22] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061893670971004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061893670971004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:14.154942] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D8D-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061894154662000, start_id=1729061894154662000, end_id=1729061894154662001) [2024-10-16 14:58:14.157359] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1349-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362025754, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894008, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.158419] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D8D-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061894157836000, start_id=1729061894157836000, end_id=1729061894157836001) [2024-10-16 14:58:14.161113] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C134B-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362029507, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894009, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.162038] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=17] current monitor number(seq_=-1) [2024-10-16 14:58:14.167060] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C134D-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362035454, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894010, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.170000] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C134F-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362038395, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894011, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.171215] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D9A-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061894170735000, start_id=1729061894170735000, end_id=1729061894170735001) [2024-10-16 14:58:14.171217] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0D9B-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061894170735000, start_id=1729061894170735000, end_id=1729061894170735001) [2024-10-16 14:58:14.171280] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0D9C-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061894170735000, start_id=1729061894170735000, end_id=1729061894170735001) [2024-10-16 14:58:14.171466] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0D9E-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061894170735000, start_id=1729061894170735000, end_id=1729061894170735001) [2024-10-16 14:58:14.171560] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54058-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:14.175426] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1351-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362043820, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894012, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.178483] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15F4-0-0] [lt=22] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:14.180945] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1353-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362049339, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894013, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.183410] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15F0-0-0] [lt=15] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:14.183942] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=24707872, this={dio_aligned_buf:{buf_write_offset:922, buf_padding_size:3174, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:93867202, truncate_used_ts:39945035}, log_block_size:67108864, dir_fd:173, io_fd:196}, aligned_buf_len=4096, aligned_block_offset=24707072, buf_len=122, total_write_size_=63609, total_write_size_after_dio_=491520, ob_pwrite_used_ts=26998, count=104, each_pwrite_cost=259) [2024-10-16 14:58:14.186473] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1355-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362054867, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894014, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.188171] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1357-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362056567, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894015, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.190203] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.191417] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1359-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362059812, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894016, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.196545] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C135B-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362064940, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894017, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.197897] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C135D-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362066292, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894018, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.199135] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C135F-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362067531, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894019, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.200328] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1361-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362068725, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894020, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.206474] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1363-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362074870, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894021, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.207694] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85223-0-0] [lt=18][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497746, log_task={header:{begin_lsn:{lsn:207589637434}, end_lsn:{lsn:207589637556}, log_id:218497746, min_scn:{val:1729061893675221003, v:0}, max_scn:{val:1729061893675221003, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589637312}, prev_proposal_id:81, committed_end_lsn:{lsn:207589637434}, data_checksum:940009244, accum_checksum:3679152564, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061894107582, freeze_ts:1729061894107582, submit_ts:1729061894107588, flushed_ts:1729061894108507, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:919}, fs_cb_begin_ts=1729061894207687, log_life_time=100105) [2024-10-16 14:58:14.207836] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1365-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362076232, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894022, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.209560] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=1) [2024-10-16 14:58:14.210754] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.212437] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504C-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:14.212459] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504C-0-0] [lt=20] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:14.214673] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1367-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362083068, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894023, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.226837] INFO [SHARE.LOCATION] renew_all_ls_locations_by_rpc (ob_ls_location_service.cpp:801) [21248][AutoLSLocRpc][T0][YB4A0A000407-00061E9B390A2278-0-0] [lt=73] [LS_LOCATION] Get ls leaders by RPC(ret=0, ret="OB_SUCCESS", dests=["10.0.4.7:2890", "10.0.4.8:2890", "10.0.4.9:2890"], leaders=[{key:{tenant_id:1, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:48}}, {key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33705}}, {key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:246}}, {key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:130}}, {key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:33576}}, {key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1717559948}, location:{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}, {key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:88}}, {key:{tenant_id:1004, ls_id:{id:1002}, cluster_id:1717559948}, location:{server:"10.0.4.8:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:23}}, {key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:81}}, {key:{tenant_id:1004, ls_id:{id:1003}, cluster_id:1717559948}, location:{server:"10.0.4.9:2890", role:1, sql_port:2889, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:17}}]) [2024-10-16 14:58:14.227639] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C136B-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362096031, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894024, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.231264] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C136D-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362099658, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894025, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.235976] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C136F-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362104370, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894026, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.242409] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1371-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362110803, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894027, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.243972] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061894243530000, start_id=1729061894243530000, end_id=1729061894243530001) [2024-10-16 14:58:14.244044] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=24, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:14.244058] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:14.249037] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061894248869000, start_id=1729061894248869000, end_id=1729061894248869001) [2024-10-16 14:58:14.250100] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:14.250117] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:14.250153] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1373-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362118547, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894028, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.255089] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1375-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362123483, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894029, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.256666] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1377-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362125062, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894030, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.257829] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1253, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:14.257851] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1533, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:14.257877] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=2768, avg_log_submit_to_slide_time=3064) [2024-10-16 14:58:14.257950] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1764, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:14.257961] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1042, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:14.258002] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:14.258011] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3302, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:14.258016] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:14.261865] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1379-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362130261, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894031, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.263978] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7] ====== checkpoint timer task ====== [2024-10-16 14:58:14.264727] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] succ to get rec scn(*this={this:0x2b0e5ac04030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129005, v:0}, max_applied_start_working_ts_:{val:1722494261263129005, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667423836000, v:0}, last_push_gc_task_ts_:1722495955146057, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:686}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.264753] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95628080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1722495955149667, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95628080, snapshot_version:{val:1722495955149667, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.264772] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=14] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1725269666419576000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:1725269666419576000, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:14.264891] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015308598471003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, }, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061910721265000, latest_log_ts:{val:1729061894075060001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:14.264941] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=46] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x2b0e5ae961b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x2b0e5aea40d0, memtable_head:143831, memtable_tail:143832, t3m:0x2b0e3fd1e030, tables:[null, null, null, null, null, null, null, 0x2b0e9561bbf0, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1729015204647633, tx_data_table:0x2b0e5aea5c50, ls_tablet_svr:0x2b0e5ae96190}, min_rec_scn={val:1729015206603552000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:14.264962] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448818782043002, v:0}) [2024-10-16 14:58:14.264972] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=0, rec_log_ts={val:1729061894075060001, v:0}) [2024-10-16 14:58:14.264979] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.264986] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015206603552000, v:0}, checkpoint_scn_in_ls_meta={val:1729015206603552000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:14.265001] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:17380134912}, new_base_lsn={lsn:17380134912}, lsn={lsn:17380134912}) [2024-10-16 14:58:14.265012] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:17380134912}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:14.265705] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5acb6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261263129002, v:0}, max_applied_start_working_ts_:{val:1722494261263129002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751300105445001, v:0}, last_push_gc_task_ts_:1722494153253287, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:332}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.265722] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e95629350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e95629350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.265733] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751300105445000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2024-10-16 14:58:14.265800] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015311100270003, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, }, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:14.265836] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=35] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015311100270003, v:0}, ls_id_={id:1001}) [2024-10-16 14:58:14.265863] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=25] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448820367858002, v:0}) [2024-10-16 14:58:14.265870] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015311100270003, v:0}, checkpoint_scn_in_ls_meta={val:1729015311100270003, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2024-10-16 14:58:14.265880] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:13555163136}, new_base_lsn={lsn:13555163136}, lsn={lsn:13555163136}) [2024-10-16 14:58:14.265888] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:13555163136}, ls->get_ls_id()={id:1001}) [2024-10-16 14:58:14.266575] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5ca04030, ls_id:{id:1002}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1002}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494433651538001, v:0}, max_applied_start_working_ts_:{val:1722494433651538001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751307386556001, v:0}, last_push_gc_task_ts_:1722494153281026, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:356}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.266587] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562a620, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562a620, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.266597] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751307386556000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1002}) [2024-10-16 14:58:14.266673] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=12] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015305082700002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, }, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:14.266702] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015305082700002, v:0}, ls_id_={id:1002}) [2024-10-16 14:58:14.266725] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1002}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822792156001, v:0}) [2024-10-16 14:58:14.266731] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015305082700002, v:0}, checkpoint_scn_in_ls_meta={val:1729015305082700002, v:0}, ls_id={id:1002}, service_type="TRANS_SERVICE") [2024-10-16 14:58:14.266737] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14427525120}, new_base_lsn={lsn:14427525120}, lsn={lsn:14427525120}) [2024-10-16 14:58:14.266742] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14427525120}, ls->get_ls_id()={id:1002}) [2024-10-16 14:58:14.267208] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000408-00061E9B451E1E12-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061894266309000, start_id=1729061894266309000, end_id=1729061894266309001) [2024-10-16 14:58:14.267428] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=3] succ to get rec scn(*this={this:0x2b0e5cab6030, ls_id:{id:1003}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1003}, cur_state_.state_val_:{state_:"F_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494600220969002, v:0}, max_applied_start_working_ts_:{val:1722494600220969002, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1717751303551481001, v:0}, last_push_gc_task_ts_:1722494153303830, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:354}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.267441] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=13] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e9562b8f0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e9562b8f0, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:14.267451] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717751303551481000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1003}) [2024-10-16 14:58:14.267525] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=5] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1729015265583755004, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, }, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:14.267562] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=36] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061891073192004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1729015265583755004, v:0}, ls_id_={id:1003}) [2024-10-16 14:58:14.267585] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=22] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1003}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722448822511595002, v:0}) [2024-10-16 14:58:14.267590] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015265583755004, v:0}, checkpoint_scn_in_ls_meta={val:1729015265583755004, v:0}, ls_id={id:1003}, service_type="TRANS_SERVICE") [2024-10-16 14:58:14.267596] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:14293315584}, new_base_lsn={lsn:14293315584}, lsn={lsn:14293315584}) [2024-10-16 14:58:14.267614] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=17] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:14293315584}, ls->get_ls_id()={id:1003}) [2024-10-16 14:58:14.267619] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [22796][T1004_TxCkpt][T1004][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=4) [2024-10-16 14:58:14.270001] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894268407000, start_id=1729061894268407000, end_id=1729061894268407001) [2024-10-16 14:58:14.272882] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [22325][T1003_COMMONLSS][T1003][YB4A0A000407-00061E9B3A19CB34-0-0] [lt=22][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:58:14.284696] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C137B-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362153090, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894032, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.284835] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=17, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:14.284848] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:14.286702] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C137D-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362155097, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894033, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.288201] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C137F-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362156597, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894034, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.289343] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061894288056000, start_id=1729061894288056000, end_id=1729061894288056001) [2024-10-16 14:58:14.289669] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1381-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362158064, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894035, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.290267] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.294104] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061894293921000, start_id=1729061894293921000, end_id=1729061894293921001) [2024-10-16 14:58:14.294537] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:14.295621] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 120.75; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 18709.33, iops: 383.12, rt: 201.02; ) [2024-10-16 14:58:14.295642] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:14.295698] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=31] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 134.75; ) [2024-10-16 14:58:14.295704] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:14.295715] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:14.307753] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85225-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:14.307786] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85225-0-0] [lt=18][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497747, log_task={header:{begin_lsn:{lsn:207589637556}, end_lsn:{lsn:207589637678}, log_id:218497747, min_scn:{val:1729061894178211000, v:0}, max_scn:{val:1729061894178211000, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589637434}, prev_proposal_id:81, committed_end_lsn:{lsn:207589637556}, data_checksum:940009244, accum_checksum:2523814119, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061894207676, freeze_ts:1729061894207676, submit_ts:1729061894207681, flushed_ts:1729061894208092, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:411}, fs_cb_begin_ts=1729061894307780, log_life_time=100104) [2024-10-16 14:58:14.308962] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000408-00061E9B453E1E07-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061894308175000, start_id=1729061894308175000, end_id=1729061894308175001) [2024-10-16 14:58:14.309314] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1729061894309312, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007858, add_bytes: 0 [2024-10-16 14:58:14.310647] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1383-0-0] [lt=9][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362179039, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894036, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.310814] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.315330] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=12] ====== check clog disk timer task ====== [2024-10-16 14:58:14.315350] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:14.315369] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373780724, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:14.315783] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=8] run rewrite rule refresh task(rule_mgr_->tenant_id_=1002) [2024-10-16 14:58:14.316503] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [22137][T1002_ReqMemEvi][T1002][YB4A0A000407-00061E9B3AC985E7-0-0] [lt=13] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=1, total_release_part_ctx_count=189) [2024-10-16 14:58:14.316595] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22137][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061894316047000, start_id=1729061894316047000, end_id=1729061894316047001) [2024-10-16 14:58:14.318957] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1577222144, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:14.320899] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1385-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362189292, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894037, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.323193] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1387-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362191589, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894038, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.327967] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E0B-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362196361, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410663, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.331113] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1723, mem_used=4159488) [2024-10-16 14:58:14.332619] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1389-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362200999, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894039, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.333941] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C138B-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362202337, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894040, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.335388] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C138D-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362203784, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894041, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.336756] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=14] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:14.340714] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000409-00061E9A11578B29-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061894339558000, start_id=1729061894339558000, end_id=1729061894339558001) [2024-10-16 14:58:14.342508] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [22315][T1003_PalfGC][T1003][Y0-0000000000000000-0-0] [lt=24] BlockGCTimerTask success(ret=0, cost_time_us=12, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"10.0.4.7:2890", log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):614, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):114, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2024-10-16 14:58:14.347281] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C138F-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362215674, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894042, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.348808] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1391-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362217202, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894043, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.350430] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:14.350484] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1764) [2024-10-16 14:58:14.351997] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:14.352014] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=12] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=12582912, ctx.total_memstore_used_=46137344, ctx.total_memstore_hold_=46137344, memstore_freeze_trigger_limit_=28230020, tenant_id=1002) [2024-10-16 14:58:14.352036] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:14.359655] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1393-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362228048, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894044, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.362747] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1395-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362231141, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894045, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.369458] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1397-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362237852, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894046, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.371471] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1399-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362239864, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894047, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.371499] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5405C-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=448, avg_log_submit_to_slide_time=20447) [2024-10-16 14:58:14.371517] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5405C-0-0] [lt=16][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173477, log_task={header:{begin_lsn:{lsn:14352414286}, end_lsn:{lsn:14352414408}, log_id:115173477, min_scn:{val:1729061894177150000, v:0}, max_scn:{val:1729061894177150000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352414164}, prev_proposal_id:17, committed_end_lsn:{lsn:14352414286}, data_checksum:940009244, accum_checksum:2942642030, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061894271321, freeze_ts:1729061894271321, submit_ts:1729061894271325, flushed_ts:1729061894271992, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:667}, fs_cb_begin_ts=1729061894371492, log_life_time=100171) [2024-10-16 14:58:14.385509] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21658][T1_L0_G0][T1][YB4A0A000409-00061E9A2ACCE8AB-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362253903, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410664, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.385784] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79E9-0-0] [lt=24][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938603, log_task={header:{begin_lsn:{lsn:213058334755}, end_lsn:{lsn:213058334877}, log_id:220938603, min_scn:{val:1729061894170735001, v:0}, max_scn:{val:1729061894170735001, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058334633}, prev_proposal_id:88, committed_end_lsn:{lsn:213058334755}, data_checksum:1715074492, accum_checksum:1476980944, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061894285765, freeze_ts:1729061894285765, submit_ts:1729061894285768, flushed_ts:1729061894286823, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:1055}, fs_cb_begin_ts=1729061894385777, log_life_time=100012) [2024-10-16 14:58:14.386054] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:14.386073] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=52657) [2024-10-16 14:58:14.390661] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.391555] INFO [SHARE] adaptive_update_limit (ob_share_throttle_define.cpp:112) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789E2-0-0] [lt=12] adaptive update(Tenant ID=1004, Config Specify Resource Limit(MB)=511, TxShare Current Memory Limit(MB)=154, Holding Memory(MB)=46, Tenant Remain Memory(MB)=179, Usable Remain Memory(MB)=107, Last Update Limit Timestamp=1729061894390415, Is Updated=true) [2024-10-16 14:58:14.391653] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000409-00061E9A11A789E2-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894390415000, start_id=1729061894390415000, end_id=1729061894390415001) [2024-10-16 14:58:14.392984] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=46283739, this={dio_aligned_buf:{buf_write_offset:3582, buf_padding_size:514, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33202095, truncate_used_ts:11502202}, log_block_size:67108864, dir_fd:195, io_fd:131}, aligned_buf_len=4096, aligned_block_offset=46280704, buf_len=547, total_write_size_=15422, total_write_size_after_dio_=421888, ob_pwrite_used_ts=62760, count=100, each_pwrite_cost=627) [2024-10-16 14:58:14.393872] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:14.393886] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54432140, tenant_id=1003) [2024-10-16 14:58:14.393897] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:14.395134] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C139B-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362263527, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894048, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.396136] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:58:14.401333] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=6] Garbage Collector is running(seq_=656752, gc_interval=10000000) [2024-10-16 14:58:14.401375] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=18] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:14.401389] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=36) [2024-10-16 14:58:14.401395] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=3] execute_gc cost time(ret=0, time_us=1) [2024-10-16 14:58:14.401401] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=4] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:14.401409] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=6] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:14.401414] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=6) [2024-10-16 14:58:14.401418] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21547][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=3] ObSafeDestroyHandler start process [2024-10-16 14:58:14.409235] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C139D-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362277628, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894049, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.409620] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:14.410874] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=31] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=27, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:14.410892] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=4212, avg_group_log_size=351) [2024-10-16 14:58:14.410923] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=28] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:14.410926] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.411253] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1538, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:14.411266] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=4090) [2024-10-16 14:58:14.411289] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=104, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:14.411306] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=4090) [2024-10-16 14:58:14.411314] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=10, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:14.411626] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C117D-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:14.411634] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C117D-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3138, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:14.411638] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C117D-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=37, avg_log_submit_to_flush_time=1970, avg_log_submit_to_slide_time=2885) [2024-10-16 14:58:14.411961] INFO [PALF] read (log_cache.cpp:103) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=50332, hit_cnt=13, read_cnt=13, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:14.417071] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA8A0-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:14.417096] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA8A0-0-0] [lt=15] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061894178211002, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061894178211002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:14.417706] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C139F-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362286098, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894050, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.418116] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA8A0-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061894417188000, start_id=1729061894417188000, end_id=1729061894417188001) [2024-10-16 14:58:14.421917] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA8A0-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061894421433000, start_id=1729061894421433000, end_id=1729061894421433001) [2024-10-16 14:58:14.422023] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA8A0-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10753, avg_group_log_size=632) [2024-10-16 14:58:14.422035] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA8A0-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10590) [2024-10-16 14:58:14.422377] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=310, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:14.422398] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=20] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10753) [2024-10-16 14:58:14.422415] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=78, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:14.422430] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10590) [2024-10-16 14:58:14.422443] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:14.422645] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F96F6-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:14.422657] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F96F6-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=666, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:14.422663] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F96F6-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=414, avg_log_submit_to_slide_time=633) [2024-10-16 14:58:14.422738] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=9] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:14.427970] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13A1-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362296363, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894051, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.432747] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:183802073, max_lsn:114265069537, max_scn:{val:1729061894288056000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:45336899169}, reuse_lsn:{lsn:114265069537}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:114223126497}}, last_submit_lsn:{lsn:114265068944}, last_submit_end_lsn:{lsn:114265069537}, last_submit_log_id:183802073, last_submit_log_pid:33576, max_flushed_lsn:{lsn:114265068944}, max_flushed_end_lsn:{lsn:114265069537}, max_flushed_log_pid:33576, committed_end_lsn:{lsn:114265069537}, last_slide_log_id:183802073, last_slide_scn:{val:1729061894288056000, v:0}, last_slide_lsn:{lsn:114265068944}, last_slide_end_lsn:{lsn:114265069537}, last_slide_log_pid:33576, last_slide_log_accum_checksum:1506837981, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e2efe2530}, [StateMgr]={this:0x2b0e2efe6830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, leader:"10.0.4.7:2890", leader_epoch:2, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:183802074, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261840845, palf_role_change_cb:0x2b0e2efed230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, reconfig_barrier:{prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_end_lsn:{lsn:45338084276}, prev_mode_pid:33576}, persistent_config_version:{proposal_id:33576, config_seq:9}, ms_ack_list:0, resend_config_version:{proposal_id:33576, config_seq:9}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494259837171, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494259837171, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e2efe3c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262041271, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1729031774558563002, v:0}, min_block_id:1697, min_block_min_scn:{val:1729025370959302001, v:0}, base_lsn_for_block_gc:{lsn:114212315136}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:33576}, log_config_meta:{version:3, proposal_id:33576, prev:{version:1, config:{config_version:{proposal_id:33575, config_seq:8}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:33576, config_seq:9}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:33575, prev_lsn:{lsn:45338083519}, prev_mode_pid:33576}, log_snapshot_meta:{version:1, base_lsn:{lsn:114212315136}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:33576, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:232079360}, readable_log_tail:{lsn:232079360}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114279419904}, min_scn:{val:0, v:0}, curr_block_id:3, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/meta", dir_fd:187, min_block_id:3, max_block_id:4, curr_writable_block_id:3}, logical_block_size_:67104768, curr_block_writable_size_:36339712, block_header_serialize_buf_:0x2b0e2efe89f8, flashback_version:0}, log_storage:{log_tail:{lsn:114265069537}, readable_log_tail:{lsn:114265069537}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:114212315136}, min_scn:{val:1729057087482360001, v:0}, curr_block_id:1702, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1003/1/log", dir_fd:190, min_block_id:1697, max_block_id:1703, curr_writable_block_id:1702}, logical_block_size_:67104768, curr_block_writable_size_:14350367, block_header_serialize_buf_:0x2b0e2efea6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261839896, this:0x2b0e2efe6f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:33576, prepare_log_ack_list:2"10.0.4.9:2890";"10.0.4.8:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:33575, majority_max_lsn:{lsn:45338084276}, saved_end_lsn:{lsn:45338084276}, last_submit_prepare_req_time_us:1722494261840968, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e2efe69b0}) [2024-10-16 14:58:14.432921] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=169] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061894290049, last_flushed_end_lsn:{lsn:114265069537}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061894290081, last_flushed_end_lsn:{lsn:114265069537}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061894290044, last_flushed_end_lsn:{lsn:114265069537}}]) [2024-10-16 14:58:14.432933] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=183802074, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061894290049, last_flushed_end_lsn:{lsn:114265069537}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061894290081, last_flushed_end_lsn:{lsn:114265069537}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061894290044, last_flushed_end_lsn:{lsn:114265069537}}]) [2024-10-16 14:58:14.444427] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061894443432000, start_id=1729061894443432000, end_id=1729061894443432001) [2024-10-16 14:58:14.444612] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22A8-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362312998, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894052, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.449867] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061894448684000, start_id=1729061894448684000, end_id=1729061894448684001) [2024-10-16 14:58:14.453151] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=48, start_service_tstamp_=1722494262618751, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:14.457976] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=254, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:14.457994] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=60, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:14.466844] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [29808][T1002_L0_G0][T1002][YB4A0A000407-00061E9B2ECA223F-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362335237, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894053, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.468165] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCCC4-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061894467127000, start_id=1729061894467127000, end_id=1729061894467127001) [2024-10-16 14:58:14.470474] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061894469247000, start_id=1729061894469247000, end_id=1729061894469247001) [2024-10-16 14:58:14.474354] INFO [STORAGE.TRANS] handle_2pc_commit_request_impl_ (ob_two_phase_downstream_committer.cpp:434) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=15] committer is under logging(ret=0, *this={this:0x2b0e0c20c5d0, ref:4, trans_id:{txid:392285937}, tenant_id:1002, is_exiting:false, trans_expired_time:1729062494435967, cluster_version:17180066304, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1729061894435967}{self_ls_id:{id:1001}, session_id:3221769135, part_trans_action:3, pending_write:0, 2pc_role:"LEAF", ctx_tx_data_:{ctx_mgr_:0x2b0e26cb6030, tx_data_guard_:{tx_data:{tx_id:{txid:392285937}, ref_cnt:1, state:"RUNNING", commit_version:{val:1729061894467127000, v:0}, start_scn:{val:1729061894316047001, v:0}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:36834334451937172, replay_completeness_:1, upstream_state:"COMMIT", collected:[], rec_log_ts:{val:1729061894316047001, v:0}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1729061894463948, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008951550040 max_table_version=1729008951550040 trans_version={val:1729061894316047001, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285937} ls_id=1001 row_callback[alloc:308, free:0, unsubmit:0] redo[fill:318,sync_succ:318, sync_fail:0] main_list_len=318 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,318,318,318,318,0,0,0)|]}} -{busy_cbs_cnt:1, oldest_busy_cb:{ObTxBaseLogCb:{base_ts:{val:1729061894467127000, v:0}, log_ts:{val:1729061894467127000, v:0}, lsn:{lsn:204023245209}, submit_ts:1729061894473584}, this:0x2b0e0c20ff40, is_inited_:true, trans_id:{txid:392285937}, ls_id:{id:1001}, ctx:0x2b0e0c20c5d0, tx_data_guard:{tx_data:NULL}, is_callbacked_:false, is_dynamic_:false, mds_range_:{range_array_.count():0, range_array_:[]}, cb_arg_array_:[{log_type:0x100, arg:null}], first_part_scn_:{val:18446744073709551615, v:3}, extra_cb_:null, need_free_extra_cb_:false, callbacks_.count():0}, final_log_cb:{ObTxBaseLogCb:{base_ts:{val:1729061894467127000, v:0}, log_ts:{val:1729061894467127000, v:0}, lsn:{lsn:204023245209}, submit_ts:1729061894473584}, this:0x2b0e0c20ff40, is_inited_:true, trans_id:{txid:392285937}, ls_id:{id:1001}, ctx:0x2b0e0c20c5d0, tx_data_guard:{tx_data:NULL}, is_callbacked_:false, is_dynamic_:false, mds_range_:{range_array_.count():0, range_array_:[]}, cb_arg_array_:[{log_type:0x100, arg:null}], first_part_scn_:{val:18446744073709551615, v:3}, extra_cb_:null, need_free_extra_cb_:false, callbacks_.count():0}, -{info_log_submitted:1 state_log_submitting:1 state_log_submitted:1 }-, coord_prepare_info_arr_:[{id_:{id:1003}, offset_:{lsn:207589668901}}, {id_:{id:1001}, offset_:{lsn:204023233432}}, {id_:{id:1002}, offset_:{lsn:213058366341}}], -{downstream_state:"PRE_COMMIT", upstream:{id:1002}, participants:[], redo_log_no:1, scheduler:"10.0.4.8:2890", prepare_version:{val:1729061894316047002, v:0}, trans_type:2, next_log_entry_no:3, max_applied_log_ts:{val:1729061894316047002, v:0}, max_appling_log_ts:{val:1729061894316047002, v:0}, max_applying_part_log_no:0, max_submitted_seq_no:{branch:0, seq:410893}, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, redo_lsns:[{lsn:204023198918}], max_durable_lsn:{lsn:204023233432}, prepare_log_info_arr:[{id_:{id:1001}, offset_:{lsn:204023233432}}], }- }-}) [2024-10-16 14:58:14.479906] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13A3-0-0] [lt=91][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362348299, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894054, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.481729] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13A5-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362350122, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894055, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.485031] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13A7-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362353426, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894056, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.485729] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2200, avg_group_log_size=200) [2024-10-16 14:58:14.485745] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2200) [2024-10-16 14:58:14.486084] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=9, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:14.486671] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44F3-0-0] [lt=32] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:14.486682] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44F3-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1444, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:14.486991] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13A9-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362355387, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894057, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.489790] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061894488273000, start_id=1729061894488273000, end_id=1729061894488273001) [2024-10-16 14:58:14.490725] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.492249] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13AB-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362360644, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894058, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.494126] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061894492294000, start_id=1729061894492294000, end_id=1729061894492294001) [2024-10-16 14:58:14.494470] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EDD-0-0] [lt=17] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:14.494875] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=218, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:14.494894] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=54, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:14.494927] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:14.498015] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13AD-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362366409, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894059, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.500122] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2AA0-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:14.500152] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2AA0-0-0] [lt=29] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:14.505932] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13AF-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362374326, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894060, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.507375] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13B1-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362375770, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894061, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.507909] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85229-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=102211) [2024-10-16 14:58:14.509316] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13B3-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362377710, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894062, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.510988] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.512618] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13B5-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362381013, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894063, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.516228] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13B7-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362384622, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894064, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.519252] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1179615232, global_cache_size=1585545216, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:14.519579] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash memory, (tenant_id=1002, cache_size=91553792, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:58:14.519709] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash time detail, (compute_wash_size_time=102, refresh_score_time=70, wash_time=247) [2024-10-16 14:58:14.521641] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22831][T1004_L5_G0][T1004][YB4A0A000408-00061E9B458E1C8C-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061894520671000, start_id=1729061894520671000, end_id=1729061894520671001) [2024-10-16 14:58:14.522447] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=7] Cache wash clean map node details(ret=0, clean_node_count=18, clean_time=0, clean_start_pos=2139076, clean_num=62914) [2024-10-16 14:58:14.526252] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13BA-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362394647, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894065, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.527447] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13BC-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061894526754000, start_id=1729061894526754000, end_id=1729061894526754001) [2024-10-16 14:58:14.530223] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22AC-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362398618, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894066, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.534956] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=10] ====== check clog disk timer task ====== [2024-10-16 14:58:14.534975] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4386) [2024-10-16 14:58:14.534993] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] cannot_recycle_log_size statistics(cannot_recycle_log_size=1445013576, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:14.537389] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13C0-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362405782, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894067, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.538892] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=24] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:14.538957] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13C2-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362407352, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894068, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.540531] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13C4-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362408927, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894069, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.543620] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13C6-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362412015, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894070, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.544378] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13C7-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362412774, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894071, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.547936] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13C9-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362416332, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894072, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.549374] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13CB-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362417770, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894073, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.551135] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5E8-0-0] [lt=17] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1211) [2024-10-16 14:58:14.552263] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13CD-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362420659, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894074, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.553874] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13CF-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362422269, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894075, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.560286] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13D1-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362428681, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894076, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.560288] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22AF-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362428683, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894077, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.561860] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13D3-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362430256, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894078, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.567004] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13D5-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362435396, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894079, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.567448] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22B1-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362435842, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894080, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.569326] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.569346] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.569358] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.569676] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.571624] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54060-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173479, log_task={header:{begin_lsn:{lsn:14352414530}, end_lsn:{lsn:14352414652}, log_id:115173479, min_scn:{val:1729061894177150002, v:0}, max_scn:{val:1729061894177150002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352414408}, prev_proposal_id:17, committed_end_lsn:{lsn:14352414530}, data_checksum:940009244, accum_checksum:3062715663, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061894471445, freeze_ts:1729061894471445, submit_ts:1729061894471449, flushed_ts:1729061894472273, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:824}, fs_cb_begin_ts=1729061894571617, log_life_time=100172) [2024-10-16 14:58:14.571676] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:14.574470] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22B3-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362442865, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894081, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.574470] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13D7-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362442865, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894082, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.576144] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13D9-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362444540, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894083, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.576195] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=3] get number(ret=0, service_type_=0, range=1, base_id=1729061894575612000, start_id=1729061894575612000, end_id=1729061894575612001) [2024-10-16 14:58:14.576211] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=12] ts response task statistics(alloc_count=261365754, free_count=130682876) [2024-10-16 14:58:14.576226] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] gts statistics(tenant_id=1001, gts_rpc_cnt=10, get_gts_cache_cnt=493, get_gts_with_stc_cnt=72, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=72, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:14.576417] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=508, get_gts_with_stc_cnt=8, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=4, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:14.581048] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13DB-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362449442, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894084, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.584797] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22B4-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362453190, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894085, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.584960] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13DE-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362453355, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894086, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.585863] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79EB-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:14.586077] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=104763) [2024-10-16 14:58:14.586421] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D564F-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:14.586432] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D564F-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=18925, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:14.586478] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=24] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=33, stat_interval=1000000, avg (count/cost)=75, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:14.586484] INFO [LIB] stat (utility.h:1165) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=6] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=33, stat_interval=1000000, avg (count/cost)=222, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:14.586707] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F44F5-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=330, avg_log_submit_to_slide_time=737) [2024-10-16 14:58:14.587361] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13E0-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362455756, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894087, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.589395] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13E3-0-0] [lt=9][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362457790, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894088, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.590787] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.595712] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13E7-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362464106, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894089, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.598410] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:14.598425] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:14.599015] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1003}, {id:1002}]) [2024-10-16 14:58:14.605166] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13E9-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362473560, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894090, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.608751] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8522B-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=16, avg_log_batch_cnt=0, total_group_log_size=68577, avg_group_log_size=4286) [2024-10-16 14:58:14.609135] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=232, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:14.609148] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=68577) [2024-10-16 14:58:14.609160] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=120, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:14.609201] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=20, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:14.610024] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13EB-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362478417, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894091, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.610939] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=5419) [2024-10-16 14:58:14.611064] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.611151] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] Cache replace map node details(ret=0, replace_node_count=41, replace_time=0, replace_start_pos=0, replace_num=31457) [2024-10-16 14:58:14.611207] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=55] replace map num details(ret=0, replace_node_count=41, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:14.611208] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23642375, this={dio_aligned_buf:{buf_write_offset:385, buf_padding_size:3711, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:47004239, truncate_used_ts:19010378}, log_block_size:67108864, dir_fd:134, io_fd:183}, aligned_buf_len=4096, aligned_block_offset=23642112, buf_len=122, total_write_size_=39540, total_write_size_after_dio_=483328, ob_pwrite_used_ts=42650, count=108, each_pwrite_cost=394) [2024-10-16 14:58:14.611620] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601534, offset=9662180}, end_cursor:ObLogCursor{file_id=192, log_id=47601535, offset=9662464}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd22f4050, len:284, log_data_len:284, seq:47601534, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:14.612627] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6028, mem_used=12478464) [2024-10-16 14:58:14.612755] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13EC-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362481150, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894092, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.613558] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A11478B33-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061894612799000, start_id=1729061894612799000, end_id=1729061894612799001) [2024-10-16 14:58:14.615401] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4C4-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061894614918000, start_id=1729061894614918000, end_id=1729061894614918001) [2024-10-16 14:58:14.616464] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601535, offset=9662464}, end_cursor:ObLogCursor{file_id=192, log_id=47601536, offset=9662738}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2150050, len:4096, log_data_len:274, seq:47601535, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:14.617942] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13EE-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362486337, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894093, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.618698] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22487][T1003_L5_G0][T1003][YB4A0A000408-00061E9B454E1E06-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061894618193000, start_id=1729061894618193000, end_id=1729061894618193001) [2024-10-16 14:58:14.618943] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=52759683, this={dio_aligned_buf:{buf_write_offset:3752, buf_padding_size:344, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:80311374, truncate_used_ts:29009631}, log_block_size:67108864, dir_fd:190, io_fd:179}, aligned_buf_len=4096, aligned_block_offset=52756480, buf_len=549, total_write_size_=101602, total_write_size_after_dio_=798720, ob_pwrite_used_ts=40180, count=171, each_pwrite_cost=234) [2024-10-16 14:58:14.621498] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22B7-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362489892, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894094, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.621796] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601536, offset=9662738}, end_cursor:ObLogCursor{file_id=192, log_id=47601537, offset=9663012}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2134050, len:3822, log_data_len:274, seq:47601536, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:14.621944] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601537, offset=9663012}, end_cursor:ObLogCursor{file_id=192, log_id=47601538, offset=9663286}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20e4050, len:3548, log_data_len:274, seq:47601537, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:14.622010] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599201, miss_cnt:9, cache_read_size:149778054355, hit_ratio:"9.999999345926476879e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:14.622024] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561616, miss_cnt:1, cache_read_size:152590827941, hit_ratio:"9.999999928347060862e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:14.623088] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13EF-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362491483, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894095, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.623239] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=9] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257882, offset=39066901}, end_cursor:ObLogCursor{file_id=197, log_id=49257883, offset=39067184}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd215c050, len:747, log_data_len:283, seq:49257882, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:14.626532] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB38F-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061894625769000, start_id=1729061894625769000, end_id=1729061894625769001) [2024-10-16 14:58:14.627746] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=14] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257883, offset=39067184}, end_cursor:ObLogCursor{file_id=197, log_id=49257884, offset=39067466}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd22d8050, len:464, log_data_len:282, seq:49257883, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:14.631200] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257884, offset=39067466}, end_cursor:ObLogCursor{file_id=197, log_id=49257885, offset=39067748}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21a4050, len:4278, log_data_len:282, seq:49257884, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:14.631363] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257885, offset=39067748}, end_cursor:ObLogCursor{file_id=197, log_id=49257886, offset=39068030}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20ec050, len:3996, log_data_len:282, seq:49257885, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:14.631408] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650543, miss_cnt:32, cache_read_size:8050457623, hit_ratio:"9.999994892305458238e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:14.631428] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657282, miss_cnt:17, cache_read_size:8050204995, hit_ratio:"9.999997286828467002e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:14.633703] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13F1-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362502097, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894096, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.635265] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13F3-0-0] [lt=34][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362503661, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894097, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.635316] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22B9-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362503710, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894098, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.641038] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13F5-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362509431, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894099, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.642665] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13F7-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362511060, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894100, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.644737] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061894642873000, start_id=1729061894642873000, end_id=1729061894642873001) [2024-10-16 14:58:14.646175] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13F8-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362514571, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894101, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.649286] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13FB-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362517681, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894102, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.649867] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=30] get number(ret=0, service_type_=0, range=1, base_id=1729061894649579000, start_id=1729061894649579000, end_id=1729061894649579001) [2024-10-16 14:58:14.650278] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22244][T1002_ArcSender][T1002][YB4A0A000407-00061E996F6960C1-0-0] [lt=15][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:14.651272] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13FD-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362519666, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894103, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.651648] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061894449967001, v:0}, new_max_version={val:1729061894449967001, v:0}, last_min_version={val:1729061894248869000, v:0}, last_max_version={val:1729061894248869000, v:0}, total_time_us=2542, delta_us=201679) [2024-10-16 14:58:14.651664] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061894449967001, v:0}, min_version={val:1729061894449967001, v:0}, max_version={val:1729061894449967001, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:58:14.659937] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C13FF-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362528331, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894104, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.661569] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1401-0-0] [lt=28][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362529963, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894105, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.664177] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1403-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362532571, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894106, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.669705] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1405-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362538100, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894107, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.670537] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061894669002000, start_id=1729061894669002000, end_id=1729061894669002001) [2024-10-16 14:58:14.672029] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0D9F-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061894670695000, start_id=1729061894670695000, end_id=1729061894670695001) [2024-10-16 14:58:14.672473] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1407-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362540868, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894108, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.672579] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0DA0-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061894670695000, start_id=1729061894670695000, end_id=1729061894670695001) [2024-10-16 14:58:14.672613] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0DA2-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061894670695000, start_id=1729061894670695000, end_id=1729061894670695001) [2024-10-16 14:58:14.672656] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0DA1-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894670695000, start_id=1729061894670695000, end_id=1729061894670695001) [2024-10-16 14:58:14.672699] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0DA3-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061894670695000, start_id=1729061894670695000, end_id=1729061894670695001) [2024-10-16 14:58:14.674575] INFO [ARCHIVE] do_thread_task_ (ob_archive_fetcher.cpp:310) [22243][T1002_ArcFetche][T1002][YB4A0A000407-00061E996F5960C1-0-0] [lt=23] ObArchiveFetcher is running(thread_index=0) [2024-10-16 14:58:14.677409] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:14.678890] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1409-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362547284, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894109, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.678919] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22BB-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362547312, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894110, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.680842] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C140B-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362549237, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894111, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.684348] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C140D-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362552736, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894112, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.685801] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79ED-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=15, avg_log_batch_cnt=0, total_group_log_size=54387, avg_group_log_size=3625) [2024-10-16 14:58:14.685907] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C140F-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362554303, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894113, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.686002] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:14.686017] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:14.686042] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=47962, avg_group_log_size=3425) [2024-10-16 14:58:14.686049] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=47962) [2024-10-16 14:58:14.686351] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2194) [2024-10-16 14:58:14.686381] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2194) [2024-10-16 14:58:14.686497] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=316, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:14.686509] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=219, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:14.686786] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46D8-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:14.686796] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46D8-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=811, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:14.687915] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1411-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362556311, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894114, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.689328] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1413-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362557725, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894115, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.689698] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061894689677000, start_id=1729061894689677000, end_id=1729061894689677001) [2024-10-16 14:58:14.690847] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.694772] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061894694163000, start_id=1729061894694163000, end_id=1729061894694163001) [2024-10-16 14:58:14.695421] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22398][T1003_TntShared][T1003][Y0-0000000000000000-0-0] [lt=14] clear dtl interm result cost(us)(clear_cost=16, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:58:14.702651] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1415-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362571045, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894116, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.705136] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=12] gc stale ls task succ [2024-10-16 14:58:14.706478] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1417-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362574872, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894117, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.708149] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1419-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362576544, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894118, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.708274] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31896][T1004_TTLTablet][T1004][YB4A0A000407-00061E9AB1FCC6DE-0-0] [lt=27] finish reload tenant task(local_tenant_task_={tenant_id:1004, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:58:14.708348] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8522D-0-0] [lt=5] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:14.708355] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8522D-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=25364, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:14.710013] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C141B-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362578408, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894119, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.711118] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=9] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2024-10-16 14:58:14.711641] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C141D-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362580036, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894120, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.711743] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.711902] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21525][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061894711773000, start_id=1729061894711773000, end_id=1729061894711773001) [2024-10-16 14:58:14.712645] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3E04-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=50, this=0x2b0dee9f4280, extra_info="") [2024-10-16 14:58:14.712657] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3E04-0-0] [lt=11] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=1030, this=0x2b0dee9f44b0, extra_info="") [2024-10-16 14:58:14.712661] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3E04-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=0, this=0x2b0dee9f46e0, extra_info="") [2024-10-16 14:58:14.712665] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3E04-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=-630, this=0x2b0dee9f4910, extra_info="") [2024-10-16 14:58:14.712668] INFO [LIB] stat (utility.h:1165) [21539][T1_ApplySrv0][T1][YB4A0A000407-00061E9B304F3E04-0-0] [lt=2] [APPLY STAT CB TOTAL TIME](cur_stat_count=28, stat_interval=5000000, avg (count/cost)=449, this=0x2b0dee9f4b40, extra_info="") [2024-10-16 14:58:14.713988] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C141F-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362582383, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894121, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.715285] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1421-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362583681, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894122, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.715354] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22BD-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362583749, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894123, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.716441] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:14.716461] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22240][T1002_TabletShe][T1002][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=219) [2024-10-16 14:58:14.716835] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1423-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362585232, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894124, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.723014] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1577222144, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:14.723763] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [21736][T1001_COMMONLSS][T1001][YB4A0A000407-00061E9B3A29CB1B-0-0] [lt=17][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2024-10-16 14:58:14.725648] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E0D-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362594042, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410665, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.727827] WDIAG get_refactored (ob_hashtable.h:1102) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED7A1-0-0] [lt=10][errcode=-4201] hashtable not init [2024-10-16 14:58:14.727837] WDIAG [SQL.SESSION] get_cursor (ob_sql_session_info.cpp:1505) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED7A1-0-0] [lt=10][errcode=-4016] get cursor info failed(cursor_id=1122, get_sessid()=3221565766) [2024-10-16 14:58:14.727914] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21678][T1_L0_G0][T1][YB4A0A000407-00061E9B274ED7A2-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061894719775000, start_id=1729061894719775000, end_id=1729061894719775001) [2024-10-16 14:58:14.731734] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:14.732669] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22272][T1002_L5_G0][T1002][YB4A0A000409-00061E9A11778B26-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894730947000, start_id=1729061894730947000, end_id=1729061894730947001) [2024-10-16 14:58:14.732987] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=42779623, this={dio_aligned_buf:{buf_write_offset:1547, buf_padding_size:2549, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:35133413, truncate_used_ts:12254405}, log_block_size:67108864, dir_fd:169, io_fd:144}, aligned_buf_len=4096, aligned_block_offset=42778624, buf_len=548, total_write_size_=15671, total_write_size_after_dio_=434176, ob_pwrite_used_ts=25247, count=102, each_pwrite_cost=247) [2024-10-16 14:58:14.733479] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=59] success to update min reserved snapshot(reserved_snapshot=1729060091854891000, duration=1800, snapshot_gc_ts=1729061891854891000) [2024-10-16 14:58:14.734510] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD5EB-0-0] [lt=25] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=31, avg_log_submit_to_flush_time=1252, avg_log_submit_to_slide_time=1656) [2024-10-16 14:58:14.735093] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=23194375, this={dio_aligned_buf:{buf_write_offset:2935, buf_padding_size:1161, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:33385133, truncate_used_ts:14052960}, log_block_size:67108864, dir_fd:200, io_fd:174}, aligned_buf_len=4096, aligned_block_offset=23191552, buf_len=112, total_write_size_=11928, total_write_size_after_dio_=413696, ob_pwrite_used_ts=42478, count=98, each_pwrite_cost=433) [2024-10-16 14:58:14.735122] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:14.735401] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=11] update reserved snapshot finished(cost_ts=1923, reserved_snapshot=1729060091854891000) [2024-10-16 14:58:14.736594] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1426-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362604988, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894125, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.745726] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1428-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362614119, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894127, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.745726] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22C1-0-0] [lt=9][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362614119, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894126, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.750507] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1210, avg_group_log_size=121) [2024-10-16 14:58:14.750523] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1201) [2024-10-16 14:58:14.756203] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C142A-0-0] [lt=16][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:58:14.762025] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22C2-0-0] [lt=14][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:58:14.762315] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=3] gc stale ls task succ [2024-10-16 14:58:14.762341] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=10] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:14.762349] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:14.773610] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:14.774057] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B5A0C6799-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061894773792000, start_id=1729061894773792000, end_id=1729061894773792001) [2024-10-16 14:58:14.777618] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B5A0C6799-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061894777564000, start_id=1729061894777564000, end_id=1729061894777564001) [2024-10-16 14:58:14.783358] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=14] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:14.786380] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79EF-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938610, log_task={header:{begin_lsn:{lsn:213058387959}, end_lsn:{lsn:213058388081}, log_id:220938610, min_scn:{val:1729061894670695000, v:0}, max_scn:{val:1729061894670695000, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058387837}, prev_proposal_id:88, committed_end_lsn:{lsn:213058387959}, data_checksum:1715074492, accum_checksum:751860349, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061894685789, freeze_ts:1729061894685789, submit_ts:1729061894685795, flushed_ts:1729061894686530, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:735}, fs_cb_begin_ts=1729061894786372, log_life_time=100583) [2024-10-16 14:58:14.786542] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=53814, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:14.788637] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.788713] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.788766] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.788779] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.788797] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.788881] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.788885] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=4] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.788902] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.788914] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.788926] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.789637] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:14.790907] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.793663] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B5A0C6799-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061894793368000, start_id=1729061894793368000, end_id=1729061894793368001) [2024-10-16 14:58:14.793828] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=12] LogLoopThread round_cost_time(us)(round_cost_time=15) [2024-10-16 14:58:14.793936] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=15][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:0, dropped:10, tid:12153}]) [2024-10-16 14:58:14.794275] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21400][T1_Occam][T1][YB4A0A000407-00061E9B361A15FA-0-0] [lt=4] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:14.794871] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EE3-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:14.794889] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EE3-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:14.797581] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C143D-0-0] [lt=1][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362665975, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894138, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.799963] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C143F-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362668359, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894139, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.804904] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD023-0-0] [lt=26] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061894795991, keepalive_ts:-1}) [2024-10-16 14:58:14.804918] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD023-0-0] [lt=14] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061894795991, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:14.805116] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1441-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362673511, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894140, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.805236] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD024-0-0] [lt=8] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061894796030, keepalive_ts:-1}) [2024-10-16 14:58:14.805247] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B65FFD024-0-0] [lt=10] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061894796030, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:14.811793] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.812107] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1443-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362680501, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894141, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.813496] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1445-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362681891, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894142, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.813589] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=57, replace_time=0, replace_start_pos=31457, replace_num=31457) [2024-10-16 14:58:14.813606] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=57, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:14.814963] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1447-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362683360, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894143, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.816246] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1449-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362684643, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894144, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.817614] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C144B-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362685996, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894145, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.819308] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C144D-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362687704, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894146, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.819683] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E4-0-0] [lt=16] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:14.819709] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E4-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061894177150004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061894177150004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:14.820651] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C144F-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362689047, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894147, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.824261] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1451-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362692657, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894148, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.826822] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E4-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894826541000, start_id=1729061894826541000, end_id=1729061894826541001) [2024-10-16 14:58:14.826894] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E4-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=11072, avg_group_log_size=651) [2024-10-16 14:58:14.826905] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E4-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=11072) [2024-10-16 14:58:14.827204] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=331, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:14.827217] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=11072) [2024-10-16 14:58:14.827227] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=75, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:14.827246] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=11072) [2024-10-16 14:58:14.827257] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=12, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:14.828531] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1453-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362696926, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894149, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.828651] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD368-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:14.828666] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD368-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=800, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:14.828671] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD368-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=437, avg_log_submit_to_slide_time=766) [2024-10-16 14:58:14.828776] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:14.829254] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=39] success to update min reserved snapshot(reserved_snapshot=1729060087261758000, duration=1800, snapshot_gc_ts=1729061887261758000) [2024-10-16 14:58:14.829840] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=12] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060087261758000) [2024-10-16 14:58:14.829856] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=13] update reserved snapshot finished(cost_ts=606, reserved_snapshot=1729060087261758000) [2024-10-16 14:58:14.832253] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1455-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362700649, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894150, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.833767] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1457-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362702162, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894151, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.833863] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22C4-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362702258, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894152, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.835414] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=17] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1001}, {id:1002}]) [2024-10-16 14:58:14.839051] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1459-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362707445, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894153, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.840713] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C145B-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362709109, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894154, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.841744] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C145C-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362710141, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894155, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.842098] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22C7-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362710492, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894156, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.843454] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C145E-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362711850, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894157, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.844941] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061894844316000, start_id=1729061894844316000, end_id=1729061894844316001) [2024-10-16 14:58:14.850497] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1460-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362718891, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894158, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.850609] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1729061894649579000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=201018, in_cluster_service=true, cluster_version={val:1729061894449967001, v:0}, min_cluster_version={val:1729061894449967001, v:0}, max_cluster_version={val:1729061894449967001, v:0}, get_cluster_version_err=0, cluster_version_delta=400630, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65249227, succ_cluster_heartbeat_count=414, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061502727285000, v:0}, local_cluster_delta=392123312, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:14.851427] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061894850653000, start_id=1729061894850653000, end_id=1729061894850653001) [2024-10-16 14:58:14.853444] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22C9-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362721838, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894159, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.854545] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22CA-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362722940, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894160, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.855671] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21888][T1001_LCLSender][T1001][Y0-0000000000000000-0-0] [lt=20] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0df6dfe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:14.858951] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1463-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362727346, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894161, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.862417] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1465-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362730811, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894162, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.863132] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6962, mem_used=14558208) [2024-10-16 14:58:14.866193] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1467-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362734587, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894163, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.869930] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C71-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:14.869953] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C71-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061894773792000, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061894773792000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:14.870726] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061894869704000, start_id=1729061894869704000, end_id=1729061894869704001) [2024-10-16 14:58:14.871937] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=59103530, this={dio_aligned_buf:{buf_write_offset:2468, buf_padding_size:1628, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:38335000, truncate_used_ts:14111071}, log_block_size:67108864, dir_fd:207, io_fd:202}, aligned_buf_len=4096, aligned_block_offset=59101184, buf_len=122, total_write_size_=12182, total_write_size_after_dio_=421888, ob_pwrite_used_ts=21460, count=100, each_pwrite_cost=214) [2024-10-16 14:58:14.873020] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C71-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061894872929000, start_id=1729061894872929000, end_id=1729061894872929001) [2024-10-16 14:58:14.875782] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1469-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362744177, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894164, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.878071] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=63] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:14.878686] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:14.878735] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015200372135000, v:0}, schema_version:1724725354537200, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061894447008000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}) [2024-10-16 14:58:14.878779] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=42] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061894447008000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1, freeze_info:{latest_snapshot_gc_scn:{val:1729061894447008000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726423200144977000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726509604287886000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726596003476641000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726682402401740000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726768801459802000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726855200486901000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1726941604690080001, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727028003883885000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727114402887183000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727200802155074000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727287201287535000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727373600479230000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727460004660296000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727546403853166000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727632803079527000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727719202031902000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727805601073019000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727892000093929000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1727978404300579000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728064803452924000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728151202651377000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728237602027199000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728324001159872000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728410400178022000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728496804317567000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728583203851180000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728669603349967000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728756002633289000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728842401978172000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728928800986272000, v:0}, schema_version:1724725354537200, data_version:17180066304}, {frozen_version:{val:1728982212394016000, v:0}, schema_version:1724725354537200, data_version:17180066304}]}}) [2024-10-16 14:58:14.879228] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C71-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894878904000, start_id=1729061894878904000, end_id=1729061894878904001) [2024-10-16 14:58:14.883911] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C146E-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362752305, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894165, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.885399] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1471-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362753794, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894166, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.886193] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1472-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362754589, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894167, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.886313] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=47962) [2024-10-16 14:58:14.886512] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=283, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:14.886531] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=56, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:14.886716] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:14.888108] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22CD-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362756503, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894168, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.888643] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B251ED0B1-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362757037, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894169, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.889809] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894888170000, start_id=1729061894888170000, end_id=1729061894888170001) [2024-10-16 14:58:14.891101] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1474-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362759497, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894170, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.891212] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.892786] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1476-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362761182, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894171, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.894860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061894894203000, start_id=1729061894894203000, end_id=1729061894894203001) [2024-10-16 14:58:14.898247] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1478-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362766641, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894172, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.899982] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C147A-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362768378, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894173, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.902776] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C147C-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362771172, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894174, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.904184] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C147E-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362772580, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894175, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.905549] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1480-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362773945, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894176, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.908543] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1482-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362776939, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894177, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.908629] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85231-0-0] [lt=6][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497757, log_task={header:{begin_lsn:{lsn:207589690158}, end_lsn:{lsn:207589690280}, log_id:218497757, min_scn:{val:1729061894678956001, v:0}, max_scn:{val:1729061894678956001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589690036}, prev_proposal_id:81, committed_end_lsn:{lsn:207589690158}, data_checksum:940009244, accum_checksum:4011554644, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061894808087, freeze_ts:1729061894808087, submit_ts:1729061894808091, flushed_ts:1729061894808592, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:501}, fs_cb_begin_ts=1729061894908623, log_life_time=100536) [2024-10-16 14:58:14.911856] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.913236] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1484-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362781631, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894178, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.916674] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1486-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362785062, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894179, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.923307] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1577222144, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:14.927702] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=7] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:14.928259] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=7] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:14.928921] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:14.928966] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=10] inner load succ(latest_freeze_info={frozen_version:{val:1729015204132784000, v:0}, schema_version:1725266205820024, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061891854891000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}) [2024-10-16 14:58:14.929005] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22696][T1004_FreInfoRe][T1004][Y0-0000000000000000-0-0] [lt=38] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061891854891000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1004, freeze_info:{latest_snapshot_gc_scn:{val:1729061891854891000, v:0}, frozen_statuses_.count():34, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726164002003574000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726250401737340000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726336802372947000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726423203190935000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726509602524510000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726596002569161000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726682402130915000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726768801773273000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726855201165331000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1726941600469262000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727028000198901000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727114404740980000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727200804508783000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727287204796965000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727373604166928000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727460003454640000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727546402572495000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727632803254701000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727719203477605000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727805602784730000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727892002136447000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1727978401755018000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728064803292612000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728151203366588000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728237602699393000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728324002428569000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728410401795658000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728496801803461000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728583202226013000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728756001187197000, v:0}, schema_version:1725266205820024, data_version:17180066304}, {frozen_version:{val:1728842400401001000, v:0}, schema_version:1725266205820024, data_version:17180066304}]}}) [2024-10-16 14:58:14.929313] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:14.929346] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=6] ====== tenant freeze timer task ====== [2024-10-16 14:58:14.929357] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27208900, tenant_id=1004) [2024-10-16 14:58:14.929366] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:14.930970] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1488-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362799364, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894180, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.930992] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22CE-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362799387, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894181, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.935051] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234D-0-0] [lt=28] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:58:14.935757] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234D-0-0] [lt=16] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1004) [2024-10-16 14:58:14.935771] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234D-0-0] [lt=12] no transfer partition job [2024-10-16 14:58:14.935777] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234D-0-0] [lt=4] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:58:14.937054] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234D-0-0] [lt=6] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1004, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:58:14.937075] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234D-0-0] [lt=16] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:58:14.938310] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22635][T1004_TBalance][T1004][YB4A0A000407-00061E9AF87C234D-0-0] [lt=20] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1002, status:0}], ls_array_=[{tenant_id:1004, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1004, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1002, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729059563578310, last_statistic_bg_stat_time=1725269669481368, last_statistic_schema_version=1725266205820024, last_statistic_max_transfer_task_id={id:5}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:58:14.940346] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C148A-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362808739, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894182, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.941566] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22142][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=27] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1761, free_obj_hold_cnt:76, allocator used:7068776, allocator total:7521920}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:389, free_obj_hold_cnt:11, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():11904, full allocator used:1600, full allocator total:77440}, tablets_mem=33835760, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:79, free_obj_hold_cnt:126, allocator used:419840, allocator total:457856}, pending_cnt=0, wait_gc_count=0, tablet count=2150) [2024-10-16 14:58:14.942029] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=13][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:14.942084] WDIAG read_multi_row (ob_table_access_helper.h:238) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=15][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1001, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:14.942869] INFO delete_row (ob_table_access_helper.h:312) [21883][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=9] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1001, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061894942094) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061894942094) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:58:14.944919] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C148C-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362813313, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894183, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.946283] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C148E-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362814679, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894184, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.949626] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1490-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362818022, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894185, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.949891] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF7-0-0] [lt=20] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061894917580, keepalive_ts:-1}) [2024-10-16 14:58:14.949906] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF7-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061894917580, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:14.950324] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5018, mem_used=10398720) [2024-10-16 14:58:14.951141] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1492-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362819535, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894186, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.951331] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:14.951342] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:14.952226] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=35, this=0x2b0e5af6c280, extra_info="") [2024-10-16 14:58:14.952242] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=15] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=2607, this=0x2b0e5af6c4b0, extra_info="") [2024-10-16 14:58:14.952245] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5af6c6e0, extra_info="") [2024-10-16 14:58:14.952249] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=-677, this=0x2b0e5af6c910, extra_info="") [2024-10-16 14:58:14.952252] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=1965, this=0x2b0e5af6cb40, extra_info="") [2024-10-16 14:58:14.953704] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=16] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:14.957631] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1495-0-0] [lt=29][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362826025, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894187, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.958549] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1496-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362826945, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894188, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.959857] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1498-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362828253, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894189, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.966092] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C149A-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362834487, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894190, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.967430] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C149C-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362835826, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894191, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.971448] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C149E-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362839844, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894192, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.971810] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54068-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:14.971829] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54068-0-0] [lt=12] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:14.972018] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=17] worker thread began to shrink(tenant_->id()=1, group_id_=19, token=0) [2024-10-16 14:58:14.972110] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=236, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:14.972130] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=122, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:14.972166] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:14.972503] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=15] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:14.972515] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=12] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:14.975187] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14A0-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362843583, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894193, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.976647] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14A3-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362845042, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894194, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.977459] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14A4-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362845855, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894195, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.978012] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22D0-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362846407, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894196, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.978280] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:419) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:14.978335] WDIAG read_multi_row (ob_table_access_helper.h:238) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2024-10-16 14:58:14.979112] INFO delete_row (ob_table_access_helper.h:312) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1004, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1729061894978342) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1729061894978342) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2024-10-16 14:58:14.979169] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=11] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2024-10-16 14:58:14.979228] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=6] finish check and clear obj lock(ls->get_ls_id()={id:1002}) [2024-10-16 14:58:14.979277] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:221) [22782][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=3] finish check and clear obj lock(ls->get_ls_id()={id:1003}) [2024-10-16 14:58:14.982020] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [106955][T1_L0_G19][T1][Y0-0000000000000000-0-0] [lt=22] worker thread exit(tenant_->id()=1, workers_.get_size()=1) [2024-10-16 14:58:14.982036] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [106955][T1_L0_G19][T0][Y0-0000000000000000-0-0] [lt=12] tenant thread end_run(id_=1, ret=0, thread_count_=239) [2024-10-16 14:58:14.983285] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14A6-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362851679, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894197, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.985127] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14A7-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362853523, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894198, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.987641] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14A9-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362856036, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894199, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.987863] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79F3-0-0] [lt=7][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938612, log_task={header:{begin_lsn:{lsn:213058388203}, end_lsn:{lsn:213058388325}, log_id:220938612, min_scn:{val:1729061894773792001, v:0}, max_scn:{val:1729061894773792001, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058388081}, prev_proposal_id:88, committed_end_lsn:{lsn:213058388203}, data_checksum:1715074492, accum_checksum:427515052, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061894885862, freeze_ts:1729061894885862, submit_ts:1729061894885868, flushed_ts:1729061894886727, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:859}, fs_cb_begin_ts=1729061894987856, log_life_time=101994) [2024-10-16 14:58:14.988260] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE201B-0-0] [lt=17] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1001, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:14.988278] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107002][T1001_L0_G19][T1001][YB4A0A000408-00061E9B44AE201B-0-0] [lt=14] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:14.989691] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14AB-0-0] [lt=29][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362858086, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894200, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.991275] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:14.991411] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107610][][T1][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1, ret=0, thread_count_=240) [2024-10-16 14:58:14.991435] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107610][][T1][Y0-0000000000000000-0-0] [lt=22] Init thread local success [2024-10-16 14:58:14.991508] INFO [SERVER.OMT] recv_group_request (ob_tenant.cpp:1419) [21189][pnio1][T0][YB4A0A000408-00061E9B44AE201C-0-0] [lt=10] worker thread created(id()=1, group->group_id_=19) [2024-10-16 14:58:14.993529] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=12] ====== tenant freeze timer task ====== [2024-10-16 14:58:14.996013] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93BF-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:14.996034] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93BF-0-0] [lt=21] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11196, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:14.996093] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14AD-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362864486, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894201, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:14.997711] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14AF-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362866106, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894202, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.000678] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14B1-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362869073, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894203, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.003932] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14B3-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362872327, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894204, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.008790] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14B5-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362877184, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894205, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.010414] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14B7-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362878810, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894206, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.015063] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.015383] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=56, replace_time=0, replace_start_pos=62914, replace_num=31457) [2024-10-16 14:58:15.015397] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=56, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:15.015755] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D59-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061895005552, keepalive_ts:-1}) [2024-10-16 14:58:15.015775] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D59-0-0] [lt=20] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061895005552, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:15.017835] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21897][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=10] ====== traversal_flush timer task ====== [2024-10-16 14:58:15.017956] INFO [MDS] flush (mds_table_mgr.cpp:195) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DB2-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1729061701145953001, v:0}}, }, max_consequent_callbacked_scn={val:1729061894894203000, v:0}, *this={this:0x2b0e076c4510, is_inited:true, freezing_scn:{val:1729035463169344005, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061905590307000, latest_log_ts:{val:1729061889039053001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:15.018036] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21897][T1001_Flush][T1001][YB4A0A000407-00061E9B37498DB2-0-0] [lt=74] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:15.024758] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14B9-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362893152, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894207, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.026978] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14BB-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362895373, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894208, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.033374] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14BD-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362901768, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894209, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.034843] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14BF-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362903238, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894210, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.036453] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14C1-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362904849, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894211, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.044378] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14C3-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362912772, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894212, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.045251] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061895044743000, start_id=1729061895044743000, end_id=1729061895044743001) [2024-10-16 14:58:15.047957] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ====== [2024-10-16 14:58:15.047975] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:15.047988] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=292060417, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:58:15.050280] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14C5-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362918675, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894213, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.051440] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061895051202000, start_id=1729061895051202000, end_id=1729061895051202001) [2024-10-16 14:58:15.051633] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO [ratelimit] time: 1729061895051632, bytes: 4446828326373, bw: 0.585611 MB/s, add_ts: 1002545, add_bytes: 615620 [2024-10-16 14:58:15.052074] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40253-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:15.052086] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40253-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1715, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:15.052314] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] ====== tenant freeze timer task ====== [2024-10-16 14:58:15.052325] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=150945360, tenant_id=1) [2024-10-16 14:58:15.052337] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:15.056490] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14C7-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362924885, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894214, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.058537] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14C9-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362926932, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894215, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.059995] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14CB-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362928391, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894216, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.063152] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14CD-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362931548, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894217, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.066551] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21599][T1_ArcSender][T1][YB4A0A000407-00061E996C5960C1-0-0] [lt=15][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:15.068947] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14CF-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362937341, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894218, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.070853] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061895070076000, start_id=1729061895070076000, end_id=1729061895070076001) [2024-10-16 14:58:15.072061] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:15.072472] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93C0-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:15.072485] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93C0-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40302, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:15.076647] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061895076457000, start_id=1729061895076457000, end_id=1729061895076457001) [2024-10-16 14:58:15.076681] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:15.076699] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=538, get_gts_with_stc_cnt=79, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=76, try_wait_gts_elapse_cnt=0) [2024-10-16 14:58:15.076708] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1729061895076696], gts:1729061895076457000, latest_srr:[mts=1729061895076696]}) [2024-10-16 14:58:15.078487] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14D1-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362946880, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894219, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.081890] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14D4-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362950282, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894220, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.083189] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14D5-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362951583, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894221, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.084234] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4895-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061895049522, keepalive_ts:-1}) [2024-10-16 14:58:15.084249] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E99FD4C4895-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061895049522, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:15.084951] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22D2-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362953345, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894222, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.086300] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14D8-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362954695, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894223, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.086651] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79F5-0-0] [lt=24] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=53936) [2024-10-16 14:58:15.086976] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46E5-0-0] [lt=11] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=38, avg_log_submit_to_flush_time=405, avg_log_submit_to_slide_time=823) [2024-10-16 14:58:15.088415] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14D9-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362956811, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894224, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.089644] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22D4-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362958039, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894225, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.089789] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14DC-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362958186, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894226, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.090702] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061895089841000, start_id=1729061895089841000, end_id=1729061895089841001) [2024-10-16 14:58:15.090764] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:15.090778] INFO [LIB] stat (utility.h:1165) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:15.091334] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.092453] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22D6-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362960848, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894227, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.093594] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14DE-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362961989, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894228, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.094550] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14DF-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362962946, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894229, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.095098] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061895094635000, start_id=1729061895094635000, end_id=1729061895094635001) [2024-10-16 14:58:15.095716] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22228][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=20] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e0bffe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:15.095788] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2024-10-16 14:58:15.096548] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1416) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] succ to get rec scn(*this={this:0x2b0dee604030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1722494261657150001, v:0}, max_applied_start_working_ts_:{val:1722494261657150001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:1722494667284931000, v:0}, last_push_gc_task_ts_:1722495943569310, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1699}, aggre_rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:15.096571] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=23] tx ctx memtable get rec scn(this={ObITable:{this:0x2b0e7203a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1729015200379232, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x2b0e7203a080, snapshot_version:{val:1729015200379232, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2024-10-16 14:58:15.096593] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:845) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:1717662766736418000, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2024-10-16 14:58:15.096700] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:260) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=17] [FREEZE]no valid mds table there, return MAX SCN(ret=0, ret="OB_SUCCESS", min_rec_scn={val:4611686018427387903, v:0}, order_flusher_for_one={high_priority_flusher:size:0, ordered_tablets:{}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061907008855000, latest_log_ts:{val:1729061890536200001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:15.096744] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:629) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=41] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x2b0dee9a5890}, min_rec_scn={val:1729015200503866000, v:0}, ls_id_={id:1}) [2024-10-16 14:58:15.096754] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:470) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1722103204703392000, v:0}) [2024-10-16 14:58:15.096760] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=0, rec_log_ts={val:1729061890536200001, v:0}) [2024-10-16 14:58:15.096764] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:15.096767] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903, v:0}) [2024-10-16 14:58:15.096774] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:187) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1729015200503866000, v:0}, checkpoint_scn_in_ls_meta={val:1729015200503866000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2024-10-16 14:58:15.096783] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1595) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, curr_base_lsn={lsn:66098196480}, new_base_lsn={lsn:66098196480}, lsn={lsn:66098196480}) [2024-10-16 14:58:15.096800] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:188) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:66098196480}, ls->get_ls_id()={id:1}) [2024-10-16 14:58:15.096805] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:195) [21591][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2024-10-16 14:58:15.096960] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40255-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=9, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=372, avg_log_submit_to_slide_time=12511) [2024-10-16 14:58:15.097104] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:15.100081] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14E1-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362968476, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894230, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.101441] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14E3-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362969837, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894231, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.101550] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22D9-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362969945, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894232, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.102942] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14E5-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362971338, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894233, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.106089] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14E7-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362974484, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894234, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.107406] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14E9-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362975788, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894235, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.109051] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14EB-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362977447, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894236, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.110407] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14ED-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362978803, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894237, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.114422] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D5A-0-0] [lt=13] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1001, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061895105618, keepalive_ts:-1}) [2024-10-16 14:58:15.114442] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7B9C7D5A-0-0] [lt=19] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1001, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061895105618, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:15.115097] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.116717] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14EF-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362985111, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894238, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.118985] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14F1-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362987380, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894239, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.120728] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14F3-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362989124, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894240, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.123500] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14F5-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362991896, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894241, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.123574] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1577222144, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:15.124922] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14F7-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362993318, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894242, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.126269] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14F9-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362994665, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894243, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.128923] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:15.128938] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:15.130973] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14FB-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510362999367, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894244, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.132401] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14FD-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363000797, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894245, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.133734] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C14FF-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363002130, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894246, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.135283] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1501-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363003679, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894247, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.136773] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1503-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363005169, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894248, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.138303] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1505-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363006699, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894249, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.138992] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5510, mem_used=10398720) [2024-10-16 14:58:15.139863] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1507-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363008259, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894250, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.144940] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1509-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363013335, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894251, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.150558] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:15.151436] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=2060, avg_group_log_size=206) [2024-10-16 14:58:15.151452] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=2066) [2024-10-16 14:58:15.151987] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C150B-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363020381, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894252, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.152352] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=2060) [2024-10-16 14:58:15.155089] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C150D-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363023484, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894253, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.158360] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C150F-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363026755, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894254, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.161336] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1511-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363029732, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894255, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.161751] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8E-0-0] [lt=14] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:15.161775] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8E-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061894805186001, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061894805186001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:15.162913] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1513-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363031309, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894256, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.162951] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=10] current monitor number(seq_=-1) [2024-10-16 14:58:15.165130] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B732E8D8E-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061895161809000, start_id=1729061895161809000, end_id=1729061895161809001) [2024-10-16 14:58:15.166676] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1515-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363035071, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894257, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.170508] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D8E-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061895169685000, start_id=1729061895169685000, end_id=1729061895169685001) [2024-10-16 14:58:15.171839] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5406C-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:15.172231] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0DA6-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061895171869000, start_id=1729061895171869000, end_id=1729061895171869001) [2024-10-16 14:58:15.172300] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0DA8-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061895171869000, start_id=1729061895171869000, end_id=1729061895171869001) [2024-10-16 14:58:15.172323] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0DA5-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061895171869000, start_id=1729061895171869000, end_id=1729061895171869001) [2024-10-16 14:58:15.172483] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0DA4-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061895171869000, start_id=1729061895171869000, end_id=1729061895171869001) [2024-10-16 14:58:15.174078] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1518-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363042472, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894258, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.174113] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22DA-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363042507, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894259, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.182180] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C151A-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363050572, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894260, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.185120] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C151C-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363053514, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894261, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.186673] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C151E-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363055068, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894262, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.188229] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79F7-0-0] [lt=10][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=220938614, log_task={header:{begin_lsn:{lsn:213058388447}, end_lsn:{lsn:213058388569}, log_id:220938614, min_scn:{val:1729061894773792003, v:0}, max_scn:{val:1729061894773792003, v:0}, data_len:66, proposal_id:88, prev_lsn:{lsn:213058388325}, prev_proposal_id:88, committed_end_lsn:{lsn:213058388447}, data_checksum:1715074492, accum_checksum:381270462, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061895086642, freeze_ts:1729061895086643, submit_ts:1729061895086646, flushed_ts:1729061895087160, gen_to_freeze cost time:1, gen_to_submit cost time:4, submit_to_flush cost time:514}, fs_cb_begin_ts=1729061895188217, log_life_time=101575) [2024-10-16 14:58:15.188549] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1520-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363056945, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894263, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.190236] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1523-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363058631, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894264, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.190452] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22DB-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363058845, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894265, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.190498] INFO [SERVER] reload_tenant_task (ob_tenant_tablet_ttl_mgr.cpp:1296) [31906][T1002_TTLTablet][T1002][YB4A0A000407-00061E9AB20CC6DE-0-0] [lt=23] finish reload tenant task(local_tenant_task_={tenant_id:1002, task_id:-1, is_usr_trigger:false, need_check:false, is_dirty:false, ttl_continue:true, rsp_time:-1, state:4, is_reused:true}, tenant_task={gmt_create:0, gmt_modified:0, tenant_id:18446744073709551615, table_id:18446744073709551615, tablet_id:18446744073709551615, task_id:0, task_start_time:0, task_update_time:0, trigger_type:0, status:19, ttl_del_cnt:0, max_version_del_cnt:0, scan_cnt:0, row_key:"", ret_code:"OB_SUCCESS"}, is_leader=false) [2024-10-16 14:58:15.191396] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.194355] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get_max_applied_scn(scn={val:1729061895094635000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33705, palf_committed_end_lsn_:{lsn:114618855911}, palf_committed_end_scn_:{val:1729061895094635000, v:0}, last_check_scn_:{val:1729061895094635000, v:0}, max_applied_cb_scn_:{val:1729061895094635000, v:0}}) [2024-10-16 14:58:15.194375] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get_min_unreplayed_log_info(lsn={lsn:45686897415}, scn={val:1722494261473967003, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:58:15.199501] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1525-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363067895, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894266, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.200850] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1527-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363069246, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894267, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.208124] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1529-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363076518, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894268, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.213326] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504D-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:15.213344] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504D-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:15.214635] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C152B-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363083029, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894269, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.215172] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.216911] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=51, replace_time=0, replace_start_pos=94371, replace_num=31457) [2024-10-16 14:58:15.216928] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=51, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:15.218172] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C152D-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363086567, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894270, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.219638] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C152F-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363088034, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894271, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.224545] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1533-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363092940, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894272, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.230037] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1535-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363098431, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894273, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.231686] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1537-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363100081, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894274, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.234212] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1539-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363102607, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894275, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.235634] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C153B-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363104029, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894276, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.237833] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C153D-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363106228, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894277, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.241294] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C153F-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363109689, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894278, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.244895] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1541-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363113291, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894279, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.245483] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061895245058000, start_id=1729061895245058000, end_id=1729061895245058001) [2024-10-16 14:58:15.245565] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=25, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:15.245595] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:15.247424] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1543-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363115819, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894280, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.250409] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1545-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363118805, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894281, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.251668] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=24, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:15.251674] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061895250489000, start_id=1729061895250489000, end_id=1729061895250489001) [2024-10-16 14:58:15.251687] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:15.254336] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1547-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363122731, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894282, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.255904] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1549-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363124299, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894283, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.264565] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C154B-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363132958, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894284, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.268575] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C154D-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363136968, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894285, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.270873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061895270065000, start_id=1729061895270065000, end_id=1729061895270065001) [2024-10-16 14:58:15.272504] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C154F-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363140898, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894286, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.273676] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107614][][T1003][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1003, ret=0, thread_count_=173) [2024-10-16 14:58:15.273700] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107614][][T1003][Y0-0000000000000000-0-0] [lt=22] Init thread local success [2024-10-16 14:58:15.273718] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=11] worker thread created(tenant_->id()=1003, group_id_=1, token=1) [2024-10-16 14:58:15.286395] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:15.286416] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:15.290360] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=4] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1729061895089841000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=200511, in_cluster_service=true, cluster_version={val:1729061894869704000, v:0}, min_cluster_version={val:1729061894869704000, v:0}, max_cluster_version={val:1729061894869704000, v:0}, get_cluster_version_err=0, cluster_version_delta=420648, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244545, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061390930197000, v:0}, local_cluster_delta=504360155, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:15.291331] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=54] get number(ret=0, service_type_=0, range=1, base_id=1729061895290125000, start_id=1729061895290125000, end_id=1729061895290125001) [2024-10-16 14:58:15.291458] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.295226] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=46] get number(ret=0, service_type_=0, range=1, base_id=1729061895294871000, start_id=1729061895294871000, end_id=1729061895294871001) [2024-10-16 14:58:15.295708] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1089) [2024-10-16 14:58:15.298160] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 171.00; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 16173.46, iops: 213.46, rt: 153.04; ) [2024-10-16 14:58:15.298183] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:15.298218] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 5120.00, iops: 3.99, rt: 162.75; ) [2024-10-16 14:58:15.298227] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:15.298238] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:15.304940] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=47] tx gc loop thread is running(MTL_ID()=1001) [2024-10-16 14:58:15.304957] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=17] try gc retain ctx [2024-10-16 14:58:15.304961] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:15.305570] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21912][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1001, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729056578237664001, v:0}, clog_base_lsn:{lsn:114547838976}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061110891994000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061905590307000, latest_log_ts:{val:1729061889039053001, v:0}}, {limited_id:195000001, latest_log_ts:{val:1729056578237664001, v:0}}, {limited_id:3540000001, latest_log_ts:{val:1729033513943668001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33705, palf_env_:0x2b0df7bb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:15.308674] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85239-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:15.315247] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.316691] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=11] PNIO [ratelimit] time: 1729061895316689, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007377, add_bytes: 0 [2024-10-16 14:58:15.323866] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1577222144, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:15.331219] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=1, size_used=1793, mem_used=4159488) [2024-10-16 14:58:15.331556] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E0E-0-0] [lt=11][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:58:15.334591] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:519) [21528][T1_HBService][T1][YB4A0A000407-00061E9B1DCD1047-0-0] [lt=12] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2024-10-16 14:58:15.337138] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=14] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:15.344469] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:15.352754] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE4025C-0-0] [lt=18] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=2060) [2024-10-16 14:58:15.353532] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B251ED0B3-0-0] [lt=10][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:58:15.353651] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=374, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:15.353668] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=349, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:15.353680] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=333, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:15.353686] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1210) [2024-10-16 14:58:15.353693] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=266, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:15.353722] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:15.353727] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1122, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:15.353731] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:15.353750] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=620, avg_log_submit_to_slide_time=1132) [2024-10-16 14:58:15.366034] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:170) [22248][T1002_MediumLoo][T1002][YB4A0A000407-00061E9973CF3859-0-0] [lt=12] MediumCheckTask(cost_ts=8) [2024-10-16 14:58:15.373042] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54070-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=366, avg_log_submit_to_slide_time=40293) [2024-10-16 14:58:15.373057] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54070-0-0] [lt=15][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173487, log_task={header:{begin_lsn:{lsn:14352415506}, end_lsn:{lsn:14352415628}, log_id:115173487, min_scn:{val:1729061895178386000, v:0}, max_scn:{val:1729061895178386000, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352415384}, prev_proposal_id:17, committed_end_lsn:{lsn:14352415506}, data_checksum:940009244, accum_checksum:1612095060, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061895272951, freeze_ts:1729061895272951, submit_ts:1729061895272955, flushed_ts:1729061895273417, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:462}, fs_cb_begin_ts=1729061895373035, log_life_time=100084) [2024-10-16 14:58:15.381381] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABE-0-0] [lt=17] [RS_MGR] force set rs list(master_rs="10.0.4.7:2890") [2024-10-16 14:58:15.381397] INFO [SERVER] handle_heartbeat (ob_service.cpp:3258) [21658][T1_L0_G0][T1][YB4A0A000407-00061E9B1CDD1ABE-0-0] [lt=13] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"10.0.4.7:2890", server_id:1, rs_addr:"10.0.4.7:2890", rs_server_status:1, epoch_id:48}, hb_response={zone:"myoceanbase1", server:"10.0.4.7:2890", sql_port:2889, build_version:"4.3.2.0_100000092024072912-7f239820d2067e8d0983ce6d0f13b5e0491571e3(Jul 29 2024 22:56:28)", start_service_time:1722494256558000, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=17) [2024-10-16 14:58:15.381407] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:15.381427] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21595][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=221) [2024-10-16 14:58:15.386783] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=47962) [2024-10-16 14:58:15.387444] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:15.391520] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.394460] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1001, server_version={version:{val:1729061895094635000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=299818, in_cluster_service=true, cluster_version={val:1729061895044743000, v:0}, min_cluster_version={val:1729061895044743000, v:0}, max_cluster_version={val:1729061895044743000, v:0}, get_cluster_version_err=0, cluster_version_delta=349710, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244790, succ_cluster_heartbeat_count=398, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061625603829000, v:0}, local_cluster_delta=269790624, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:15.396200] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=13] gc stale ls task succ [2024-10-16 14:58:15.397180] INFO pn_ratelimit (group.c:617) [21320][IngressService][T0][Y0-0000000000000000-0-0] [lt=11] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2024-10-16 14:58:15.399494] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [21272][Blacklist][T0][Y0-0000000000000000-0-0] [lt=16] blacklist_loop exec finished(cost_time=15, is_enabled=true, send_cnt=0) [2024-10-16 14:58:15.409097] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8523B-0-0] [lt=13][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497762, log_task={header:{begin_lsn:{lsn:207589690768}, end_lsn:{lsn:207589690890}, log_id:218497762, min_scn:{val:1729061895178386001, v:0}, max_scn:{val:1729061895178386001, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589690646}, prev_proposal_id:81, committed_end_lsn:{lsn:207589690768}, data_checksum:940009244, accum_checksum:885764042, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061895308669, freeze_ts:1729061895308669, submit_ts:1729061895308688, flushed_ts:1729061895308992, gen_to_freeze cost time:0, gen_to_submit cost time:19, submit_to_flush cost time:304}, fs_cb_begin_ts=1729061895409087, log_life_time=100418) [2024-10-16 14:58:15.414865] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=24, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:15.414885] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=6400, avg_group_log_size=457) [2024-10-16 14:58:15.414895] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:15.415156] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=19] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=382, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:15.415174] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=6400) [2024-10-16 14:58:15.415186] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=150, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:15.415211] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=6400) [2024-10-16 14:58:15.415222] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=11, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:15.415311] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.416197] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C118B-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:15.416208] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C118B-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=1392, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:15.416212] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B804C118B-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=10, avg_log_gen_to_submit_time=41, avg_log_submit_to_flush_time=598, avg_log_submit_to_slide_time=1350) [2024-10-16 14:58:15.418437] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=42, replace_time=0, replace_start_pos=125828, replace_num=31457) [2024-10-16 14:58:15.418454] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=42, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:15.433981] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA8A1-0-0] [lt=1] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:15.434018] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [123287][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA8A1-0-0] [lt=26] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061895178386001, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061895178386001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:15.435155] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA8A1-0-0] [lt=49] get number(ret=0, service_type_=0, range=1, base_id=1729061895434555000, start_id=1729061895434555000, end_id=1729061895434555001) [2024-10-16 14:58:15.438923] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA8A1-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061895437927000, start_id=1729061895437927000, end_id=1729061895437927001) [2024-10-16 14:58:15.439008] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA8A1-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=9926, avg_group_log_size=522) [2024-10-16 14:58:15.439021] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA8A1-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9928) [2024-10-16 14:58:15.439606] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E9A398D5A65-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9928) [2024-10-16 14:58:15.440181] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=422, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:15.440192] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9926) [2024-10-16 14:58:15.440203] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=105, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:15.440226] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=6, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:15.440231] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=907, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:15.440235] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=572, avg_log_submit_to_slide_time=872) [2024-10-16 14:58:15.440242] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=8, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:15.440285] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=8] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:15.446425] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061895445368000, start_id=1729061895445368000, end_id=1729061895445368001) [2024-10-16 14:58:15.451648] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21590][T1_TenantWeakRe][T1][YB4A0A000407-00061E9B3B3AD207-0-0] [lt=15] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.731511650085449219e+02, this=0x2b0ddc2e3208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=1023, Resource Limit(MB)=865, Throttle Trigger(MB)=519, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.167297611430484494e+01) [2024-10-16 14:58:15.451789] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061895450616000, start_id=1729061895450616000, end_id=1729061895450616001) [2024-10-16 14:58:15.456539] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=8] replay status is not follower(min_unreplayed_lsn={lsn:7668488755}, base_lsn={lsn:7582838784}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204648991, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31972352, ret_code_:0}}, next_to_submit_lsn_:{lsn:7668488755}, next_to_submit_scn_:{val:1722494261263129005, v:0}, base_lsn_:{lsn:7582838784}, base_scn_:{val:1722448818782043002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e98005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7668488633}, end_lsn:{lsn:7668488755}, read_buf:{buf_len_:3747840, buf_:0x2b0e98005000}, block_size:67104768, log_storage_:0x2b0e431dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129004, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129004, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:966528749, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129004, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:82, miss_cnt:24, cache_read_size:155016, hit_ratio:"7.735849056603774088e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:24, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:45, read_io_size:86192128, read_disk_cost_ts:1042320, start_lsn:{lsn:7582838784}}}}}}) [2024-10-16 14:58:15.456591] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=52] get_replay_process success(id={id:1}, replayed_log_size=85649971, unreplayed_log_size=0) [2024-10-16 14:58:15.456606] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=5] replay status is not follower(min_unreplayed_lsn={lsn:6198988121}, base_lsn={lsn:6106533888}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015204649023, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:31151481, ret_code_:0}}, next_to_submit_lsn_:{lsn:6198988121}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:6106533888}, base_scn_:{val:1722448820367858002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e9f005000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:6198987999}, end_lsn:{lsn:6198988121}, read_buf:{buf_len_:3747840, buf_:0x2b0e9f005000}, block_size:67104768, log_storage_:0x2b0e431f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:3860329158, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:77, miss_cnt:26, cache_read_size:10230, hit_ratio:"7.475728155339805836e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:26, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:48, read_io_size:93233152, read_disk_cost_ts:1481530, start_lsn:{lsn:6106533888}}}}}}) [2024-10-16 14:58:15.456629] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=22] get_replay_process success(id={id:1001}, replayed_log_size=92454233, unreplayed_log_size=0) [2024-10-16 14:58:15.456636] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1002}, replayed_log_size=8163576423, unreplayed_log_size=0) [2024-10-16 14:58:15.456642] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] get_replay_process success(id={id:1003}, replayed_log_size=8111672204, unreplayed_log_size=0) [2024-10-16 14:58:15.456657] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [22713][T1004_ReplayPro][T1004][Y0-0000000000000000-0-0] [lt=3] dump tenant replay process(tenant_id=1004, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=15691, last_replayed_log_size(MB)=15691, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:58:15.472980] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061895472257000, start_id=1729061895472257000, end_id=1729061895472257001) [2024-10-16 14:58:15.473637] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107614][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD371-0-0] [lt=5] init point thread id with(&point=0x55e3435cc600, idx_=1118, point=[thread id=107614, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107614) [2024-10-16 14:58:15.475579] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [29808][T1002_L0_G0][T1002][YB4A0A000407-00061E9B2ECA2241-0-0] [lt=12][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:58:15.486479] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1646, avg_group_log_size=164) [2024-10-16 14:58:15.486499] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1646) [2024-10-16 14:58:15.486851] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=30] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=276, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:15.486869] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=87, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:15.486890] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:15.487175] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46F0-0-0] [lt=25] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:15.487182] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46F0-0-0] [lt=7] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=784, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:15.490502] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] no need to update min/max version(new_version={val:1729061895089841000, v:0}, min_version={val:1729061895089841000, v:0}, max_version={val:1729061895089841000, v:0}) [2024-10-16 14:58:15.490521] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33576, start_service_tstamp_=1722494263708085, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:15.491583] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.493836] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:15.493861] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=451) [2024-10-16 14:58:15.493884] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21898][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=10] cannot_recycle_log_size statistics(cannot_recycle_log_size=71019821, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:15.495483] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1729061895494083000, start_id=1729061895494083000, end_id=1729061895494083001) [2024-10-16 14:58:15.496286] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EEF-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=9, avg_log_batch_cnt=0, total_group_log_size=1089, avg_group_log_size=121) [2024-10-16 14:58:15.496564] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=328, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:15.496580] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=122, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:15.496616] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:15.500931] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2AA1-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:15.500950] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2AA1-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:15.508281] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8523D-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:15.508384] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=53099, hit_cnt=13, read_cnt=13, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:15.509463] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1080) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=9] replay status is not follower(min_unreplayed_lsn={lsn:45686897415}, base_lsn={lsn:45497032704}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729059828924793, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:11807285, ret_code_:0}}, next_to_submit_lsn_:{lsn:45686897415}, next_to_submit_scn_:{val:1722494261473967003, v:0}, base_lsn_:{lsn:45497032704}, base_scn_:{val:1722481627256610001, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5b605000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:45686897293}, end_lsn:{lsn:45686897415}, read_buf:{buf_len_:3747840, buf_:0x2b0e5b605000}, block_size:67104768, log_storage_:0x2b0df99e9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261473967002, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261473967002, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:2856452666, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261473967002, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:480, miss_cnt:53, cache_read_size:488027, hit_ratio:"9.005628517823639489e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:53, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:101, read_io_size:191037440, read_disk_cost_ts:1811383, start_lsn:{lsn:45497032704}}}}}}) [2024-10-16 14:58:15.509520] INFO [CLOG] operator() (ob_log_replay_service.cpp:1646) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=57] get_replay_process success(id={id:1}, replayed_log_size=189864711, unreplayed_log_size=0) [2024-10-16 14:58:15.509539] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:159) [21820][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1001, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=181, last_replayed_log_size(MB)=181, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2024-10-16 14:58:15.509794] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:446) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:15.509804] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10] try reload schema success(schema_version=1724725354537200, refresh_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_ret=0) [2024-10-16 14:58:15.509814] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1005, ret=-4201) [2024-10-16 14:58:15.509812] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:262) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED8-0-0] [lt=6] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:15.509819] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] fail got version(tenant_id=1005, version=2, ret=-4201) [2024-10-16 14:58:15.509823] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4201] No tenant config found(tenant_id=1006, ret=-4201) [2024-10-16 14:58:15.509825] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1006, version=1724648414518006, ret=-4201) [2024-10-16 14:58:15.509824] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:287) [21205][SerScheQueue1][T0][YB4A0A000407-00061E9AE6FE9ED8-0-0] [lt=10] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}, schema_info={schema_version:1729008953019688, tenant_id:1002, sequence_id:8796093024705}) [2024-10-16 14:58:15.509828] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1007, ret=-4201) [2024-10-16 14:58:15.509831] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1007, version=2, ret=-4201) [2024-10-16 14:58:15.509833] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:619) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] No tenant config found(tenant_id=1008, ret=-4201) [2024-10-16 14:58:15.509835] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:607) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=2][errcode=-4201] fail got version(tenant_id=1008, version=1724665784261916, ret=-4201) [2024-10-16 14:58:15.509839] WDIAG [SERVER] do_heartbeat_event (ob_heartbeat.cpp:288) [24944][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:58:15.515373] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.523644] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=41] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1001, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:15.524161] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1577222144, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:15.524309] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=18] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:15.524361] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [21804][T1001_FreInfoRe][T1001][Y0-0000000000000000-0-0] [lt=11] inner load succ(latest_freeze_info={frozen_version:{val:1729015203681882000, v:0}, schema_version:1722494878780592, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061888741781000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801956053001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726423200971538000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726509604883844000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726596003978580000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726682402904319000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726768801830194001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726855200826598000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1726941604709294000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727028003626187000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727114402511050000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727200801281951000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727287200169112000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727373604057431000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727460003037870000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727546401768784000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727632800508408000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727719204490148000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727805603412694000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727892002224540001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1727978401079652000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728064804985207000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728151203942286001, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728237602953147000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728324001805777000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728410400561219000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728496804431110000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728583203461341000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728669602827103000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728756001940150000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728842400715649000, v:0}, schema_version:1722494878780592, data_version:17180066304}, {frozen_version:{val:1728928804726674000, v:0}, schema_version:1722494878780592, data_version:17180066304}]}) [2024-10-16 14:58:15.535318] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C15D0-0-0] [lt=1] get number(ret=0, service_type_=0, range=1, base_id=1729061895534567000, start_id=1729061895534567000, end_id=1729061895534567001) [2024-10-16 14:58:15.539254] INFO [SHARE] get_server_tenant_configs (ob_unit_getter.cpp:231) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=5] get_server_tenant_configs(unit_infos=[{unit:unit_id:1, resource_pool_id:1, unit_group_id:1, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, pool:{resource_pool_id:1, name:"sys_pool", unit_count:1, unit_config_id:1, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1, replica_type:0}}, {unit:unit_id:1001, resource_pool_id:1001, unit_group_id:1001, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1001, name:"ocp_pool", unit_count:1, unit_config_id:1001, zone_list:["myoceanbase1", "myoceanbase2", "myoceanbase3"], tenant_id:1002, replica_type:0}}, {unit:unit_id:1005, resource_pool_id:1003, unit_group_id:1002, zone:"myoceanbase1", server:"10.0.4.7:2890", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1003, name:"pool_bddb_myoceanbase1_oav", unit_count:1, unit_config_id:1003, zone_list:["myoceanbase1"], tenant_id:1004, replica_type:0}}]) [2024-10-16 14:58:15.539300] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=43] jx_debug: get tenant compatibility mode(tenant_id=1002, compat_mode=0) [2024-10-16 14:58:15.539307] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=6] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:58:15.539319] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=11] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:58:15.539335] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=15] jx_debug: get tenant compatibility mode(tenant_id=1004, compat_mode=0) [2024-10-16 14:58:15.539338] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:705) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=3] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2024-10-16 14:58:15.539347] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=9] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2024-10-16 14:58:15.540644] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:191) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=11] refresh resource plan success(tenant_id=1, plan_name=, directives=[]) [2024-10-16 14:58:15.541304] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=12] refresh resource user mapping rule(tenant_id=1, plan=, user_rules=[]) [2024-10-16 14:58:15.541962] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=10] refresh_resource_function_mapping_rule(tenant_id=1, plan=, rules=[]) [2024-10-16 14:58:15.541971] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=9] refresh resource mapping rule success(tenant_id=1, plan=) [2024-10-16 14:58:15.542646] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=30, max_blocked_buffer_size_=9830400, max_buffer_size_=19660800) [2024-10-16 14:58:15.542704] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=12] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:15.542714] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=8] tenant dfc status(ret=0, get_tenant_id()=1, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=30, get_max_blocked_buffer_size()=9830400, get_max_buffer_size()=19660800, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:58:15.544806] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=10] refresh resource user mapping rule(tenant_id=1001, plan=, user_rules=[]) [2024-10-16 14:58:15.545452] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=13] refresh_resource_function_mapping_rule(tenant_id=1001, plan=, rules=[]) [2024-10-16 14:58:15.545468] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=16] refresh resource mapping rule success(tenant_id=1001, plan=) [2024-10-16 14:58:15.546185] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=7] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:58:15.546251] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=10] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:15.546258] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=6] tenant dfc status(ret=0, get_tenant_id()=1001, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:58:15.546340] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=6] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=913199104, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=160542720, wash_ratio=6, tenant_id=1001) [2024-10-16 14:58:15.546404] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=21] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:15.546412] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1001][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=18359, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1001, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:15.546444] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=2097152, tenant_work_area_max_size=53687091, max_wa_memory_size=51589939, tmp_max_wa_memory_size=51589939, pre_mem_target=53687091, remain_memory_size=51589939, ratio=0.000000000000000000e+00, alloc_ratio=0.000000000000000000e+00, hold_ratio=3.906250014551915228e-02, tenant_memory_hold=973893632, washable_size=-2, max_workarea_memory_size=51589939, max_tenant_memory_size=99848192, wash_ratio=6, tenant_id=1002) [2024-10-16 14:58:15.546478] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=7] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6448742, calc_global_bound_size=1048576000000, mem_target_=51589939, wa_max_memory_size=51589939, profile_cnt_=0, total_memory_size=0, max_wa_size=51589939, avg_bound_size=6448742, max_bound_size=6448742, min_bound_size_=1048576) [2024-10-16 14:58:15.546483] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1002][Y0-0000000000000000-0-0] [lt=5] timer to calc global bound size(ret=0, global_bound_size_=6448742, manual_calc_cnt_=133749, drift_size_=0, pre_drift_size=0, wa_max_memory_size=51589939, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1002, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6448742, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=51589939, auto_calc=true, sql_mem_callback_.get_total_dump_size()=334976) [2024-10-16 14:58:15.546511] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=51589939, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=929976320, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=143765504, wash_ratio=6, tenant_id=1003) [2024-10-16 14:58:15.546547] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:15.546551] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1003][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=3011880, drift_size_=0, pre_drift_size=-1373433, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:15.546582] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=5] trace max work area(auto_calc=true, tenant_max_memory_limit=1073741824, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=53687091, max_wa_memory_size=53687091, tmp_max_wa_memory_size=53687091, pre_mem_target=53687091, remain_memory_size=53687091, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=961036288, washable_size=-2, max_workarea_memory_size=53687091, max_tenant_memory_size=112705536, wash_ratio=6, tenant_id=1004) [2024-10-16 14:58:15.546623] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=6] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=6710886, calc_global_bound_size=1048576000000, mem_target_=53687091, wa_max_memory_size=53687091, profile_cnt_=0, total_memory_size=0, max_wa_size=53687091, avg_bound_size=6710886, max_bound_size=6710886, min_bound_size_=1048576) [2024-10-16 14:58:15.546628] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [21076][SqlMemTimer][T1004][Y0-0000000000000000-0-0] [lt=4] timer to calc global bound size(ret=0, global_bound_size_=6710886, manual_calc_cnt_=32444, drift_size_=0, pre_drift_size=2417152, wa_max_memory_size=53687091, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=6710886, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=53687091, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2024-10-16 14:58:15.548640] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=7] refresh resource user mapping rule(tenant_id=1002, plan=, user_rules=[]) [2024-10-16 14:58:15.549355] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=12] refresh_resource_function_mapping_rule(tenant_id=1002, plan=, rules=[]) [2024-10-16 14:58:15.549366] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=10] refresh resource mapping rule success(tenant_id=1002, plan=) [2024-10-16 14:58:15.550017] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:58:15.550083] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=15] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:15.550093] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=8] tenant dfc status(ret=0, get_tenant_id()=1002, get_channel_cnt()=1, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=4, get_max_size_per_channel()=655360) [2024-10-16 14:58:15.551077] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] get_max_applied_scn(scn={val:1729061895450616000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:48, palf_committed_end_lsn_:{lsn:66390258902}, palf_committed_end_scn_:{val:1729061895450616000, v:0}, last_check_scn_:{val:1729061895450616000, v:0}, max_applied_cb_scn_:{val:1729061895450616000, v:0}}) [2024-10-16 14:58:15.551098] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info(lsn={lsn:33216819320}, scn={val:1722494261657150001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729015202827975, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1534711, ret_code_:0}}, next_to_submit_lsn_:{lsn:33216819320}, next_to_submit_scn_:{val:1722494261657150001, v:0}, base_lsn_:{lsn:33149755392}, base_scn_:{val:1722489888437877000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5c205000, next_round_pread_size:3678208, curr_read_pos:1023, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1023, log_storage_:{IteratorStorage:{start_lsn:{lsn:33216818297}, end_lsn:{lsn:33216819320}, read_buf:{buf_len_:3747840, buf_:0x2b0e5c205000}, block_size:67104768, log_storage_:0x2b0de0ad3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261657150000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:935, scn_:{val:1722494261657150000, v:0}, data_checksum:1236553573, flag:0}}, init_mode_version:0, accumulate_checksum:3157638874, curr_entry_is_padding:0, padding_entry_size:967, padding_entry_scn:{val:1722494261657150000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:593, miss_cnt:19, cache_read_size:381549, hit_ratio:"9.689542483660130712e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:19, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:36, read_io_size:67293184, read_disk_cost_ts:666110, start_lsn:{lsn:33149755392}}}}}}) [2024-10-16 14:58:15.552102] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=10] refresh resource user mapping rule(tenant_id=1003, plan=, user_rules=[]) [2024-10-16 14:58:15.552821] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=11] refresh_resource_function_mapping_rule(tenant_id=1003, plan=, rules=[]) [2024-10-16 14:58:15.552836] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=14] refresh resource mapping rule success(tenant_id=1003, plan=) [2024-10-16 14:58:15.553049] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1088) [2024-10-16 14:58:15.553637] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:58:15.553700] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=14] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:15.553710] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=9] tenant dfc status(ret=0, get_tenant_id()=1003, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:58:15.555709] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:159) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=9] refresh resource user mapping rule(tenant_id=1004, plan=, user_rules=[]) [2024-10-16 14:58:15.557215] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:196) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=11] refresh_resource_function_mapping_rule(tenant_id=1004, plan=, rules=[]) [2024-10-16 14:58:15.557230] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:100) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=14] refresh resource mapping rule success(tenant_id=1004, plan=) [2024-10-16 14:58:15.558230] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=5] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2024-10-16 14:58:15.558312] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=18] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2024-10-16 14:58:15.558325] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=12] tenant dfc status(ret=0, get_tenant_id()=1004, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2024-10-16 14:58:15.558346] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:108) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=13] refresh tenant units(sys_unit_cnt=3, units=[{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}, {tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}, {tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}, {tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}], ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:15.558394] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=42] finish set nway(tenant_id_=1, nway=3) [2024-10-16 14:58:15.558401] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=7] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1, nway=3, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:3, max_cpu:3, memory_size:"1GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:3}}, mode:0, create_timestamp:1717560171402717, is_removed:false}) [2024-10-16 14:58:15.558418] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=16] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=1.638399963378906250e+02, this=0x2b0ddc2dc170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=819, Throttle Trigger(MB)=491, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=3.948457122979588263e+01) [2024-10-16 14:58:15.558437] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=18] succ to set_memstore_threshold of memstore allocator(tenant_id=1, ret=0) [2024-10-16 14:58:15.558445] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=5] finish set nway(tenant_id_=1002, nway=1) [2024-10-16 14:58:15.558452] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=7] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1002, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:58:15.558466] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e0c298170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:58:15.558479] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1002][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=12] succ to set_memstore_threshold of memstore allocator(tenant_id=1002, ret=0) [2024-10-16 14:58:15.558487] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=4] finish set nway(tenant_id_=1001, nway=1) [2024-10-16 14:58:15.558495] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=7] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1001, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1001, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717560264557486, is_removed:false}) [2024-10-16 14:58:15.558509] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0df7098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:58:15.558522] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1001][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=13] succ to set_memstore_threshold of memstore allocator(tenant_id=1001, ret=0) [2024-10-16 14:58:15.558529] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=5] finish set nway(tenant_id_=1004, nway=1) [2024-10-16 14:58:15.558536] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=7] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1004, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1004, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"5.4GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:58:15.558550] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=14] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e40098170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:58:15.558562] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1004][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=12] succ to set_memstore_threshold of memstore allocator(tenant_id=1004, ret=0) [2024-10-16 14:58:15.558570] INFO set_nway (ob_tenant_mutil_allocator.cpp:357) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=4] finish set nway(tenant_id_=1003, nway=1) [2024-10-16 14:58:15.558577] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=7] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1003, nway=1, new_tma_limit=644245080, pre_tma_limit=644245080, cur_memstore_limit_percent=40, tenant_config={tenant_id:1003, unit_id:1005, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_bddb_myoceanbase1_1C2G_oav", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.6GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1717582234132088, is_removed:false}) [2024-10-16 14:58:15.558592] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:266) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=13] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=8.191999816894531250e+01, this=0x2b0e2c698170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=409, Resource Limit(MB)=409, Throttle Trigger(MB)=245, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.241572899671315326e+02) [2024-10-16 14:58:15.558611] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [21387][OmtNodeBalancer][T1003][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=20] succ to set_memstore_threshold of memstore allocator(tenant_id=1003, ret=0) [2024-10-16 14:58:15.559969] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=4] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:15.559985] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=16] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1001, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:15.559992] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=7] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:15.559998] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=5] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1003, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:15.560004] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=5] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, value_updated=true, data_version={version_str:"4.3.2.0", version_val:17180066304}) [2024-10-16 14:58:15.560026] INFO [SERVER] cal_all_part_disk_default_percentage (ob_server_utils.cpp:301) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=4] cal_all_part_disk_default_percentage succ(data_dir="/mnt/myoceanbase/oceanbase/data//sstable", clog_dir="/mnt/myoceanbase/oceanbase/data//clog", shared_mode=true, data_disk_total_size=214429704192, data_disk_default_percentage=60, clog_disk_total_size=214429704192, clog_disk_default_percentage=30) [2024-10-16 14:58:15.560034] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:337) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=9] decide disk size finished(suggested_disk_size=32212254720, suggested_disk_percentage=0, default_disk_percentage=30, total_space=214429704192, disk_size=32212254720) [2024-10-16 14:58:15.560041] INFO [SERVER] get_log_disk_info_in_config (ob_server_utils.cpp:88) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=6] get_log_disk_info_in_config(suggested_data_disk_size=32212254720, suggested_clog_disk_size=32212254720, suggested_data_disk_percentage=0, suggested_clog_disk_percentage=0, log_disk_size=32212254720, log_disk_percentage=0, total_log_disk_size=214429704192) [2024-10-16 14:58:15.560049] INFO [CLOG] try_resize (ob_server_log_block_mgr.cpp:800) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=6] try_resize success(ret=0, log_disk_size=32212254720, total_log_disk_size=214429704192, this={dir::"/mnt/myoceanbase/oceanbase/data//clog/log_pool", dir_fd:15, meta_fd:16, log_pool_meta:{curr_total_size:32212254720, next_total_size:32212254720, status:0}, min_block_id:15037, max_block_id:15335, min_log_disk_size_for_all_tenants_:15032385536, is_inited:true}) [2024-10-16 14:58:15.560100] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:132) [21387][OmtNodeBalancer][T0][YB4A0A000407-00061E9ABF1C7C0F-0-0] [lt=50] refresh tenant config(tenants=[1, 1001, 1002, 1003, 1004], ret=0) [2024-10-16 14:58:15.569189] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.569211] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.569223] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=3] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.569569] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.572198] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1729061895076457002, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=495733, in_cluster_service=true, cluster_version={val:1729061894678956004, v:0}, min_cluster_version={val:1729061894678956004, v:0}, max_cluster_version={val:1729061894678956004, v:0}, get_cluster_version_err=0, cluster_version_delta=893234, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65228570, succ_cluster_heartbeat_count=126, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061454608479003, v:0}, local_cluster_delta=440963711, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:15.577060] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061895576296000, start_id=1729061895576296000, end_id=1729061895576296001) [2024-10-16 14:58:15.577088] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1729061895577083], gts:1729061895576296000, latest_srr:[mts=1729061895577083]}) [2024-10-16 14:58:15.579136] INFO [STORAGE] get_kept_snapshot_info (ob_tablet.cpp:5208) [22809][T1004_SSTableGC][T1004][YB4A0A000407-00061E997A9CB61D-0-0] [lt=14] tablet multi version start not advance for a long time(ret=0, ls_id={id:1}, tablet_id={id:60379}, snapshot_info={type:"SNAPSHOT_ON_TABLET", snapshot:1729015311100270042}, old_min_reserved_snapshot=1729060091854891000, min_medium_snapshot=9223372036854775807, min_reserved_snapshot_on_ls=1729060091854891000) [2024-10-16 14:58:15.589711] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=8] [APPLY STAT CB APPEND COST TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=46, this=0x2b0e260ee280, extra_info="") [2024-10-16 14:58:15.589726] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=15] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=1126, this=0x2b0e260ee4b0, extra_info="") [2024-10-16 14:58:15.589731] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e260ee6e0, extra_info="") [2024-10-16 14:58:15.589734] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=-707, this=0x2b0e260ee910, extra_info="") [2024-10-16 14:58:15.589738] INFO [LIB] stat (utility.h:1165) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=29, stat_interval=5000000, avg (count/cost)=465, this=0x2b0e260eeb40, extra_info="") [2024-10-16 14:58:15.589957] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C46F3-0-0] [lt=12] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=426, avg_log_submit_to_slide_time=1132) [2024-10-16 14:58:15.590198] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE79FF-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:15.590429] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1211) [2024-10-16 14:58:15.591433] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061895589535000, start_id=1729061895589535000, end_id=1729061895589535001) [2024-10-16 14:58:15.591669] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.598324] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:15.598338] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:15.599110] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1002}, {id:1003}]) [2024-10-16 14:58:15.607338] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:124) [22809][T1004_SSTableGC][T1004][YB4A0A000407-00061E997A9CB61D-0-0] [lt=19] SSTableGCTask(cost_ts=28392) [2024-10-16 14:58:15.607411] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC58-0-0] [lt=9] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1004, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:58:15.607442] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC58-0-0] [lt=24] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:15.607465] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107006][T1004_L0_G19][T1004][YB4A0A000409-00061E9A3404DC58-0-0] [lt=21] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:58:15.608941] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE8523F-0-0] [lt=6] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:15.609244] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=255, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:15.609257] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:15.609269] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=86, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:15.609303] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:15.612737] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6074, mem_used=12478464) [2024-10-16 14:58:15.615552] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=5071) [2024-10-16 14:58:15.615578] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.620001] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=48, replace_time=0, replace_start_pos=157285, replace_num=31457) [2024-10-16 14:58:15.620016] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=48, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:15.625317] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=11] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601538, offset=9663286}, end_cursor:ObLogCursor{file_id=192, log_id=47601539, offset=9663570}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2254050, len:3274, log_data_len:284, seq:47601538, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:15.627371] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4C5-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061895626467000, start_id=1729061895626467000, end_id=1729061895626467001) [2024-10-16 14:58:15.628662] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601539, offset=9663570}, end_cursor:ObLogCursor{file_id=192, log_id=47601540, offset=9663844}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2178050, len:2990, log_data_len:274, seq:47601539, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:15.631862] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601540, offset=9663844}, end_cursor:ObLogCursor{file_id=192, log_id=47601541, offset=9664118}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21f0050, len:2716, log_data_len:274, seq:47601540, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:15.632012] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601541, offset=9664118}, end_cursor:ObLogCursor{file_id=192, log_id=47601542, offset=9664392}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd226c050, len:2442, log_data_len:274, seq:47601541, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:15.632075] INFO [PALF] next (palf_iterator.h:170) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561624, miss_cnt:1, cache_read_size:152590829030, hit_ratio:"9.999999928347065303e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:15.632088] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599210, miss_cnt:9, cache_read_size:149778055575, hit_ratio:"9.999999345926520178e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:15.634444] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=10] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257886, offset=39068030}, end_cursor:ObLogCursor{file_id=197, log_id=49257887, offset=39068313}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd219c050, len:3714, log_data_len:283, seq:49257886, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:15.637062] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:473) [22628][T1004_TeRec][T1004][YB4A0A000407-00061E9B221FB390-0-0] [lt=20] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=343520996, MAX_GAP=900000000000, new_sync_scn={val:1729061894520671000, v:0}, new_replayable_scn={val:1729061894520671000, v:0}, new_readable_scn={val:1729061894177150004, v:0}, old_tenant_info={tenant_id:1004, tenant_role:{value:1}, switchover_status:{value:1}, switchover_epoch:0, sync_scn:{val:1729061893074505004, v:0}, replayable_scn:{val:1729061893074505004, v:0}, standby_scn:{val:1729061893074505003, v:0}, recovery_until_scn:{val:4611686018427387903, v:0}, log_mode:{archive mode:"ARCHIVELOG"}, max_ls_id:{id:1003}, restore_data_mode:{restore data mode:"NORMAL"}}) [2024-10-16 14:58:15.637270] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB390-0-0] [lt=35] get number(ret=0, service_type_=0, range=1, base_id=1729061895636310000, start_id=1729061895636310000, end_id=1729061895636310001) [2024-10-16 14:58:15.639900] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=22] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257887, offset=39068313}, end_cursor:ObLogCursor{file_id=197, log_id=49257888, offset=39068595}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2260050, len:3431, log_data_len:282, seq:49257887, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:15.642871] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=18] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257888, offset=39068595}, end_cursor:ObLogCursor{file_id=197, log_id=49257889, offset=39068877}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd20a4050, len:3149, log_data_len:282, seq:49257888, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:15.643788] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257889, offset=39068877}, end_cursor:ObLogCursor{file_id=197, log_id=49257890, offset=39069159}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2220050, len:2867, log_data_len:282, seq:49257889, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:15.643923] INFO [PALF] next (palf_iterator.h:170) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650551, miss_cnt:32, cache_read_size:8050458590, hit_ratio:"9.999994892306109939e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:15.643954] INFO [PALF] next (palf_iterator.h:170) [22825][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657291, miss_cnt:17, cache_read_size:8050206093, hit_ratio:"9.999997286828856691e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:15.646669] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:540) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1729061895076457003, v:0}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=570206, in_cluster_service=true, cluster_version={val:1729061895076457001, v:0}, min_cluster_version={val:1729061895076457001, v:0}, max_cluster_version={val:1729061895076457001, v:0}, get_cluster_version_err=0, cluster_version_delta=570206, cluster_service_master="10.0.4.7:2890", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=65244491, succ_cluster_heartbeat_count=264, cluster_heartbeat_interval=50000, local_cluster_version={val:1729061328670161003, v:0}, local_cluster_delta=566976502, force_self_check=false, weak_read_refresh_interval=100000) [2024-10-16 14:58:15.647481] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1729061895646883000, start_id=1729061895646883000, end_id=1729061895646883001) [2024-10-16 14:58:15.649053] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=37] success to update min reserved snapshot(reserved_snapshot=1729060090214512000, duration=1800, snapshot_gc_ts=1729061890214512000) [2024-10-16 14:58:15.650332] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=15] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060090214512000) [2024-10-16 14:58:15.651937] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=40] get number(ret=0, service_type_=0, range=1, base_id=1729061895651624000, start_id=1729061895651624000, end_id=1729061895651624001) [2024-10-16 14:58:15.652527] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=17] submit reserved snapshot log success(ls_id={id:1001}, new_snapshot_version=1729060090214512000) [2024-10-16 14:58:15.652541] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=12] update reserved snapshot finished(cost_ts=3490, reserved_snapshot=1729060090214512000) [2024-10-16 14:58:15.655108] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1729061895450616000, v:0}, new_max_version={val:1729061895450616000, v:0}, last_min_version={val:1729061895076457000, v:0}, last_max_version={val:1729061895076457000, v:0}, total_time_us=3973, delta_us=204490) [2024-10-16 14:58:15.655131] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1729061895450616000, v:0}, min_version={val:1729061895450616000, v:0}, max_version={val:1729061895450616000, v:0}, in_service=true, leader_epoch=48) [2024-10-16 14:58:15.673245] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061895673141000, start_id=1729061895673141000, end_id=1729061895673141001) [2024-10-16 14:58:15.673506] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0DAC-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061895673141000, start_id=1729061895673141001, end_id=1729061895673141002) [2024-10-16 14:58:15.673605] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B800D0DAB-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061895673141000, start_id=1729061895673141000, end_id=1729061895673141001) [2024-10-16 14:58:15.673725] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0DA9-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061895673141000, start_id=1729061895673141000, end_id=1729061895673141001) [2024-10-16 14:58:15.673751] INFO [PALF] read (log_cache.cpp:103) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:15.673790] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0DAD-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061895673141000, start_id=1729061895673141000, end_id=1729061895673141001) [2024-10-16 14:58:15.673812] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0DAA-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061895673141000, start_id=1729061895673141000, end_id=1729061895673141001) [2024-10-16 14:58:15.675615] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [22804][T1004_ArcSender][T1004][YB4A0A000407-00061E99728960C1-0-0] [lt=11][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:15.677515] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:15.686609] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=27, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:15.686633] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:15.686658] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=1332, avg_group_log_size=121) [2024-10-16 14:58:15.686668] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1332) [2024-10-16 14:58:15.686919] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1758) [2024-10-16 14:58:15.686942] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1758) [2024-10-16 14:58:15.687292] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4517-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:15.687302] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4517-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1154, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:15.687346] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7A01-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121) [2024-10-16 14:58:15.687371] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7A01-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0dac16a0, extra_info="palf_id:1002") [2024-10-16 14:58:15.687374] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B7FDE7A01-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=60302, this=0x2b0e0dac18d0, extra_info="palf_id:1002") [2024-10-16 14:58:15.687498] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=19] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=33, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:15.687514] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:15.687619] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=263, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:15.687629] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=122, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:15.691726] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.694857] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:175) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=6] query dest_info_ succ(dest_info=[[0, "file:///mnt/oceanbasedata/bddb/archive"]], archive_dest_changed=false) [2024-10-16 14:58:15.694873] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [22716][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=15] finish to resize log external storage handler(current_ts=1729061895694871, tenant_max_cpu=1, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:58:15.695420] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061895694467000, start_id=1729061895694467000, end_id=1729061895694467001) [2024-10-16 14:58:15.705194] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=11] gc stale ls task succ [2024-10-16 14:58:15.709904] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85241-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:15.709920] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85241-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=50547, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:15.709926] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3AE85241-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=218497765, log_task={header:{begin_lsn:{lsn:207589691134}, end_lsn:{lsn:207589691256}, log_id:218497765, min_scn:{val:1729061895178386004, v:0}, max_scn:{val:1729061895178386004, v:0}, data_len:66, proposal_id:81, prev_lsn:{lsn:207589691012}, prev_proposal_id:81, committed_end_lsn:{lsn:207589691134}, data_checksum:940009244, accum_checksum:3720516486, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061895608930, freeze_ts:1729061895608930, submit_ts:1729061895608936, flushed_ts:1729061895609279, gen_to_freeze cost time:0, gen_to_submit cost time:6, submit_to_flush cost time:343}, fs_cb_begin_ts=1729061895709897, log_life_time=100967) [2024-10-16 14:58:15.715564] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] tx gc loop thread is running(MTL_ID()=1) [2024-10-16 14:58:15.715578] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=14] try gc retain ctx [2024-10-16 14:58:15.715582] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:15.716222] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061907008855000, latest_log_ts:{val:1729061890536200001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:15.716344] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:15.716356] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:15.716361] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.724450] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1577222144, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:15.731823] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:15.737007] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=7] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:15.737031] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [21900][T1001_TabletShe][T1001][Y0-0000000000000000-0-0] [lt=15] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=221) [2024-10-16 14:58:15.754177] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:15.754193] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:15.754590] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431f7ca0, extra_info="") [2024-10-16 14:58:15.754831] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD60B-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=35, avg_log_submit_to_flush_time=683, avg_log_submit_to_slide_time=1089) [2024-10-16 14:58:15.755538] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=34] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:15.762380] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=4] gc stale ls task succ [2024-10-16 14:58:15.762410] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=16] ls not need do archive on this server(ls_id={id:1002}) [2024-10-16 14:58:15.762418] INFO [ARCHIVE] add_ls_task_ (ob_ls_mgr.cpp:454) [22801][T1004_LSArchive][T1004][YB4A0A000407-00061E99725960C1-0-0] [lt=7] ls not need do archive on this server(ls_id={id:1003}) [2024-10-16 14:58:15.767841] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1616-0-0] [lt=2][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:58:15.773672] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22241][T1002_LSArchive][T1002][YB4A0A000407-00061E996F3960C1-0-0] [lt=17] gc stale ls task succ [2024-10-16 14:58:15.773850] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22F3-0-0] [lt=2][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2024-10-16 14:58:15.774076] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54078-0-0] [lt=14][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173491, log_task={header:{begin_lsn:{lsn:14352415994}, end_lsn:{lsn:14352416116}, log_id:115173491, min_scn:{val:1729061895178386004, v:0}, max_scn:{val:1729061895178386004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352415872}, prev_proposal_id:17, committed_end_lsn:{lsn:14352415994}, data_checksum:940009244, accum_checksum:2141451727, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061895673693, freeze_ts:1729061895673693, submit_ts:1729061895673696, flushed_ts:1729061895673901, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:205}, fs_cb_begin_ts=1729061895774069, log_life_time=100376) [2024-10-16 14:58:15.783462] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [22805][T1004_ArcTimer][T1004][YB4A0A000407-00061E99729960C1-0-0] [lt=16] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1728967049237798000, v:0}, record_count=129, valid_record_count=2) [2024-10-16 14:58:15.788307] INFO [PALF] read (log_cache.cpp:103) [22265][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1333, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:15.788619] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21309][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.788696] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21305][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.788714] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21306][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.788713] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21300][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.788723] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21304][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.788730] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21301][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.788737] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21302][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.788749] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21298][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.788764] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:641) [21299][BatchIO][T0][Y0-0000000000000000-0-0] [lt=3] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.788840] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:676) [21307][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.789627] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21311][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:15.791560] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061895790136000, start_id=1729061895790136000, end_id=1729061895790136001) [2024-10-16 14:58:15.791799] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.794880] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=20] get_max_applied_scn(scn={val:1729061895673141000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:33576, palf_committed_end_lsn_:{lsn:114265084983}, palf_committed_end_scn_:{val:1729061895790136000, v:0}, last_check_scn_:{val:1729061895790136000, v:0}, max_applied_cb_scn_:{val:1729061895673141000, v:0}}) [2024-10-16 14:58:15.794914] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=29] get_min_unreplayed_log_info(lsn={lsn:45338084276}, scn={val:1722494261661057002, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729057302384041, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:14197405, ret_code_:0}}, next_to_submit_lsn_:{lsn:45338084276}, next_to_submit_scn_:{val:1722494261661057002, v:0}, base_lsn_:{lsn:45161508864}, base_scn_:{val:1722482620740045000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e95205000, next_round_pread_size:3678208, curr_read_pos:1350, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1350, log_storage_:{IteratorStorage:{start_lsn:{lsn:45338082926}, end_lsn:{lsn:45338084276}, read_buf:{buf_len_:3747840, buf_:0x2b0e95205000}, block_size:67104768, log_storage_:0x2b0e2efe9a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261661057001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:669, scn_:{val:1722494261661057001, v:0}, data_checksum:940282441, flag:1}}, init_mode_version:0, accumulate_checksum:3026293488, curr_entry_is_padding:0, padding_entry_size:701, padding_entry_scn:{val:1722494261661057001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:434, miss_cnt:50, cache_read_size:280261, hit_ratio:"8.966942148760330689e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:50, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:93, read_io_size:177795072, read_disk_cost_ts:2267593, start_lsn:{lsn:45161508864}}}}}}) [2024-10-16 14:58:15.795693] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=8][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:0, dropped:118, tid:21154}]) [2024-10-16 14:58:15.796702] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EF5-0-0] [lt=32] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43cc1470, extra_info="") [2024-10-16 14:58:15.796719] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2EF5-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=1089) [2024-10-16 14:58:15.800742] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1626-0-0] [lt=1][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363669136, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894403, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.802143] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1628-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363670538, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894404, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.807782] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C162A-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363676175, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894405, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.810996] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C162C-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363679388, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894406, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.812207] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C162F-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363680602, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894407, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.813008] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1630-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363681405, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894408, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.814901] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:111799719, max_lsn:17493521871, max_scn:{val:1729061895576296001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:7667642653}, reuse_lsn:{lsn:17493521871}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:17451578831}}, last_submit_lsn:{lsn:17493521749}, last_submit_end_lsn:{lsn:17493521871}, last_submit_log_id:111799719, last_submit_log_pid:17, max_flushed_lsn:{lsn:17493521749}, max_flushed_end_lsn:{lsn:17493521871}, max_flushed_log_pid:17, committed_end_lsn:{lsn:17493521871}, last_slide_log_id:111799719, last_slide_scn:{val:1729061895576296001, v:0}, last_slide_lsn:{lsn:17493521749}, last_slide_end_lsn:{lsn:17493521871}, last_slide_log_pid:17, last_slide_log_accum_checksum:1261949231, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0e431d4530}, [StateMgr]={this:0x2b0e431d8830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, leader:"10.0.4.7:2890", leader_epoch:5, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:111799720, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261853177, palf_role_change_cb:0x2b0e431df230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, reconfig_barrier:{prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_end_lsn:{lsn:7668488755}, prev_mode_pid:17}, persistent_config_version:{proposal_id:17, config_seq:18}, ms_ack_list:0, resend_config_version:{proposal_id:17, config_seq:18}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494261753074, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494261753074, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0e431d5c70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494262053331, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728249494772393002, v:0}, min_block_id:241, min_block_min_scn:{val:1728207118648006001, v:0}, base_lsn_for_block_gc:{lsn:17380134912}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:17}, log_config_meta:{version:3, proposal_id:17, prev:{version:1, config:{config_version:{proposal_id:16, config_seq:17}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:17, config_seq:18}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:16, prev_lsn:{lsn:7668488633}, prev_mode_pid:17}, log_snapshot_meta:{version:1, base_lsn:{lsn:17380134912}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:17, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:757760}, readable_log_tail:{lsn:757760}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17514344448}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/meta", dir_fd:193, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66347008, block_header_serialize_buf_:0x2b0e431da9f8, flashback_version:0}, log_storage:{log_tail:{lsn:17493521871}, readable_log_tail:{lsn:17493521871}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:17447239680}, min_scn:{val:1729032503563174001, v:0}, curr_block_id:260, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1004/1/log", dir_fd:195, min_block_id:241, max_block_id:261, curr_writable_block_id:260}, logical_block_size_:67104768, curr_block_writable_size_:20822577, block_header_serialize_buf_:0x2b0e431dc6b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261852794, this:0x2b0e431d8f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:17, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:16, majority_max_lsn:{lsn:7668488755}, saved_end_lsn:{lsn:7668488755}, last_submit_prepare_req_time_us:1722494261853360, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0e431d89b0}) [2024-10-16 14:58:15.815041] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=136] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061895754632, last_flushed_end_lsn:{lsn:17493521871}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061895754798, last_flushed_end_lsn:{lsn:17493521871}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061895756128, last_flushed_end_lsn:{lsn:17493521871}}]) [2024-10-16 14:58:15.815053] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=111799720, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061895754632, last_flushed_end_lsn:{lsn:17493521871}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061895754798, last_flushed_end_lsn:{lsn:17493521871}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061895756128, last_flushed_end_lsn:{lsn:17493521871}}]) [2024-10-16 14:58:15.816414] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.816812] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4816-0-0] [lt=11] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061895784529, keepalive_ts:-1}) [2024-10-16 14:58:15.816824] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107369][T1001_L0_G1][T1001][YB4A0A000409-00061E99FD5C4816-0-0] [lt=11] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061895784529, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:15.816964] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1632-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363685339, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894409, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.820482] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1634-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363688877, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894410, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.821501] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=42, replace_time=0, replace_start_pos=188742, replace_num=31457) [2024-10-16 14:58:15.821517] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] replace map num details(ret=0, replace_node_count=42, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:15.828390] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1636-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363696783, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894411, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.830086] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1638-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363698480, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894412, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.830226] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:15.830236] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=8] get_disk_usage(ret=0, capacity(MB):=614, used(MB):=434) [2024-10-16 14:58:15.830247] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22458][T1003_CKClogDis][T1003][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=52769847, threshold=193273528, need_update_checkpoint_scn=false) [2024-10-16 14:58:15.831387] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C163B-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363699783, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894413, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.831501] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22F4-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363699895, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894414, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.832836] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C163D-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363701232, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894415, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.834131] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C163F-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363702527, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894416, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.835530] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=15] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1003}, {id:1002}]) [2024-10-16 14:58:15.837556] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1641-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363705952, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894417, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.837823] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E5-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:15.837852] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E5-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061895178386004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061895178386004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:15.839056] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1643-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363707452, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894418, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.840435] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9E5-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061895839497000, start_id=1729061895839497000, end_id=1729061895839497001) [2024-10-16 14:58:15.840444] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1645-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363708840, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894419, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.844507] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E5-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061895843780000, start_id=1729061895843780000, end_id=1729061895843780001) [2024-10-16 14:58:15.844587] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E5-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=9442, avg_group_log_size=629) [2024-10-16 14:58:15.844597] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E5-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=9442) [2024-10-16 14:58:15.844880] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=312, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:15.844892] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=9442) [2024-10-16 14:58:15.844903] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=214, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:15.844924] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=9442) [2024-10-16 14:58:15.844931] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:15.845189] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797E8B-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:15.845195] INFO [LIB] stat (utility.h:1165) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797E8B-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=1149, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:15.845199] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107485][T1003_L0_G1][T1003][YB4A0A000409-00061E9A3A797E8B-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=660, avg_log_submit_to_slide_time=1114) [2024-10-16 14:58:15.845263] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=7] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:15.847645] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1648-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363716038, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894420, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.847663] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061895845894000, start_id=1729061895845894000, end_id=1729061895845894001) [2024-10-16 14:58:15.848963] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22F5-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363717358, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894421, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.850429] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C164A-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363718824, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894422, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.852053] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061895851437000, start_id=1729061895851437000, end_id=1729061895851437001) [2024-10-16 14:58:15.853415] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C164C-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363721810, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894423, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.854994] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C164E-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363723390, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894424, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.860270] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1650-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363728665, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894425, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.861489] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [41376][T1002_L0_G0][T1002][YB4A0A000407-00061E9B251ED0BA-0-0] [lt=1][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363729883, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894426, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.861566] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:121) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=13] new mysql sessid created(conn.sessid_=3221613899, support_ssl=false) [2024-10-16 14:58:15.861610] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:141) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=12] sm conn init succ(conn.sessid_=3221613899, sess.client_addr_="10.0.4.7:53488") [2024-10-16 14:58:15.861619] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1082) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=7] accept one succ(*s={this:0x2b0e5cdcc030, session_id:3221613899, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:15.861671] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1001) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x2b0e5cdcc030, session_id:3221613899, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:15.861682] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:268) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=9] kill and revert session(conn.sessid_=3221613899, proxy_sessid=0, server_id=1, ret=0) [2024-10-16 14:58:15.861687] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:976) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=4] sock ref clean, do destroy(*s={this:0x2b0e5cdcc030, session_id:3221613899, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:211}, err:5, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2024-10-16 14:58:15.861693] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:223) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] mark session id unused(conn.sessid_=3221613899) [2024-10-16 14:58:15.861697] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:243) [21315][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=3] connection close(sessid=3221613899, proxy_sessid=0, tenant_id=0, server_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=false, ret=0, trace_id=Y0-0000000000000000-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:0], disconnect_state=0) [2024-10-16 14:58:15.863234] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1652-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363731628, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894427, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.863257] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=16] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7039, mem_used=14558208) [2024-10-16 14:58:15.864694] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1654-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363733090, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894428, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.870520] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1656-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363738914, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894429, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.872364] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [22795][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=13] no need to update min/max version(new_version={val:1729061895076457003, v:0}, min_version={val:1729061895076457003, v:0}, max_version={val:1729061895076457003, v:0}) [2024-10-16 14:58:15.879297] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1658-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363747690, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894430, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.884072] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C72-0-0] [lt=15] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:15.884109] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C72-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061895673141001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061895673141001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:15.885489] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C72-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061895884859000, start_id=1729061895884859000, end_id=1729061895884859001) [2024-10-16 14:58:15.887036] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=297, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:15.887056] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=136, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:15.887067] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1332) [2024-10-16 14:58:15.887076] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:15.888956] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C165C-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363757350, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894431, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.889052] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C72-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1729061895889000000, start_id=1729061895889000000, end_id=1729061895889000001) [2024-10-16 14:58:15.891864] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.892660] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22237][T1002_Flush][T1002][Y0-0000000000000000-0-0] [lt=7] ====== traversal_flush timer task ====== [2024-10-16 14:58:15.892849] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4E-0-0] [lt=10] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015248776112002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015248776112003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015248990476001, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015248990476002, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015248990476003, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015248990476004, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015248990476005, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015248990476006, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015248990476007, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015248990476008, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015248990476009, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015248990476010, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015248990476011, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015248990476012, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015248990476013, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015248990476014, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015248990476015, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015248990476016, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015248990476017, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015248990476018, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015248990476019, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015248990476020, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015248990476021, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015248990476022, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015248990476023, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015248990476024, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015248990476025, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015248990476026, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015248990476027, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015248990476028, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015248990476029, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015248990476030, v:0}}, }, max_consequent_callbacked_scn={val:1729061895576296003, v:0}, *this={this:0x2b0e26096510, is_inited:true, freezing_scn:{val:1722494667464207000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015248776112002, v:0}, clog_base_lsn:{lsn:17849868288}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061893574836000, v:0}, tablet_change_checkpoint_scn:{val:1728978212337222000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061910098662000, latest_log_ts:{val:1729061893511066001, v:0}}, {limited_id:394000001, latest_log_ts:{val:1729061599022421002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:15.892980] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4E-0-0] [lt=86] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209243}, rec_scn:{val:1729045387747012001, v:0}}, {tablet_id:{id:209231}, rec_scn:{val:1729045387747012002, v:0}}, {tablet_id:{id:209249}, rec_scn:{val:1729045387747012003, v:0}}, {tablet_id:{id:209255}, rec_scn:{val:1729045387747012005, v:0}}, {tablet_id:{id:209237}, rec_scn:{val:1729045387747012006, v:0}}, }, max_consequent_callbacked_scn={val:1729061895576296003, v:0}, *this={this:0x2b0e2610c510, is_inited:true, freezing_scn:{val:1729024996167575002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045387747012001, v:0}, clog_base_lsn:{lsn:203595866112}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061893574836000, v:0}, tablet_change_checkpoint_scn:{val:1729008973800733001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:15.893052] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4E-0-0] [lt=36] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:7, ordered_tablets:{{tablet_id:{id:209259}, rec_scn:{val:1729044692383096004, v:0}}, {tablet_id:{id:209241}, rec_scn:{val:1729044692383096005, v:0}}, {tablet_id:{id:209247}, rec_scn:{val:1729044692383096006, v:0}}, {tablet_id:{id:209235}, rec_scn:{val:1729044692383096007, v:0}}, {tablet_id:{id:209253}, rec_scn:{val:1729044692383096009, v:0}}, {tablet_id:{id:209283}, rec_scn:{val:1729060657594851005, v:0}}, {tablet_id:{id:209289}, rec_scn:{val:1729060657594851006, v:0}}, }, max_consequent_callbacked_scn={val:1729061895673141002, v:0}, *this={this:0x2b0e26180510, is_inited:true, freezing_scn:{val:1729027575505744001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729044692383096004, v:0}, clog_base_lsn:{lsn:212587905024}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061893574836000, v:0}, tablet_change_checkpoint_scn:{val:1729008958520189002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:87, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:15.893124] INFO [MDS] flush (mds_table_mgr.cpp:195) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4E-0-0] [lt=39] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:5, ordered_tablets:{{tablet_id:{id:209245}, rec_scn:{val:1729045459272503001, v:0}}, {tablet_id:{id:209251}, rec_scn:{val:1729045459272503002, v:0}}, {tablet_id:{id:209257}, rec_scn:{val:1729045459272503003, v:0}}, {tablet_id:{id:209239}, rec_scn:{val:1729045459272503004, v:0}}, {tablet_id:{id:209233}, rec_scn:{val:1729045459272503005, v:0}}, }, max_consequent_callbacked_scn={val:1729061895678607001, v:0}, *this={this:0x2b0e27a04510, is_inited:true, freezing_scn:{val:1729025598956677002, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1002, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729045459272503001, v:0}, clog_base_lsn:{lsn:207152418816}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061893574836000, v:0}, tablet_change_checkpoint_scn:{val:1729008963462925004, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:80, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:15.893157] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22237][T1002_Flush][T1002][YB4A0A000407-00061E9B37898D4E-0-0] [lt=33] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:58:15.895756] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061895895275000, start_id=1729061895895275000, end_id=1729061895895275001) [2024-10-16 14:58:15.899550] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C165E-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363767944, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894432, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.901128] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1660-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363769523, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894433, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.901181] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF22F9-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363769575, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894434, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.902597] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1662-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363770993, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894435, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.903414] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1663-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363771810, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894436, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.904825] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1665-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363773222, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894437, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.914553] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1667-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363782947, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894438, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.916508] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.919474] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1669-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363787867, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894439, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.920137] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:256) [21904][T1001_ArcSender][T1001][YB4A0A000407-00061E996DB960C1-0-0] [lt=14][errcode=-4018] try free send task failed(ret=-4018) [2024-10-16 14:58:15.920975] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C166B-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363789370, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894440, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.922413] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C166D-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363790807, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894441, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.923701] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C166F-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363792097, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894442, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.924742] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1577222144, tenant_max_wash_size=3331686, tenant_min_wash_size=3331686, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:15.925024] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1671-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363793420, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894443, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.926328] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1673-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363794724, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894444, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.927782] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=14] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:15.927801] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1675-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363796198, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894445, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.929399] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=7] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:15.933055] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1677-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363801450, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894446, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.934961] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:791) [22056][T1002_PalfGC][T1002][Y0-0000000000000000-0-0] [lt=22] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):5529, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):1634, sequence:1}) [2024-10-16 14:58:15.941080] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1679-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363809474, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894447, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.946293] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C167B-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363814687, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894448, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.950198] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=4] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:15.950222] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=16] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:15.950231] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [21899][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=7] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=156) [2024-10-16 14:58:15.950402] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5587, mem_used=12478464) [2024-10-16 14:58:15.954284] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:15.954303] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:15.957365] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [105511][T1004_ComTimerT][T1004][Y0-0000000000000000-0-0] [lt=16] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:15.960813] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C167D-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363829206, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894449, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.966284] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C167F-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363834676, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894450, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.967839] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1681-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363836225, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894451, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.969127] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1683-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363837523, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894452, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.972330] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1685-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363840724, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894453, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.973275] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061895972048000, start_id=1729061895972048000, end_id=1729061895972048001) [2024-10-16 14:58:15.973962] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061895973431000, start_id=1729061895973431000, end_id=1729061895973431001) [2024-10-16 14:58:15.974004] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5407C-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:15.974016] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5407C-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:15.974533] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22282][T1003_Occam][T1003][YB4A0A000407-00061E9B364A15F4-0-0] [lt=13] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:15.974987] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1687-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363843383, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894454, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.975031] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=354, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:15.975045] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=26, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:15.975116] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=39] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:15.976683] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1689-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363845080, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894455, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.978189] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C168B-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363846572, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894456, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.989270] INFO [PALF] inner_write_once_ (log_block_handler.cpp:411) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=758480, this={dio_aligned_buf:{buf_write_offset:2344, buf_padding_size:1752, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:111160443, truncate_used_ts:41411039}, log_block_size:67108864, dir_fd:177, io_fd:142}, aligned_buf_len=4096, aligned_block_offset=757760, buf_len=1624, total_write_size_=121658, total_write_size_after_dio_=569344, ob_pwrite_used_ts=31437, count=109, each_pwrite_cost=288) [2024-10-16 14:58:15.990405] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f56bf68 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285947} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0f7f4e1678 trans_version={val:1729061835478278000, v:0} scn={val:1729061835478278000, v:0} tx_id={txid:392284901} prev=0x2b0f7f4e1538 next=(nil) modify_count=1 acc_checksum=670580532 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5368}, snapshot_version={val:1729061895989261000, v:0}, *this={this=0x2b0f7f4e1230 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4e1678 latest_compact_node=(nil) max_trans_version={val:1729061835478278000, v:0} max_trans_id=392284901 max_elr_trans_version={val:1729061835478278000, v:0} max_elr_trans_id=392284901 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835478278000, v:0} min_modify_scn={val:1729061835478278000, v:0}}) [2024-10-16 14:58:15.990438] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f56bf68 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285947} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851220, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851220, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061895989261000, v:0}, tx_id:{txid:392285947}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285947}, tx_desc:{this:0x2b0ec379e210, tx_id:{txid:392285947}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061895989261, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061895989261, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:15.990494] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285947} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285947}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:15.990511] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830044559560704},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285947} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:15.990526] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:15.990532] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044559560704},{"BIGINT":1729058400},{"VARCHAR":"?±ë…¸Qì?®¸Që…¸?®¸Që…¸?®¸Që…¸?©™™™™™š?®¸Që…¸?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?· =p£× ?±ë…¸Qì?´záG®{?®¸Që…¸?©™™™™™š?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?´záG®{?´záG®{?´záG®{?Ç =p£× ?±ë…¸Qì?±ë…¸Qì?®¸Që…¸?®¸Që…¸?©™™™™™š?Íp£× =q?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?· =p£× ?´záG®{?¼(õÂ\)?´záG®{?©™™™™™š?®¸Që…¸?´záG®{?· =p£× ?´záG®{?· =p£× ?´záG®{?Áë…¸Qì?´záG®{?±ë…¸Qì?±ë…¸Qì?´záG®{?´záG®{?À£× =p¤?· =p£× ?· =p£× ?Õ¸Që…?®¸Që…¸?©™™™™™š¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:15.990580] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:15.990977] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1515}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285947} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:15.991011] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18] rpc statistics(total_trans_msg_count=8, total_batch_msg_count=7) [2024-10-16 14:58:15.991016] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:145) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=5] transaction rpc statistics(total_rt=57889, total_process=1142, avg_rt=50) [2024-10-16 14:58:15.991677] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=60] get number(ret=0, service_type_=0, range=1, base_id=1729061895990775000, start_id=1729061895990775000, end_id=1729061895990775001) [2024-10-16 14:58:15.992037] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:15.992983] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [41376][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061895992915000, start_id=1729061895992915000, end_id=1729061895992915001) [2024-10-16 14:58:15.993276] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C168D-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363861670, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894457, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:15.996918] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93D3-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:15.996939] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000407-00061E9B3B2B93D3-0-0] [lt=22] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=40281, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:15.997605] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=5] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=38, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:15.997617] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=11] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:16.003059] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ef91a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285949} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3169}, prev=this=0x2b0f22eae060 trans_version={val:1729061835486815000, v:0} scn={val:1729061835486815000, v:0} tx_id={txid:392284903} prev=0x2b0f22eadf20 next=(nil) modify_count=1 acc_checksum=2638806984 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3169}, snapshot_version={val:1729061896001371000, v:0}, *this={this=0x2b0f22eadc18 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22eae060 latest_compact_node=(nil) max_trans_version={val:1729061835486815000, v:0} max_trans_id=392284903 max_elr_trans_version={val:1729061835486815000, v:0} max_elr_trans_id=392284903 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835486815000, v:0} min_modify_scn={val:1729061835486815000, v:0}}) [2024-10-16 14:58:16.003091] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ef91a0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285949} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3169}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851233, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851233, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896001371000, v:0}, tx_id:{txid:392285949}, scn:{branch:0, seq:1059}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285949}, tx_desc:{this:0x2b0ec37bbc70, tx_id:{txid:392285949}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896002422, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061895999254, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:3169}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.003149] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285949} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285949}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:3169}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.003164] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830044697972736},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285949} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.003175] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.003184] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044697972736},{"BIGINT":1729058400},{"VARCHAR":"?±ë…¸Qì?®¸Që…¸?®¸Që…¸?®¸Që…¸?©™™™™™š?®¸Që…¸?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?· =p£× ?±ë…¸Qì?´záG®{?®¸Që…¸?©™™™™™š?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?´záG®{?´záG®{?´záG®{?Ç =p£× ?±ë…¸Qì?±ë…¸Qì?®¸Që…¸?®¸Që…¸?©™™™™™š?Íp£× =q?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?· =p£× ?´záG®{?¼(õÂ\)?´záG®{?©™™™™™š?®¸Që…¸?´záG®{?· =p£× ?´záG®{?· =p£× ?´záG®{?Áë…¸Qì?´záG®{?±ë…¸Qì?±ë…¸Qì?´záG®{?´záG®{?À£× =p¤?· =p£× ?· =p£× ?Õ¸Që…?®¸Që…¸?©™™™™™š¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.003240] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.003468] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3172}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285949} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.005860] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061896005683000, start_id=1729061896005683000, end_id=1729061896005683001) [2024-10-16 14:58:16.007413] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f56c648 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285950} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, prev=this=0x2b0f7f4e1d78 trans_version={val:1729061835490020000, v:0} scn={val:1729061835490020000, v:0} tx_id={txid:392284904} prev=0x2b0f7f4e1c38 next=(nil) modify_count=1 acc_checksum=3345589090 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2120}, snapshot_version={val:1729061896006742000, v:0}, *this={this=0x2b0f7f4e1930 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4e1d78 latest_compact_node=(nil) max_trans_version={val:1729061835490020000, v:0} max_trans_id=392284904 max_elr_trans_version={val:1729061835490020000, v:0} max_elr_trans_id=392284904 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835490020000, v:0} min_modify_scn={val:1729061835490020000, v:0}}) [2024-10-16 14:58:16.007450] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=36][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f56c648 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285950} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851383, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851383, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896006742000, v:0}, tx_id:{txid:392285950}, scn:{branch:0, seq:1058}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285950}, tx_desc:{this:0x2b0ec37bdc10, tx_id:{txid:392285950}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896006742, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896005685, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1062}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.007510] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285950} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285950}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1062}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.007528] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":654830044748304384},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285950} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.007543] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.007552] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044748304384},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.007598] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.007844] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2122}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285950} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.011574] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C168F-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363879967, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894458, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.013526] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0e885d3b40 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285952} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1058}, prev=this=0x2b0e8858abb8 trans_version={val:1729061835499676000, v:0} scn={val:1729061835499676000, v:0} tx_id={txid:392284906} prev=0x2b0e8858aa78 next=(nil) modify_count=1 acc_checksum=3774658675 version=1729061779336102 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4254}, snapshot_version={val:1729061896013112000, v:0}, *this={this=0x2b0e8858a770 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e8858abb8 latest_compact_node=(nil) max_trans_version={val:1729061835499676000, v:0} max_trans_id=392284906 max_elr_trans_version={val:1729061835499676000, v:0} max_elr_trans_id=392284906 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835499676000, v:0} min_modify_scn={val:1729061835499676000, v:0}}) [2024-10-16 14:58:16.013568] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=43][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0e885d3b40 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285952} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1058}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851218, tablet_id:{id:209276}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851218, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896013112000, v:0}, tx_id:{txid:392285952}, scn:{branch:0, seq:1055}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285952}, tx_desc:{this:0x2b0ec37d1850, tx_id:{txid:392285952}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896013112, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896012058, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1058}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.013648] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=79][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285952} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285952}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779336102, seq_no:{branch:0, seq:1058}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.013667] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":654830044832190465},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285952} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.013680] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.013685] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044832190465},{"BIGINT":1729058400},{"VARCHAR":"?Ù™™™™™š?ØQë…¸R?Öffffff?ØõÂ\(ö?× =p£× ?ÕÂ\(õÃ?× =p£× ?×®záG®?ØQë…¸R?ØQë…¸R?ÜÌÌÌÌÌÍ?×®záG®?×®záG®?ØõÂ\(ö?Öffffff?Ù™™™™™š?ØQë…¸R?×®záG®?× =p£× ?ØõÂ\(ö?ØQë…¸R?Ú=p£× =?Öffffff?× =p£× ?×®záG®?×®záG®?ÕÂ\(õÃ?ÕÂ\(õÃ?à£× =p¤?× =p£× ?Öffffff?ØõÂ\(ö?×®záG®?Õ¸Që…?ÕÂ\(õÃ?Ù™™™™™š?Û…¸Që…?ÕÂ\(õÃ?à", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.013733] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.013947] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285952} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.016576] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.017548] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061896017330000, start_id=1729061896017330000, end_id=1729061896017330001) [2024-10-16 14:58:16.017988] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1691-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363886382, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894459, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.021696] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1693-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363890090, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894460, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.023142] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=55, replace_time=0, replace_start_pos=220199, replace_num=31457) [2024-10-16 14:58:16.023155] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=55, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:16.024952] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1695-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363893347, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894461, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.028930] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E997D4B9E2D-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061896028513000, start_id=1729061896028513000, end_id=1729061896028513001) [2024-10-16 14:58:16.029905] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A16-0-0] [lt=6] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061896020471, keepalive_ts:-1}) [2024-10-16 14:58:16.029921] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [106960][T1_L0_G1][T1][YB4A0A000408-00061E9B4A5E7A16-0-0] [lt=16] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061896020471, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:16.036020] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1697-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363904414, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894462, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.039351] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [22317][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=16] LogLoopThread round_cost_time(us)(round_cost_time=12) [2024-10-16 14:58:16.039334] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb48f50 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285956} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2244}, prev=this=0x2b0fadaecc28 trans_version={val:1729061835514617000, v:0} scn={val:1729061835514617000, v:0} tx_id={txid:392284910} prev=0x2b0fadaecae8 next=(nil) modify_count=1 acc_checksum=2683265308 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4391}, snapshot_version={val:1729061896038222000, v:0}, *this={this=0x2b0fadaec7e0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaecc28 latest_compact_node=(nil) max_trans_version={val:1729061835514617000, v:0} max_trans_id=392284910 max_elr_trans_version={val:1729061835514617000, v:0} max_elr_trans_id=392284910 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835514617000, v:0} min_modify_scn={val:1729061835514617000, v:0}}) [2024-10-16 14:58:16.039364] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb48f50 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285956} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2244}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851788, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851788, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896038222000, v:0}, tx_id:{txid:392285956}, scn:{branch:0, seq:2241}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285956}, tx_desc:{this:0x2b0ec37c5a90, tx_id:{txid:392285956}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896039296, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896035982, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2244}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.039415] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=51][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285956} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285956}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:2244}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.039429] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654830044962213888},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285956} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.039441] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.039447] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044962213888},{"BIGINT":1729058400},{"VARCHAR":"Aló÷ =qAlÿ€£× Al £× Al¸£× Al"dzáHAl-ú\)Al9¹™™šAlE^¸QìAlQ× >Al\°", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.039490] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.039683] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3315}, to_seq_no={branch:0, seq:2242}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285956} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.044637] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1699-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363913029, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894463, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.047631] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1729061896045929000, start_id=1729061896045929000, end_id=1729061896045929001) [2024-10-16 14:58:16.050705] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C169B-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363919099, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894464, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.052089] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061896051711000, start_id=1729061896051711000, end_id=1729061896051711001) [2024-10-16 14:58:16.052403] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061896052402, bytes: 4446828841932, bw: 0.491297 MB/s, add_ts: 1000770, add_bytes: 515559 [2024-10-16 14:58:16.053652] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb49630 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285958} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1400}, prev=this=0x2b0fadaed328 trans_version={val:1729061835556790000, v:0} scn={val:1729061835556790000, v:0} tx_id={txid:392284912} prev=0x2b0fadaed1e8 next=(nil) modify_count=1 acc_checksum=2361534062 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2313}, snapshot_version={val:1729061896053110000, v:0}, *this={this=0x2b0fadaecee0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaed328 latest_compact_node=(nil) max_trans_version={val:1729061835556790000, v:0} max_trans_id=392284912 max_elr_trans_version={val:1729061835556790000, v:0} max_elr_trans_id=392284912 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835556790000, v:0} min_modify_scn={val:1729061835556790000, v:0}}) [2024-10-16 14:58:16.053690] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb49630 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285958} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1400}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851236, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851236, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896053110000, v:0}, tx_id:{txid:392285958}, scn:{branch:0, seq:1397}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285958}, tx_desc:{this:0x2b0ec37b5d90, tx_id:{txid:392285958}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896053110, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896051714, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1400}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.053741] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=50][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285958} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285958}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1400}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.053755] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830045088043008},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285958} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.053767] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.053772] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045088043008},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.053812] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.055111] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2490}, to_seq_no={branch:0, seq:1398}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285958} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.055641] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD614-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=7, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:16.055652] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD614-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1968, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:16.056653] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C169D-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363925047, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894465, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.059545] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C169F-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363927938, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894466, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.064225] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16A1-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363932619, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894467, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.067844] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16A3-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363936237, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894468, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.068310] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f4871e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285961} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2121}, prev=this=0x2b0f3f41ca70 trans_version={val:1729061835570967000, v:0} scn={val:1729061835570967000, v:0} tx_id={txid:392284915} prev=0x2b0f3f41c930 next=(nil) modify_count=1 acc_checksum=2495797373 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5646}, snapshot_version={val:1729061896067131000, v:0}, *this={this=0x2b0f3f41c628 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f41ca70 latest_compact_node=(nil) max_trans_version={val:1729061835570967000, v:0} max_trans_id=392284915 max_elr_trans_version={val:1729061835570967000, v:0} max_elr_trans_id=392284915 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835570967000, v:0} min_modify_scn={val:1729061835570967000, v:0}}) [2024-10-16 14:58:16.068339] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f4871e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285961} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2121}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851223, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851223, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896067131000, v:0}, tx_id:{txid:392285961}, scn:{branch:0, seq:1062}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285961}, tx_desc:{this:0x2b0ec37b3df0, tx_id:{txid:392285961}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896068190, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896066070, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2121}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.068398] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285961} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285961}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:2121}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.068416] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830045176123392},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285961} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.068430] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.068436] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045176123392},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.068485] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.068845] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2124}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285961} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.072714] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5407E-0-0] [lt=4] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:16.072729] INFO [LIB] stat (utility.h:1165) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD5407E-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=66088, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:16.072872] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:16.074513] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061896073752000, start_id=1729061896073752000, end_id=1729061896073752001) [2024-10-16 14:58:16.074894] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb49d10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285963} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1133}, prev=this=0x2b0fadaeda28 trans_version={val:1729061835577335000, v:0} scn={val:1729061835577335000, v:0} tx_id={txid:392284917} prev=0x2b0fadaed8e8 next=(nil) modify_count=1 acc_checksum=3699211858 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4213}, snapshot_version={val:1729061896073752000, v:0}, *this={this=0x2b0fadaed5e0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaeda28 latest_compact_node=(nil) max_trans_version={val:1729061835577335000, v:0} max_trans_id=392284917 max_elr_trans_version={val:1729061835577335000, v:0} max_elr_trans_id=392284917 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835577335000, v:0} min_modify_scn={val:1729061835577335000, v:0}}) [2024-10-16 14:58:16.074934] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=40][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb49d10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285963} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1133}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851250, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851250, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896073752000, v:0}, tx_id:{txid:392285963}, scn:{branch:0, seq:1130}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285963}, tx_desc:{this:0x2b0ec37c1b50, tx_id:{txid:392285963}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896074852, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896072623, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1133}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.075004] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=69][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285963} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285963}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1133}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.075022] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830045402615808},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285963} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.075037] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.075048] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045402615808},{"BIGINT":1729058400},{"VARCHAR":"AAoŸÌÌÌÍAAo§J=p¤AAo®šáG®AAoµà", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.075098] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.075430] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2230}, to_seq_no={branch:0, seq:1131}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285963} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.078139] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16A4-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363946532, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894469, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.078309] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1729061896076966000, start_id=1729061896076966000, end_id=1729061896076966001) [2024-10-16 14:58:16.078326] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:16.078344] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1729061896078341], gts:1729061896076966000, latest_srr:[mts=1729061896078341]}) [2024-10-16 14:58:16.085629] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16A6-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363954021, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894470, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.086933] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D5679-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=12454) [2024-10-16 14:58:16.087240] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4713-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=4, avg_log_gen_to_submit_time=38, avg_log_submit_to_flush_time=480, avg_log_submit_to_slide_time=997) [2024-10-16 14:58:16.089294] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16A8-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363957687, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894471, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.091556] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f5b939888 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285967} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778513125 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2181}, prev=this=0x2b0f5b8e3070 trans_version={val:1729061835594388000, v:0} scn={val:1729061835594388000, v:0} tx_id={txid:392284921} prev=0x2b0f5b8e2f30 next=(nil) modify_count=1 acc_checksum=3176230487 version=1729061778513125 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7516}, snapshot_version={val:1729061896091159000, v:0}, *this={this=0x2b0f5b8e2c28 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f5b8e3070 latest_compact_node=(nil) max_trans_version={val:1729061835594388000, v:0} max_trans_id=392284921 max_elr_trans_version={val:1729061835594388000, v:0} max_elr_trans_id=392284921 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835594388000, v:0} min_modify_scn={val:1729061835594388000, v:0}}) [2024-10-16 14:58:16.091591] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f5b939888 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285967} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778513125 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2181}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851220, tablet_id:{id:209288}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851220, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896091159000, v:0}, tx_id:{txid:392285967}, scn:{branch:0, seq:2178}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285967}, tx_desc:{this:0x2b0ec37bbc70, tx_id:{txid:392285967}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896091159, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896088982, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2181}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.091655] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=63][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285967} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285967}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778513125, seq_no:{branch:0, seq:2181}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.091670] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830045536833537},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285967} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.091681] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.091691] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045536833537},{"BIGINT":1729058400},{"VARCHAR":"BŸ¹òµ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.091744] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.092111] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.093543] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16AA-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363961937, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894472, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.093851] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4301}, to_seq_no={branch:0, seq:2179}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285967} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.095888] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061896095402000, start_id=1729061896095402000, end_id=1729061896095402001) [2024-10-16 14:58:16.097185] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22797][T1004_Flush][T1004][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:58:16.097360] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4C-0-0] [lt=12] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1729015308598471003, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1729015308598471004, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1729015308598471005, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1729015308598471006, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1729015308598471007, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1729015308598471008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1729015308598471009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1729015308598471010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1729015308598471011, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1729015308598471012, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1729015308598471013, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1729015308598471014, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1729015308598471015, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1729015308598471016, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1729015308598471018, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1729015308598471019, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1729015308598471020, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1729015308598471021, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1729015308598471022, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1729015308598471023, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1729015308598471024, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1729015308598471025, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1729015308598471026, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1729015308598471027, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1729015308598471028, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1729015308598471029, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1729015308598471030, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1729015308598471031, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1729015308598471032, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1729015308598471033, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1729015308598471034, v:0}}, {tablet_id:{id:103}, rec_scn:{val:1729015308598471035, v:0}}, }, max_consequent_callbacked_scn={val:1729061895576296004, v:0}, *this={this:0x2b0e5ae96510, is_inited:true, freezing_scn:{val:1722494667423836000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015206603552000, v:0}, clog_base_lsn:{lsn:17380134912}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061893074505004, v:0}, tablet_change_checkpoint_scn:{val:1728976273502119000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061910721265000, latest_log_ts:{val:1729061894075060001, v:0}}, {limited_id:60000001, latest_log_ts:{val:1728944817644426002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:16.097533] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4C-0-0] [lt=87] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200589}, rec_scn:{val:1729015311100270003, v:0}}, {tablet_id:{id:200529}, rec_scn:{val:1729015311100270004, v:0}}, {tablet_id:{id:200197}, rec_scn:{val:1729015311100270005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1729015311100270006, v:0}}, {tablet_id:{id:200325}, rec_scn:{val:1729015311100270007, v:0}}, {tablet_id:{id:200880}, rec_scn:{val:1729015311100270009, v:0}}, {tablet_id:{id:200913}, rec_scn:{val:1729015311100270010, v:0}}, {tablet_id:{id:200193}, rec_scn:{val:1729015311100270011, v:0}}, {tablet_id:{id:200301}, rec_scn:{val:1729015311100270012, v:0}}, {tablet_id:{id:200379}, rec_scn:{val:1729015311100270013, v:0}}, {tablet_id:{id:200808}, rec_scn:{val:1729015311100270014, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1729015311100270015, v:0}}, {tablet_id:{id:1152921504606847168}, rec_scn:{val:1729015311100270016, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1729015311100270017, v:0}}, {tablet_id:{id:200304}, rec_scn:{val:1729015311100270018, v:0}}, {tablet_id:{id:200355}, rec_scn:{val:1729015311100270019, v:0}}, {tablet_id:{id:200739}, rec_scn:{val:1729015311100270021, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1729015311100270022, v:0}}, {tablet_id:{id:200799}, rec_scn:{val:1729015311100270023, v:0}}, {tablet_id:{id:200691}, rec_scn:{val:1729015311100270024, v:0}}, {tablet_id:{id:200433}, rec_scn:{val:1729015311100270025, v:0}}, {tablet_id:{id:200706}, rec_scn:{val:1729015311100270026, v:0}}, {tablet_id:{id:200364}, rec_scn:{val:1729015311100270027, v:0}}, {tablet_id:{id:200533}, rec_scn:{val:1729015311100270028, v:0}}, {tablet_id:{id:200163}, rec_scn:{val:1729015311100270029, v:0}}, {tablet_id:{id:1152921504606847081}, rec_scn:{val:1729015311100270030, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1729015311100270031, v:0}}, {tablet_id:{id:200109}, rec_scn:{val:1729015311100270032, v:0}}, {tablet_id:{id:200310}, rec_scn:{val:1729015311100270033, v:0}}, {tablet_id:{id:200700}, rec_scn:{val:1729015311100270034, v:0}}, {tablet_id:{id:200391}, rec_scn:{val:1729015311100270035, v:0}}, {tablet_id:{id:200951}, rec_scn:{val:1729015311100270036, v:0}}, }, max_consequent_callbacked_scn={val:1729061895576296004, v:0}, *this={this:0x2b0e5af0c510, is_inited:true, freezing_scn:{val:1724954401274529001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015311100270003, v:0}, clog_base_lsn:{lsn:13555163136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061893074505004, v:0}, tablet_change_checkpoint_scn:{val:1728976278507840000, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725269661658771002, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:17, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:16.097682] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4C-0-0] [lt=74] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1002}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200929}, rec_scn:{val:1729015305082700002, v:0}}, {tablet_id:{id:1152921504606847306}, rec_scn:{val:1729015305082700003, v:0}}, {tablet_id:{id:200608}, rec_scn:{val:1729015305082700004, v:0}}, {tablet_id:{id:200767}, rec_scn:{val:1729015305082700005, v:0}}, {tablet_id:{id:200347}, rec_scn:{val:1729015305082700006, v:0}}, {tablet_id:{id:1152921504606846994}, rec_scn:{val:1729015305082700007, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1729015305082700008, v:0}}, {tablet_id:{id:200566}, rec_scn:{val:1729015305082700009, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1729015305082700010, v:0}}, {tablet_id:{id:1152921504606847198}, rec_scn:{val:1729015305082700011, v:0}}, {tablet_id:{id:1152921504606847084}, rec_scn:{val:1729015305082700012, v:0}}, {tablet_id:{id:200911}, rec_scn:{val:1729015305082700013, v:0}}, {tablet_id:{id:200107}, rec_scn:{val:1729015305082700014, v:0}}, {tablet_id:{id:1152921504606847359}, rec_scn:{val:1729015305082700015, v:0}}, {tablet_id:{id:200707}, rec_scn:{val:1729015305082700016, v:0}}, {tablet_id:{id:200314}, rec_scn:{val:1729015305082700017, v:0}}, {tablet_id:{id:200662}, rec_scn:{val:1729015305082700018, v:0}}, {tablet_id:{id:200275}, rec_scn:{val:1729015305082700019, v:0}}, {tablet_id:{id:200725}, rec_scn:{val:1729015305082700020, v:0}}, {tablet_id:{id:200332}, rec_scn:{val:1729015305082700021, v:0}}, {tablet_id:{id:1152921504606847312}, rec_scn:{val:1729015305082700022, v:0}}, {tablet_id:{id:200206}, rec_scn:{val:1729015305082700023, v:0}}, {tablet_id:{id:1152921504606847330}, rec_scn:{val:1729015305082700024, v:0}}, {tablet_id:{id:200746}, rec_scn:{val:1729015305082700025, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1729015305082700026, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1729015305082700027, v:0}}, {tablet_id:{id:200764}, rec_scn:{val:1729015305082700028, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1729015305082700029, v:0}}, {tablet_id:{id:1152921504606847093}, rec_scn:{val:1729015305082700030, v:0}}, {tablet_id:{id:200950}, rec_scn:{val:1729015305082700031, v:0}}, {tablet_id:{id:200731}, rec_scn:{val:1729015305082700032, v:0}}, {tablet_id:{id:200914}, rec_scn:{val:1729015305082700033, v:0}}, }, max_consequent_callbacked_scn={val:1729061895673141004, v:0}, *this={this:0x2b0e5af7e510, is_inited:true, freezing_scn:{val:1728969824079193000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1002}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015305082700002, v:0}, clog_base_lsn:{lsn:14427525120}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061893074505004, v:0}, tablet_change_checkpoint_scn:{val:1728976283531354002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1725017521048842001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:23, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1002}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:16.097831] INFO [MDS] flush (mds_table_mgr.cpp:195) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4C-0-0] [lt=69] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1003}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200426}, rec_scn:{val:1729015265583755004, v:0}}, {tablet_id:{id:200810}, rec_scn:{val:1729015265583755005, v:0}}, {tablet_id:{id:1152921504606847145}, rec_scn:{val:1729015265583755006, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1729015265583755007, v:0}}, {tablet_id:{id:200822}, rec_scn:{val:1729015265583755008, v:0}}, {tablet_id:{id:1152921504606847170}, rec_scn:{val:1729015265583755009, v:0}}, {tablet_id:{id:1152921504606847260}, rec_scn:{val:1729015265583755010, v:0}}, {tablet_id:{id:200312}, rec_scn:{val:1729015265583755011, v:0}}, {tablet_id:{id:200690}, rec_scn:{val:1729015265583755012, v:0}}, {tablet_id:{id:200703}, rec_scn:{val:1729015265583755014, v:0}}, {tablet_id:{id:200678}, rec_scn:{val:1729015265583755015, v:0}}, {tablet_id:{id:200300}, rec_scn:{val:1729015265583755016, v:0}}, {tablet_id:{id:1152921504606846997}, rec_scn:{val:1729015265583755017, v:0}}, {tablet_id:{id:200867}, rec_scn:{val:1729015265583755018, v:0}}, {tablet_id:{id:200732}, rec_scn:{val:1729015265583755019, v:0}}, {tablet_id:{id:1152921504606847199}, rec_scn:{val:1729015265583755020, v:0}}, {tablet_id:{id:1152921504606847178}, rec_scn:{val:1729015265583755021, v:0}}, {tablet_id:{id:200747}, rec_scn:{val:1729015265583755022, v:0}}, {tablet_id:{id:1152921504606847267}, rec_scn:{val:1729015265583755023, v:0}}, {tablet_id:{id:200183}, rec_scn:{val:1729015265583755024, v:0}}, {tablet_id:{id:1152921504606847155}, rec_scn:{val:1729015265583755025, v:0}}, {tablet_id:{id:1152921504606847345}, rec_scn:{val:1729015265583755026, v:0}}, {tablet_id:{id:200882}, rec_scn:{val:1729015265583755027, v:0}}, {tablet_id:{id:200330}, rec_scn:{val:1729015265583755028, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1729015265583755029, v:0}}, {tablet_id:{id:1152921504606847363}, rec_scn:{val:1729015265583755030, v:0}}, {tablet_id:{id:200918}, rec_scn:{val:1729015265583755031, v:0}}, {tablet_id:{id:200387}, rec_scn:{val:1729015265583755032, v:0}}, {tablet_id:{id:1152921504606847317}, rec_scn:{val:1729015265583755033, v:0}}, {tablet_id:{id:200579}, rec_scn:{val:1729015265583755034, v:0}}, {tablet_id:{id:1152921504606847362}, rec_scn:{val:1729015265583755035, v:0}}, {tablet_id:{id:1152921504606847289}, rec_scn:{val:1729015265583755036, v:0}}, }, max_consequent_callbacked_scn={val:1729061895678607003, v:0}, *this={this:0x2b0e5b004510, is_inited:true, freezing_scn:{val:1728969834630392000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1003}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015265583755004, v:0}, clog_base_lsn:{lsn:14293315584}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1729061893074505004, v:0}, tablet_change_checkpoint_scn:{val:1728976283486973002, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1722847258634015001, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:2, proposal_id:16, palf_env_:0x2b0e0f9b4030, is_in_stop_state_:false, is_inited_:true, id_:1003}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:16.097902] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22797][T1004_Flush][T1004][YB4A0A000407-00061E9B37998D4C-0-0] [lt=69] succeed to traversal_flush(ret=0, ls_cnt=4) [2024-10-16 14:58:16.099228] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=22][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f56cd28 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285968} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2399}, prev=this=0x2b0f7f4e2478 trans_version={val:1729061835597537000, v:0} scn={val:1729061835597537000, v:0} tx_id={txid:392284922} prev=0x2b0f7f4e2338 next=(nil) modify_count=1 acc_checksum=3568375954 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2109}, snapshot_version={val:1729061896097569000, v:0}, *this={this=0x2b0f7f4e2030 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4e2478 latest_compact_node=(nil) max_trans_version={val:1729061835597537000, v:0} max_trans_id=392284922 max_elr_trans_version={val:1729061835597537000, v:0} max_elr_trans_id=392284922 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835597537000, v:0} min_modify_scn={val:1729061835597537000, v:0}}) [2024-10-16 14:58:16.099262] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f56cd28 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285968} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2399}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851222, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851222, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896097569000, v:0}, tx_id:{txid:392285968}, scn:{branch:0, seq:1057}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285968}, tx_desc:{this:0x2b0ec37cd910, tx_id:{txid:392285968}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896098911, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896096513, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2399}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.099318] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285968} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285968}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:2399}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.099336] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830045566193664},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285968} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.099350] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.099356] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045566193664},{"BIGINT":1729058400},{"VARCHAR":"BŸ¹õh", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.099401] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.099636] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2402}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285968} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.103731] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16AC-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363972124, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894473, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.111495] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=5] re_register_parent reason(palf_id=1002, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061892107637, last_first_register_time_us=1722494150998907, register_time_us=1729061892107637, curr_time_us=1729061896111494) [2024-10-16 14:58:16.111531] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=18] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890", reason=3, leader="10.0.4.8:2890", register_time_us=1729061896111515, last_submit_register_req_time_us=1729061896111515, register_parent_reason=3) [2024-10-16 14:58:16.111541] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=9] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1002, self="10.0.4.7:2890") [2024-10-16 14:58:16.112855] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb4a3f0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285971} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2108}, prev=this=0x2b0fadaee128 trans_version={val:1729061835612332000, v:0} scn={val:1729061835612332000, v:0} tx_id={txid:392284925} prev=0x2b0fadaedfe8 next=(nil) modify_count=1 acc_checksum=3431027656 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4274}, snapshot_version={val:1729061896111672000, v:0}, *this={this=0x2b0fadaedce0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaee128 latest_compact_node=(nil) max_trans_version={val:1729061835612332000, v:0} max_trans_id=392284925 max_elr_trans_version={val:1729061835612332000, v:0} max_elr_trans_id=392284925 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835612332000, v:0} min_modify_scn={val:1729061835612332000, v:0}}) [2024-10-16 14:58:16.112890] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb4a3f0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285971} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2108}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851241, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851241, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896111672000, v:0}, tx_id:{txid:392285971}, scn:{branch:0, seq:2105}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285971}, tx_desc:{this:0x2b0ec37b1e50, tx_id:{txid:392285971}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896112731, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896109568, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2108}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.112947] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285971} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285971}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:2108}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.112970] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21][errcode=0] primary key duplication conflict(key={"BIGINT":654830045654274048},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285971} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.112984] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.112990] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045654274048},{"BIGINT":1729058400},{"VARCHAR":"Aô_9XbAô_r° ÅAô_‡+ Aô_ãS÷ÏAôb¸QìAôixÔýôAôp‡+AôvÐå`BAôx¡ÊÀƒAôz/Ÿ¿Aô|", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.113037] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.113641] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3164}, to_seq_no={branch:0, seq:2106}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285971} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.114319] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16AE-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363982713, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894474, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.116507] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16B0-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363984902, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894475, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.116651] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.119959] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16B2-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510363988353, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894476, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.121152] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be44330 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285973} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2111}, prev=this=0x2b0ec6daf900 trans_version={val:1729061835624107000, v:0} scn={val:1729061835624107000, v:0} tx_id={txid:392284927} prev=0x2b0ec6daf7c0 next=(nil) modify_count=1 acc_checksum=345024750 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4299}, snapshot_version={val:1729061896119790001, v:0}, *this={this=0x2b0ec6daf4b8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6daf900 latest_compact_node=(nil) max_trans_version={val:1729061835624107000, v:0} max_trans_id=392284927 max_elr_trans_version={val:1729061835624107000, v:0} max_elr_trans_id=392284927 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835624107000, v:0} min_modify_scn={val:1729061835624107000, v:0}}) [2024-10-16 14:58:16.121182] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be44330 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285973} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2111}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851233, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851233, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896119790001, v:0}, tx_id:{txid:392285973}, scn:{branch:0, seq:1082}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285973}, tx_desc:{this:0x2b0ec37d1850, tx_id:{txid:392285973}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896120821, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896118711, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2111}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.121246] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=63][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285973} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285973}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:2111}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.121268] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21][errcode=0] primary key duplication conflict(key={"BIGINT":654830045729771520},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285973} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.121280] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.121285] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830045729771520},{"BIGINT":1729058400},{"VARCHAR":"A°«Ó+", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.121332] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.122412] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3171}, to_seq_no={branch:0, seq:1083}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285973} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.123900] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061896122941000, start_id=1729061896122941000, end_id=1729061896122941001) [2024-10-16 14:58:16.125062] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1185906688, global_cache_size=1579302912, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:16.128930] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:16.128947] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:16.139073] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5575, mem_used=10398720) [2024-10-16 14:58:16.140442] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:638) [21543][T1_CdcSrv][T1][Y0-0000000000000000-0-0] [lt=12] finish to resize log external storage handler(current_ts=1729061896140439, tenant_max_cpu=3, valid_ls_count=0, other_ls_count=0, new_concurrency=0) [2024-10-16 14:58:16.149398] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16B4-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364017792, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894477, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.150674] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:58:16.153029] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16B6-0-0] [lt=19][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364021421, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894478, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.154947] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:16.154963] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:16.155123] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:16.155621] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107618][][T1004][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1004, ret=0, thread_count_=192) [2024-10-16 14:58:16.155639] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107618][][T1004][Y0-0000000000000000-0-0] [lt=17] Init thread local success [2024-10-16 14:58:16.155961] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=9] worker thread created(tenant_->id()=1004, group_id_=1, token=1) [2024-10-16 14:58:16.156210] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ef9880 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285982} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2178}, prev=this=0x2b0f22eae760 trans_version={val:1729061835669736000, v:0} scn={val:1729061835669736000, v:0} tx_id={txid:392284936} prev=0x2b0f22eae620 next=(nil) modify_count=1 acc_checksum=2217499714 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3193}, snapshot_version={val:1729061896155655000, v:0}, *this={this=0x2b0f22eae318 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22eae760 latest_compact_node=(nil) max_trans_version={val:1729061835669736000, v:0} max_trans_id=392284936 max_elr_trans_version={val:1729061835669736000, v:0} max_elr_trans_id=392284936 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835669736000, v:0} min_modify_scn={val:1729061835669736000, v:0}}) [2024-10-16 14:58:16.156241] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ef9880 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285982} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2178}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851229, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851229, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896155655000, v:0}, tx_id:{txid:392285982}, scn:{branch:0, seq:2175}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285982}, tx_desc:{this:0x2b0ec37b7d30, tx_id:{txid:392285982}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896155655, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896153481, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2178}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.156304] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285982} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285982}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:2178}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.156324] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19][errcode=0] primary key duplication conflict(key={"BIGINT":654830046145007616},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285982} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.156340] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.156346] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046145007616},{"BIGINT":1729058400},{"VARCHAR":"B¯jnè", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.156408] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.156677] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2181}, to_seq_no={branch:0, seq:2176}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285982} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.156924] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=34] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:16.163025] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=5] current monitor number(seq_=-1) [2024-10-16 14:58:16.165724] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16B8-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364034117, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894479, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.170789] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=8] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2024-10-16 14:58:16.170807] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2024-10-16 14:58:16.170814] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2024-10-16 14:58:16.170817] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0) [2024-10-16 14:58:16.170827] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0) [2024-10-16 14:58:16.170831] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:240) [22239][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=3] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=16732) [2024-10-16 14:58:16.172126] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16BA-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364040521, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894480, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.173141] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0DAE-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061896172130000, start_id=1729061896172130000, end_id=1729061896172130001) [2024-10-16 14:58:16.173171] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0DB2-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061896172130000, start_id=1729061896172130000, end_id=1729061896172130001) [2024-10-16 14:58:16.173241] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0DB1-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061896172130000, start_id=1729061896172130000, end_id=1729061896172130001) [2024-10-16 14:58:16.173532] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107618][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD619-0-0] [lt=5] init point thread id with(&point=0x55e3435cc700, idx_=1122, point=[thread id=107618, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107618) [2024-10-16 14:58:16.173566] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061896172130000, start_id=1729061896172130001, end_id=1729061896172130002) [2024-10-16 14:58:16.173637] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=25, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:16.173652] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:16.173712] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0DAF-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061896173619000, start_id=1729061896173619000, end_id=1729061896173619001) [2024-10-16 14:58:16.173748] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54080-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:16.173897] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8F-0-0] [lt=23] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:16.173923] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D8F-0-0] [lt=16] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061895673141004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061895673141004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:16.174890] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16BC-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364043285, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894481, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.176503] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061896176019000, start_id=1729061896176019000, end_id=1729061896176019001) [2024-10-16 14:58:16.178240] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D8F-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1729061896178086000, start_id=1729061896178086000, end_id=1729061896178086001) [2024-10-16 14:58:16.180924] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16BE-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364049318, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894482, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.181546] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22ef9f60 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285986} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2128}, prev=this=0x2b0f22eaee60 trans_version={val:1729061835687704000, v:0} scn={val:1729061835687704000, v:0} tx_id={txid:392284940} prev=0x2b0f22eaed20 next=(nil) modify_count=1 acc_checksum=3550334457 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4179}, snapshot_version={val:1729061896180210001, v:0}, *this={this=0x2b0f22eaea18 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22eaee60 latest_compact_node=(nil) max_trans_version={val:1729061835687704000, v:0} max_trans_id=392284940 max_elr_trans_version={val:1729061835687704000, v:0} max_elr_trans_id=392284940 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835687704000, v:0} min_modify_scn={val:1729061835687704000, v:0}}) [2024-10-16 14:58:16.181579] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22ef9f60 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285986} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2128}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851219, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851219, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896180210001, v:0}, tx_id:{txid:392285986}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285986}, tx_desc:{this:0x2b0ec37a2150, tx_id:{txid:392285986}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896181275, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896179148, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:2128}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.181645] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=65][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285986} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285986}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:2128}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.181659] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":674740107403186176},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285986} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.181673] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.181679] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":674740107403186176},{"BIGINT":1729058400},{"VARCHAR":"B;›°", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.181726] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.181962] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2131}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285986} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.184615] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f4878c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285987} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779396338 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1038}, prev=this=0x2b0f3f41d170 trans_version={val:1729061835694034000, v:0} scn={val:1729061835694034000, v:0} tx_id={txid:392284941} prev=0x2b0f3f41d030 next=(nil) modify_count=1 acc_checksum=558076675 version=1729061779396338 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5276}, snapshot_version={val:1729061896183418001, v:0}, *this={this=0x2b0f3f41cd28 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f41d170 latest_compact_node=(nil) max_trans_version={val:1729061835694034000, v:0} max_trans_id=392284941 max_elr_trans_version={val:1729061835694034000, v:0} max_elr_trans_id=392284941 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835694034000, v:0} min_modify_scn={val:1729061835694034000, v:0}}) [2024-10-16 14:58:16.184646] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f4878c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285987} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779396338 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1038}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851212, tablet_id:{id:209282}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851212, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896183418001, v:0}, tx_id:{txid:392285987}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285987}, tx_desc:{this:0x2b0ec37abf70, tx_id:{txid:392285987}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896184455, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896183418, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1038}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.184700] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285987} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285987}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779396338, seq_no:{branch:0, seq:1038}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.184717] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":680985887415848961},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285987} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.184731] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.184736] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":680985887415848961},{"BIGINT":1729058400},{"VARCHAR":"B6,", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.184776] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.185053] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1041}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285987} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.191715] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061896190843000, start_id=1729061896190843000, end_id=1729061896190843001) [2024-10-16 14:58:16.192177] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.197017] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE40276-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1002, self="10.0.4.7:2890", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=430, avg_log_submit_to_slide_time=36554) [2024-10-16 14:58:16.197040] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:16.200465] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC96-0-0] [lt=9] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1002, self_addr_:"10.0.4.9:2890", ls_array_:[{id:1}, {id:1001}]}) [2024-10-16 14:58:16.200492] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC96-0-0] [lt=22] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:246, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:16.200510] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107454][T1002_L0_G19][T1002][YB4A0A000409-00061E9A33F4DC96-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.9:2890", this={role:1, proposal_id:130, palf_env_:0x2b0e0d9b4030, is_in_stop_state_:false, is_inited_:true, id_:1001}) [2024-10-16 14:58:16.211146] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16C0-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364079539, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894483, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.213373] INFO [CLOG] is_in_sync (ob_log_handler.cpp:569) [21661][T1_L0_G0][T1][YB4A0A000408-00061E9B592E5885-0-0] [lt=2] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2024-10-16 14:58:16.213482] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21661][T1_L0_G0][T1][YB4A0A000408-00061E9B592E5885-0-0] [lt=19][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:16.214235] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504E-0-0] [lt=6] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:16.214252] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504E-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:16.216814] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.218373] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16C2-0-0] [lt=26][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364086766, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894484, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.224652] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=34, replace_time=0, replace_start_pos=251656, replace_num=31457) [2024-10-16 14:58:16.224676] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] replace map num details(ret=0, replace_node_count=34, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:16.226245] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f487fa0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285992} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3019}, prev=this=0x2b0f3f41d870 trans_version={val:1729061835715402000, v:0} scn={val:1729061835715402000, v:0} tx_id={txid:392284946} prev=0x2b0f3f41d730 next=(nil) modify_count=1 acc_checksum=2571855236 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3403}, snapshot_version={val:1729061896224818000, v:0}, *this={this=0x2b0f3f41d428 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f41d870 latest_compact_node=(nil) max_trans_version={val:1729061835715402000, v:0} max_trans_id=392284946 max_elr_trans_version={val:1729061835715402000, v:0} max_elr_trans_id=392284946 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835715402000, v:0} min_modify_scn={val:1729061835715402000, v:0}}) [2024-10-16 14:58:16.226278] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f487fa0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285992} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3019}, ctx={this:0x2b0e2b854080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851241, tablet_id:{id:209273}, table_iter:0x2b0e2b853210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851241, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896224818000, v:0}, tx_id:{txid:392285992}, scn:{branch:0, seq:1954}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285992}, tx_desc:{this:0x2b0e0fccfeb0, tx_id:{txid:392285992}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896225883, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896222865, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:3019}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.226340] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285992} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285992}, data:0x2b0e2b850438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:3019}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.226358] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830046623158272},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285992} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.226373] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.226379] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046623158272},{"BIGINT":1729058400},{"VARCHAR":"BHö€Ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.226432] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.226688] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3022}, to_seq_no={branch:0, seq:1955}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285992} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.228416] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061896228008000, start_id=1729061896228008000, end_id=1729061896228008001) [2024-10-16 14:58:16.240430] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16C4-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364108823, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894485, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.242784] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be44d18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285996} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1080}, prev=this=0x2b0ec6db03d8 trans_version={val:1729061835735492000, v:0} scn={val:1729061835735492000, v:0} tx_id={txid:392284950} prev=0x2b0ec6db0298 next=(nil) modify_count=1 acc_checksum=3604581540 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5469}, snapshot_version={val:1729061896241560000, v:0}, *this={this=0x2b0ec6daff90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6db03d8 latest_compact_node=(nil) max_trans_version={val:1729061835735492000, v:0} max_trans_id=392284950 max_elr_trans_version={val:1729061835735492000, v:0} max_elr_trans_id=392284950 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835735492000, v:0} min_modify_scn={val:1729061835735492000, v:0}}) [2024-10-16 14:58:16.242816] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be44d18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285996} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1080}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851356, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851356, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896241560000, v:0}, tx_id:{txid:392285996}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285996}, tx_desc:{this:0x2b0ec37bbc70, tx_id:{txid:392285996}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896242639, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896241560, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1080}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.242875] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285996} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285996}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:1080}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.242893] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830046799319040},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285996} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.242907] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.242915] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830046799319040},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.242963] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.243164] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1083}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285996} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.247211] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1002, min_version={val:1729061895576296005, v:0}, base_version={val:1729061895576296004, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061895576296005, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061896247197, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061896012055000, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061896192131, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061896025004000, v:0}, valid_part_count:4, total_part_count:4, generate_tstamp:1729061896226527, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:16.247978] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061896247028000, start_id=1729061896247028000, end_id=1729061896247028001) [2024-10-16 14:58:16.248048] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=26, this=0x2b0df99e3470, extra_info="") [2024-10-16 14:58:16.248062] INFO [LIB] stat (utility.h:1165) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e076d1210, extra_info="id:1") [2024-10-16 14:58:16.248069] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=8] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002) [2024-10-16 14:58:16.248087] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=15] start lib cache evict(tenant_id=1002, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:58:16.248093] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=47341568, mem_limit=53687090, cache_obj_num=86, cache_node_num=86) [2024-10-16 14:58:16.248098] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:16.249158] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000409-00061E9A24F6AD16-0-0] [lt=10][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:16.250341] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=86, alloc_obj_list=[{obj_id:77140741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:77140743, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:76946792, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175180}, {obj_id:76946793, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:78280091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:78088005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175721}, {obj_id:78088017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:78088019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:78088031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:78735443, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181162}, {obj_id:78735464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:78735469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175632}, {obj_id:78735471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:78735472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180965}, {obj_id:78735475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:78735476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220648}, {obj_id:78735477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227404}, {obj_id:78735478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198790}, {obj_id:78735479, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:314589}, {obj_id:78735480, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316654}, {obj_id:78735481, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269043}, {obj_id:78735483, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191615}, {obj_id:75835279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:78735823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239615}, {obj_id:78195282, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:387855}, {obj_id:78195297, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:78195306, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227324}, {obj_id:78736136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183614}, {obj_id:78736281, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:78736288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:573850}, {obj_id:78736317, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:78736326, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161461}, {obj_id:78736327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180286}, {obj_id:78736332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192154}, {obj_id:78737914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78737916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:78741090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:78200949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:78200962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:78200971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:78200972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:78200973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:78891514, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:78891543, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:78891843, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182949}, {obj_id:78891844, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183572}, {obj_id:78891874, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:78891879, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175779}, {obj_id:78891884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:78891978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180711}, {obj_id:78892002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:78892003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:78892005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174480}, {obj_id:78892041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:78892076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:78892159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:78892163, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194733}, {obj_id:78892165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:78892166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:78892167, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:78892168, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:78892169, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:78892170, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:78892221, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:78892223, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:78892225, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:78892304, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196395}, {obj_id:78892316, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183433}, {obj_id:78892318, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:78892319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:78892320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:78892321, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:78892322, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156439}, {obj_id:78892323, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247748}, {obj_id:78892324, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200156}, {obj_id:78893115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:756788}, {obj_id:78893117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:611355}, {obj_id:78893118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:655880}, {obj_id:78893119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893120, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:78893121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480968}, {obj_id:78893122, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165552}, {obj_id:78893917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:78893918, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893919, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:78893920, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}]) [2024-10-16 14:58:16.250453] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22079][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=110] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:16.252078] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16C6-0-0] [lt=25][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364120472, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894486, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.252392] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061896250577000, start_id=1729061896250577000, end_id=1729061896250577001) [2024-10-16 14:58:16.254582] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16C8-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364122977, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894487, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.254994] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e431d5470, extra_info="") [2024-10-16 14:58:16.255011] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5aea3210, extra_info="id:1") [2024-10-16 14:58:16.256272] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16CA-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364124667, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894488, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.256403] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB APPEND COST TIME](cur_stat_count=12, stat_interval=5000000, avg (count/cost)=47, this=0x2b0e5aeee280, extra_info="") [2024-10-16 14:58:16.256413] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=12, stat_interval=5000000, avg (count/cost)=842, this=0x2b0e5aeee4b0, extra_info="") [2024-10-16 14:58:16.256417] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=12, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e5aeee6e0, extra_info="") [2024-10-16 14:58:16.256421] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=12, stat_interval=5000000, avg (count/cost)=-562, this=0x2b0e5aeee910, extra_info="") [2024-10-16 14:58:16.256424] INFO [LIB] stat (utility.h:1165) [22711][T1004_ApplySrv0][T1004][YB4A0A000407-00061E9A3D496BE4-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=12, stat_interval=5000000, avg (count/cost)=327, this=0x2b0e5aeeeb40, extra_info="") [2024-10-16 14:58:16.257939] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=14] run rewrite rule refresh task(rule_mgr_->tenant_id_=1001) [2024-10-16 14:58:16.258728] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21806][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061896258099000, start_id=1729061896258099000, end_id=1729061896258099001) [2024-10-16 14:58:16.259242] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be453f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285999} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0ec6db0ad8 trans_version={val:1729061835756480000, v:0} scn={val:1729061835756480000, v:0} tx_id={txid:392284953} prev=0x2b0ec6db0998 next=(nil) modify_count=1 acc_checksum=1645265093 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6330}, snapshot_version={val:1729061896258099000, v:0}, *this={this=0x2b0ec6db0690 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6db0ad8 latest_compact_node=(nil) max_trans_version={val:1729061835756480000, v:0} max_trans_id=392284953 max_elr_trans_version={val:1729061835756480000, v:0} max_elr_trans_id=392284953 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835756480000, v:0} min_modify_scn={val:1729061835756480000, v:0}}) [2024-10-16 14:58:16.259275] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be453f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392285999} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851227, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851227, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896258099000, v:0}, tx_id:{txid:392285999}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392285999}, tx_desc:{this:0x2b0ec37bdc10, tx_id:{txid:392285999}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896259158, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896258099, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.259357] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=82][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285999} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392285999}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.259375] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830047017422850},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392285999} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.259389] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.259395] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047017422850},{"BIGINT":1729058400},{"VARCHAR":"BMå€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.259442] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.259650] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392285999} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.264979] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16CC-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364133368, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894489, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.272520] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16CE-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364140913, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894490, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.274042] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16D0-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364142437, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894491, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.274125] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54082-0-0] [lt=18][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173496, log_task={header:{begin_lsn:{lsn:14352416604}, end_lsn:{lsn:14352416726}, log_id:115173496, min_scn:{val:1729061895678607004, v:0}, max_scn:{val:1729061895678607004, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352416482}, prev_proposal_id:17, committed_end_lsn:{lsn:14352416604}, data_checksum:940009244, accum_checksum:4108938680, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061896173740, freeze_ts:1729061896173740, submit_ts:1729061896173744, flushed_ts:1729061896175016, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:1272}, fs_cb_begin_ts=1729061896274112, log_life_time=100372) [2024-10-16 14:58:16.276752] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f488680 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286002} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2118}, prev=this=0x2b0f3f41df70 trans_version={val:1729061835770258000, v:0} scn={val:1729061835770258000, v:0} tx_id={txid:392284956} prev=0x2b0f3f41de30 next=(nil) modify_count=1 acc_checksum=2387966360 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:6352}, snapshot_version={val:1729061896274533000, v:0}, *this={this=0x2b0f3f41db28 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f41df70 latest_compact_node=(nil) max_trans_version={val:1729061835770258000, v:0} max_trans_id=392284956 max_elr_trans_version={val:1729061835770258000, v:0} max_elr_trans_id=392284956 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835770258000, v:0} min_modify_scn={val:1729061835770258000, v:0}}) [2024-10-16 14:58:16.276785] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f488680 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286002} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2118}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851638, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851638, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896274533000, v:0}, tx_id:{txid:392286002}, scn:{branch:0, seq:1060}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286002}, tx_desc:{this:0x2b0ec37d1850, tx_id:{txid:392286002}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896276654, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896273474, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2118}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.276847] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286002} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286002}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:2118}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.276864] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830047189389312},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286002} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.276879] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.276884] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047189389312},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.276930] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.277153] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3181}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286002} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.277705] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=15] worker thread began to shrink(tenant_->id()=1003, group_id_=1, token=0) [2024-10-16 14:58:16.283288] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [24920][BlackListServic][T1][YB4A0A000407-00061E9B3769ABA4-0-0] [lt=12] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12239, add_timestamp:1729061896283278}) [2024-10-16 14:58:16.283317] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E71-0-0] [lt=8] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12239, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061896283303, sql_renew_time:1729061896283303, is_mark_fail:false}]) [2024-10-16 14:58:16.283342] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [21255][VTblLocAsyncUp0][T0][YB4A0A000407-00061E9B323C5E71-0-0] [lt=24] success to process renew task(task={tenant_id:1, table_id:12239, add_timestamp:1729061896283278}, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"10.0.4.7:2890", role:1, sql_port:2889, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1729061896283303, sql_renew_time:1729061896283303, is_mark_fail:false}]) [2024-10-16 14:58:16.285224] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107485][T1003_L0_G1][T1003][Y0-0000000000000000-0-0] [lt=6] worker thread exit(tenant_->id()=1003, workers_.get_size()=2) [2024-10-16 14:58:16.285239] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107485][T1003_L0_G1][T0][Y0-0000000000000000-0-0] [lt=11] tenant thread end_run(id_=1003, ret=0, thread_count_=172) [2024-10-16 14:58:16.287843] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_ls_info.cpp:54) [21648][T1_L5_G0][T1][YB4A0A000407-00061E9B3769ABA4-0-0] [lt=10][errcode=-4008] execute fail(ret=-4008) [2024-10-16 14:58:16.287974] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22267][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=46] get number(ret=0, service_type_=0, range=1, base_id=1729061896286667000, start_id=1729061896286667000, end_id=1729061896286667001) [2024-10-16 14:58:16.288049] INFO [LIB] stat (utility.h:1165) [22267][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=27, stat_interval=1000000, avg (count/cost)=18, this=0x2b0e0f1ed470, extra_info="") [2024-10-16 14:58:16.288064] INFO [LIB] stat (utility.h:1165) [22267][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=27, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e26119210, extra_info="id:1001") [2024-10-16 14:58:16.289203] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061896288315000, start_id=1729061896288315000, end_id=1729061896288315001) [2024-10-16 14:58:16.291873] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:193) [24920][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=11] ls blacklist refresh finish(cost_time=9127) [2024-10-16 14:58:16.292221] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.296100] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061896296027000, start_id=1729061896296027000, end_id=1729061896296027001) [2024-10-16 14:58:16.296828] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:16.298425] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22efa640 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286006} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3292}, prev=this=0x2b0f22eaf560 trans_version={val:1729061835785854000, v:0} scn={val:1729061835785854000, v:0} tx_id={txid:392284960} prev=0x2b0f22eaf420 next=(nil) modify_count=1 acc_checksum=1457312692 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2216}, snapshot_version={val:1729061896296027000, v:0}, *this={this=0x2b0f22eaf118 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22eaf560 latest_compact_node=(nil) max_trans_version={val:1729061835785854000, v:0} max_trans_id=392284960 max_elr_trans_version={val:1729061835785854000, v:0} max_elr_trans_id=392284960 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835785854000, v:0} min_modify_scn={val:1729061835785854000, v:0}}) [2024-10-16 14:58:16.298460] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22efa640 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286006} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3292}, ctx={this:0x2b0e2b254080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495852150, tablet_id:{id:209267}, table_iter:0x2b0e2b253210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495852150, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896296027000, v:0}, tx_id:{txid:392286006}, scn:{branch:0, seq:1394}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286006}, tx_desc:{this:0x2b0dba0c3af0, tx_id:{txid:392286006}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896297927, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896294636, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c278d50, mem_ctx:0x2b0e0c27a2d0, tx_scn:{branch:0, seq:3292}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.298517] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286006} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286006}, data:0x2b0e2b250438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:3292}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.298535] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830047403298816},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286006} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.298547] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.298554] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047403298816},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.298609] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.298942] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3296}, to_seq_no={branch:0, seq:1395}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286006} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.300170] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 488.50; sys_group_name: MICRO_BLOCK_CACHE_IO, mode: read, size: 16990.81, iops: 26.95, rt: 111.37; ) [2024-10-16 14:58:16.300191] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:16.300228] INFO [COMMON] print_io_status (ob_io_manager.cpp:1646) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1004, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 3.99, rt: 737.00; ) [2024-10-16 14:58:16.300233] INFO [COMMON] print_io_status (ob_io_manager.cpp:1666) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CONFIG](tenant_id=1004, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=23433792, free_request_count=50000, free_result_count=50000, callback_queues=[0]) [2024-10-16 14:58:16.300241] INFO [COMMON] print_sender_status (ob_io_struct.cpp:906) [21155][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 9, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 10, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 11, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 12, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 13, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 14, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 15, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 16, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; ) [2024-10-16 14:58:16.302861] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16D2-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364171254, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894492, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.304832] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16D4-0-0] [lt=39][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364173227, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894493, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.313517] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [22457][T1003_Flush][T1003][Y0-0000000000000000-0-0] [lt=8] ====== traversal_flush timer task ====== [2024-10-16 14:58:16.313619] INFO [MDS] flush (mds_table_mgr.cpp:172) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DB1-0-0] [lt=12] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0e3aec4510, is_inited:true, freezing_scn:{val:1729018983572868001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061909029444000, latest_log_ts:{val:1729061892483852001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:16.313683] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [22457][T1003_Flush][T1003][YB4A0A000407-00061E9B37598DB1-0-0] [lt=60] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:16.315020] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16D6-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364183413, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894494, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.315444] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:16.315461] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4388) [2024-10-16 14:58:16.315479] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22798][T1004_CKClogDis][T1004][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=373791183, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:16.315599] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=14] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=32, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0db33470, extra_info="") [2024-10-16 14:58:16.316880] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.318783] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16D8-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364187177, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894495, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.322292] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0e885d4e40 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286012} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1995}, prev=this=0x2b0e8858c218 trans_version={val:1729061835814605000, v:0} scn={val:1729061835814605000, v:0} tx_id={txid:392284966} prev=0x2b0e8858c0d8 next=(nil) modify_count=1 acc_checksum=4203208945 version=1729061779336102 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:9587}, snapshot_version={val:1729061896321927000, v:0}, *this={this=0x2b0e8858bdd0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0e8858c218 latest_compact_node=(nil) max_trans_version={val:1729061835814605000, v:0} max_trans_id=392284966 max_elr_trans_version={val:1729061835814605000, v:0} max_elr_trans_id=392284966 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835814605000, v:0} min_modify_scn={val:1729061835814605000, v:0}}) [2024-10-16 14:58:16.322323] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0e885d4e40 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286012} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779336102 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1995}, ctx={this:0x2b0e2b854080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851228, tablet_id:{id:209276}, table_iter:0x2b0e2b853210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851228, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896321927000, v:0}, tx_id:{txid:392286012}, scn:{branch:0, seq:1992}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286012}, tx_desc:{this:0x2b0e0fcc40f0, tx_id:{txid:392286012}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896321927, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896319936, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1995}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.322377] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286012} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286012}, data:0x2b0e2b850438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779336102, seq_no:{branch:0, seq:1995}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.322391] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830047663345665},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286012} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.322403] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.322408] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047663345665},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.322454] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.323417] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16D9-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364191811, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894496, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.323811] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3050}, to_seq_no={branch:0, seq:1993}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286012} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.324311] INFO eloop_run (eloop.c:140) [21193][pnio2][T0][Y0-0000000000000000-0-0] [lt=15] PNIO [ratelimit] time: 1729061896324309, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007620, add_bytes: 0 [2024-10-16 14:58:16.325353] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1183809536, global_cache_size=1579302912, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:16.325553] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] [CACHE] tenant_id= 1 cache memory info: [CACHE] tenant_id= 1 | cache_name= schema_cache | cache_size= 91552768 | cache_store_size= 89473024 | cache_map_size= 2079744 | kv_cnt= 5669 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= tablet_table_cache | cache_size= 12481536 | cache_store_size= 6242304 | cache_map_size= 6239232 | kv_cnt= 43492 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= vtable_cache | cache_size= 147733504 | cache_store_size= 145653760 | cache_map_size= 2079744 | kv_cnt= 76 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= index_block_cache | cache_size= 68662272 | cache_store_size= 62423040 | cache_map_size= 6239232 | kv_cnt= 31222 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= user_block_cache | cache_size= 222639104 | cache_store_size= 216399872 | cache_map_size= 6239232 | kv_cnt= 32713 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= user_row_cache | cache_size= 12483584 | cache_store_size= 10403840 | cache_map_size= 2079744 | kv_cnt= 2891 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= bf_cache | cache_size= 10402816 | cache_store_size= 8323072 | cache_map_size= 2079744 | kv_cnt= 193 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= fuse_row_cache | cache_size= 14564352 | cache_store_size= 12484608 | cache_map_size= 2079744 | kv_cnt= 4381 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= storage_meta_cache | cache_size= 285058048 | cache_store_size= 270499840 | cache_map_size= 14558208 | kv_cnt= 80863 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 17 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 25 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 234 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_column_stat_cache | cache_size= 16645120 | cache_store_size= 14565376 | cache_map_size= 2079744 | kv_cnt= 1027 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_ds_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 9 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1 | cache_name= BACKUP_INDEX_CACHE | cache_size= 0 | cache_store_size= 0 | cache_map_size= 0 | kv_cnt= 0 | hold_size= 0 [2024-10-16 14:58:16.325595] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] [CACHE] tenant_id= 1001 cache memory info: [CACHE] tenant_id= 1001 | cache_name= index_block_cache | cache_size= 22887424 | cache_store_size= 20807680 | cache_map_size= 2079744 | kv_cnt= 5229 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= user_block_cache | cache_size= 187268096 | cache_store_size= 185188352 | cache_map_size= 2079744 | kv_cnt= 10159 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= user_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1229 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= bf_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 28 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= fuse_row_cache | cache_size= 10402816 | cache_store_size= 8323072 | cache_map_size= 2079744 | kv_cnt= 2520 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= storage_meta_cache | cache_size= 41614336 | cache_store_size= 39534592 | cache_map_size= 2079744 | kv_cnt= 10145 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 39 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 6 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 133 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= opt_column_stat_cache | cache_size= 8322048 | cache_store_size= 6242304 | cache_map_size= 2079744 | kv_cnt= 362 | hold_size= 0 [CACHE] tenant_id= 1001 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [2024-10-16 14:58:16.325642] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] [CACHE] tenant_id= 1002 cache memory info: [CACHE] tenant_id= 1002 | cache_name= index_block_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 441 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= user_block_cache | cache_size= 35372032 | cache_store_size= 33292288 | cache_map_size= 2079744 | kv_cnt= 946 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= user_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1406 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= bf_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 19 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= fuse_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 2761 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= storage_meta_cache | cache_size= 16645120 | cache_store_size= 14565376 | cache_map_size= 2079744 | kv_cnt= 2728 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= tmp_block_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 3691 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 24 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1078 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= opt_column_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 624 | hold_size= 0 [CACHE] tenant_id= 1002 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [2024-10-16 14:58:16.325679] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=8] [CACHE] tenant_id= 1003 cache memory info: [CACHE] tenant_id= 1003 | cache_name= index_block_cache | cache_size= 24968192 | cache_store_size= 22888448 | cache_map_size= 2079744 | kv_cnt= 5100 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= user_block_cache | cache_size= 164379648 | cache_store_size= 162299904 | cache_map_size= 2079744 | kv_cnt= 11493 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= user_row_cache | cache_size= 8322048 | cache_store_size= 6242304 | cache_map_size= 2079744 | kv_cnt= 5065 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= bf_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 49 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= fuse_row_cache | cache_size= 8322048 | cache_store_size= 6242304 | cache_map_size= 2079744 | kv_cnt= 2220 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= storage_meta_cache | cache_size= 49937408 | cache_store_size= 47857664 | cache_map_size= 2079744 | kv_cnt= 10774 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= tmp_block_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 31 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 24 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 134 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= opt_column_stat_cache | cache_size= 12483584 | cache_store_size= 10403840 | cache_map_size= 2079744 | kv_cnt= 764 | hold_size= 0 [CACHE] tenant_id= 1003 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [2024-10-16 14:58:16.325716] INFO print_tenant_cache_info (ob_kvcache_inst_map.cpp:520) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=8] [CACHE] tenant_id= 1004 cache memory info: [CACHE] tenant_id= 1004 | cache_name= index_block_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 747 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= user_block_cache | cache_size= 14564352 | cache_store_size= 12484608 | cache_map_size= 2079744 | kv_cnt= 388 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= user_row_cache | cache_size= 10402816 | cache_store_size= 8323072 | cache_map_size= 2079744 | kv_cnt= 573 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= fuse_row_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1098 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= storage_meta_cache | cache_size= 24968192 | cache_store_size= 22888448 | cache_map_size= 2079744 | kv_cnt= 5924 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= tmp_block_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= tx_data_kv_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 6 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= log_kv_cache | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 27 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_table_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 161 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_column_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 461 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_ds_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 9 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= opt_system_stat_cache | cache_size= 6241280 | cache_store_size= 4161536 | cache_map_size= 2079744 | kv_cnt= 1 | hold_size= 0 [CACHE] tenant_id= 1004 | cache_name= BACKUP_INDEX_CACHE | cache_size= 4160512 | cache_store_size= 2080768 | cache_map_size= 2079744 | kv_cnt= 3 | hold_size= 0 [2024-10-16 14:58:16.325735] INFO print_current_status (ob_kvcache_hazard_version.cpp:328) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=8] [KVCACHE-HAZARD] hazard version status info: current_version: 207919091 | min_version=207919091 | total_nodes_count: 471 | [KVCACHE-HAZARD] i= 25 | acquire_version=18446744073709551615 | waiting_nodes_count= 18 | last_retire_version=207918620 | [KVCACHE-HAZARD] i= 26 | acquire_version=18446744073709551615 | waiting_nodes_count= 426 | last_retire_version=207918498 | [KVCACHE-HAZARD] i= 108 | acquire_version=18446744073709551615 | waiting_nodes_count= 1 | last_retire_version=207918498 | [KVCACHE-HAZARD] i= 732 | acquire_version=18446744073709551615 | waiting_nodes_count= 25 | last_retire_version=207918620 | [KVCACHE-HAZARD] i= 975 | acquire_version=18446744073709551615 | waiting_nodes_count= 1 | last_retire_version=207918224 | [2024-10-16 14:58:16.327396] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb4aad0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286013} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1059}, prev=this=0x2b0fadaee828 trans_version={val:1729061835818566000, v:0} scn={val:1729061835818566000, v:0} tx_id={txid:392284967} prev=0x2b0fadaee6e8 next=(nil) modify_count=1 acc_checksum=2859661457 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2915}, snapshot_version={val:1729061896326206000, v:0}, *this={this=0x2b0fadaee3e0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaee828 latest_compact_node=(nil) max_trans_version={val:1729061835818566000, v:0} max_trans_id=392284967 max_elr_trans_version={val:1729061835818566000, v:0} max_elr_trans_id=392284967 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061835818566000, v:0} min_modify_scn={val:1729061835818566000, v:0}}) [2024-10-16 14:58:16.327429] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb4aad0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286013} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1059}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851206, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851206, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896326206000, v:0}, tx_id:{txid:392286013}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286013}, tx_desc:{this:0x2b0ec37a6090, tx_id:{txid:392286013}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896327264, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896326206, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1059}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.327486] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286013} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286013}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1059}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.327503] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830047730454528},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286013} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.327518] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.327523] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830047730454528},{"BIGINT":1729058400},{"VARCHAR":"?ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.327569] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.327751] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286013} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.329163] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061896328397000, start_id=1729061896328397000, end_id=1729061896328397001) [2024-10-16 14:58:16.330872] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22efad20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286014} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1066}, prev=this=0x2b0f22ed0038 trans_version={val:1729061839188859000, v:0} scn={val:1729061839188859000, v:0} tx_id={txid:392202424} prev=0x2b0f22ecfef8 next=(nil) modify_count=1 acc_checksum=639015428 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3919}, snapshot_version={val:1729061896330518000, v:0}, *this={this=0x2b0f22ecf8c8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ed0038 latest_compact_node=(nil) max_trans_version={val:1729061839188859000, v:0} max_trans_id=392202424 max_elr_trans_version={val:1729061839188859000, v:0} max_elr_trans_id=392202424 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839188859000, v:0} min_modify_scn={val:1729061839188859000, v:0}}) [2024-10-16 14:58:16.330902] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22efad20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286014} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1066}, ctx={this:0x2b0e2b854080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851225, tablet_id:{id:209267}, table_iter:0x2b0e2b853210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851225, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896330518000, v:0}, tx_id:{txid:392286014}, scn:{branch:0, seq:1063}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286014}, tx_desc:{this:0x2b0e0fcba2d0, tx_id:{txid:392286014}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896330518, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896329456, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1066}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.330960] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286014} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286014}, data:0x2b0e2b850438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:1066}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.330974] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830041237671936},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286014} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.330985] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.330991] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041237671936},{"BIGINT":1729058400},{"VARCHAR":"@", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.331032] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.331320] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1069}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286014} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.332197] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16DB-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364200590, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894497, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.332284] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22454][T1003_ReqMemEvi][T1003][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1894, mem_used=4159488) [2024-10-16 14:58:16.334435] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21653][T1_L0_G0][T1][YB4A0A000407-00061E9B304F3E10-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364202830, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410667, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.336047] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16DD-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364204442, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894498, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.337210] INFO do_work (ob_rl_mgr.cpp:705) [21202][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=11] swc wakeup.(stat_period_=1000000, ready=false) [2024-10-16 14:58:16.337860] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16DF-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364206256, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894499, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.341846] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ee461b7c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286016} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779088644 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1085}, prev=this=0x2b0ecd3e0bc8 trans_version={val:1729061839195259002, v:0} scn={val:1729061839195259002, v:0} tx_id={txid:392202426} prev=0x2b0ecd3e0a88 next=(nil) modify_count=1 acc_checksum=1495598197 version=1729061779088644 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2121}, snapshot_version={val:1729061896341420000, v:0}, *this={this=0x2b0ecd3e0780 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ecd3e0bc8 latest_compact_node=(nil) max_trans_version={val:1729061839195259002, v:0} max_trans_id=392202426 max_elr_trans_version={val:1729061839195259002, v:0} max_elr_trans_id=392202426 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839195259002, v:0} min_modify_scn={val:1729061839195259002, v:0}}) [2024-10-16 14:58:16.341881] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ee461b7c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286016} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779088644 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1085}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851230, tablet_id:{id:209270}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851230, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896341420000, v:0}, tx_id:{txid:392286016}, scn:{branch:0, seq:1082}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286016}, tx_desc:{this:0x2b0ec37afeb0, tx_id:{txid:392286016}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896341420, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896340339, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1085}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.341942] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286016} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286016}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779088644, seq_no:{branch:0, seq:1085}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.341970] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=0] primary key duplication conflict(key={"BIGINT":654830041334140929},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286016} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.341989] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.342000] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041334140929},{"BIGINT":1729058400},{"VARCHAR":"?©™™™™™š?ÄzáG®{?â=p£× =?®¸Që…¸?±ë…¸Qì?©™™™™™š?©™™™™™š?©™™™™™š?©™™™™™š?©™™™™™š?±ë…¸Qì?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?©™™™™™š?®¸Që…¸?À£× =p¤?®¸Që…¸?®¸Që…¸?®¸Që…¸?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?®¸Që…¸?Ã333333?±ë…¸Qì?±ë…¸Qì?· =p£× ?±ë…¸Qì?©™™™™™š?®¸Që…¸?®¸Që…¸?©™™™™™š?©™™™™™š?±ë…¸Qì?©™™™™™š?´záG®{?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?±ë…¸Qì?©™™™™™š?±ë…¸Qì?®¸Që…¸?±ë…¸Qì?±ë…¸Qì?®¸Që…¸?®¸Që…¸?±ë…¸Qì?®¸Që…¸?´záG®{?¹™™™™™š?±ë…¸Qì?´záG®{¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.342065] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=39][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.342323] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1088}, to_seq_no={branch:0, seq:1083}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286016} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.345760] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be45de0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286017} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2125}, prev=this=0x2b0ec6dda508 trans_version={val:1729061839198410000, v:0} scn={val:1729061839198410000, v:0} tx_id={txid:392202427} prev=0x2b0ec6dda3c8 next=(nil) modify_count=1 acc_checksum=1900416214 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2092}, snapshot_version={val:1729061896344586000, v:0}, *this={this=0x2b0ec6dda0c0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6dda508 latest_compact_node=(nil) max_trans_version={val:1729061839198410000, v:0} max_trans_id=392202427 max_elr_trans_version={val:1729061839198410000, v:0} max_elr_trans_id=392202427 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839198410000, v:0} min_modify_scn={val:1729061839198410000, v:0}}) [2024-10-16 14:58:16.345794] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be45de0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286017} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2125}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851264, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851264, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896344586000, v:0}, tx_id:{txid:392286017}, scn:{branch:0, seq:1053}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286017}, tx_desc:{this:0x2b0ec37a9fd0, tx_id:{txid:392286017}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896345658, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896343534, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2125}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.345851] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286017} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286017}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:2125}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.345869] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830041388666880},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286017} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.345884] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.345889] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041388666880},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.345937] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.346141] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2128}, to_seq_no={branch:0, seq:1054}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286017} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.349698] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16E1-0-0] [lt=21][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364218092, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894500, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.350210] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb4b1b0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286018} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2123}, prev=this=0x2b0fadb10a58 trans_version={val:1729061839201589000, v:0} scn={val:1729061839201589000, v:0} tx_id={txid:392202428} prev=0x2b0fadb10918 next=(nil) modify_count=1 acc_checksum=2918367756 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2121}, snapshot_version={val:1729061896349862000, v:0}, *this={this=0x2b0fadb10610 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadb10a58 latest_compact_node=(nil) max_trans_version={val:1729061839201589000, v:0} max_trans_id=392202428 max_elr_trans_version={val:1729061839201589000, v:0} max_elr_trans_id=392202428 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839201589000, v:0} min_modify_scn={val:1729061839201589000, v:0}}) [2024-10-16 14:58:16.350241] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb4b1b0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286018} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2123}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851233, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851233, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896349862000, v:0}, tx_id:{txid:392286018}, scn:{branch:0, seq:1063}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286018}, tx_desc:{this:0x2b0ec37d37f0, tx_id:{txid:392286018}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896349862, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896347740, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2123}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.350296] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286018} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286018}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:2123}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.350313] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830041438998528},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286018} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.350327] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.350333] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041438998528},{"BIGINT":1729058400},{"VARCHAR":"?©™™™™™š?ÄzáG®{?â=p£× =?®¸Që…¸?±ë…¸Qì?©™™™™™š?©™™™™™š?©™™™™™š?©™™™™™š?©™™™™™š?±ë…¸Qì?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?©™™™™™š?®¸Që…¸?À£× =p¤?®¸Që…¸?®¸Që…¸?®¸Që…¸?±ë…¸Qì?±ë…¸Qì?±ë…¸Qì?®¸Që…¸?Ã333333?±ë…¸Qì?±ë…¸Qì?· =p£× ?±ë…¸Qì?©™™™™™š?®¸Që…¸?®¸Që…¸?©™™™™™š?©™™™™™š?±ë…¸Qì?©™™™™™š?´záG®{?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?®¸Që…¸?±ë…¸Qì?©™™™™™š?±ë…¸Qì?®¸Që…¸?±ë…¸Qì?±ë…¸Qì?®¸Që…¸?®¸Që…¸?±ë…¸Qì?®¸Që…¸?´záG®{?¹™™™™™š?±ë…¸Qì?´záG®{¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.350375] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.350590] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2126}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286018} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.354121] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=28][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22efb400 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286019} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0f22ed0df8 trans_version={val:1729061839205995000, v:0} scn={val:1729061839205995000, v:0} tx_id={txid:392202429} prev=0x2b0f22ed0cb8 next=(nil) modify_count=1 acc_checksum=819416759 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4406}, snapshot_version={val:1729061896353044000, v:0}, *this={this=0x2b0f22ed09b0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ed0df8 latest_compact_node=(nil) max_trans_version={val:1729061839205995000, v:0} max_trans_id=392202429 max_elr_trans_version={val:1729061839205995000, v:0} max_elr_trans_id=392202429 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839205995000, v:0} min_modify_scn={val:1729061839205995000, v:0}}) [2024-10-16 14:58:16.354157] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=36][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22efb400 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286019} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851446, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851446, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896353044000, v:0}, tx_id:{txid:392286019}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286019}, tx_desc:{this:0x2b0ec37c5a90, tx_id:{txid:392286019}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896353044, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896353044, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.354215] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286019} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286019}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.354232] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830041489330176},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286019} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.354246] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.354251] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041489330176},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.354301] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.354479] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1080}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286019} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.355384] INFO [LIB] stat (utility.h:1165) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=230, this=0x2b0e431f7a70, extra_info="") [2024-10-16 14:58:16.355391] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=8] ====== tenant freeze timer task ====== [2024-10-16 14:58:16.355399] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:16.355411] INFO [LIB] stat (utility.h:1182) [22611][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=74, this=0x2b0e0f9bac70, extra_info="") [2024-10-16 14:58:16.355421] INFO [LIB] stat (utility.h:1165) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=270, this=0x2b0e431dfa70, extra_info="") [2024-10-16 14:58:16.355427] INFO [LIB] stat (utility.h:1182) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=66, this=0x2b0e0f9ba530, extra_info="") [2024-10-16 14:58:16.355404] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=10] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=12582912, ctx.total_memstore_used_=46137344, ctx.total_memstore_hold_=46137344, memstore_freeze_trigger_limit_=28020300, tenant_id=1002) [2024-10-16 14:58:16.355448] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22081][T1002_Occam][T1002][YB4A0A000407-00061E99750960C7-0-0] [lt=34] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:16.355491] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:16.355500] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e431dfca0, extra_info="") [2024-10-16 14:58:16.355546] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE4027A-0-0] [lt=23] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=15, avg_log_submit_to_flush_time=341, avg_log_submit_to_slide_time=1748) [2024-10-16 14:58:16.355650] INFO [LIB] stat (utility.h:1165) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE4027B-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431d56a0, extra_info="palf_id:1") [2024-10-16 14:58:16.355655] INFO [LIB] stat (utility.h:1165) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BE4027B-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=846, this=0x2b0e431d58d0, extra_info="palf_id:1") [2024-10-16 14:58:16.356875] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061896356256000, start_id=1729061896356256000, end_id=1729061896356256001) [2024-10-16 14:58:16.357718] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16E2-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364226112, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894501, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.359481] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16E4-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364227875, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894502, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.362596] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22efbae0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286021} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, prev=this=0x2b0f22ed14f8 trans_version={val:1729061839211310001, v:0} scn={val:1729061839211310001, v:0} tx_id={txid:392202431} prev=0x2b0f22ed13b8 next=(nil) modify_count=1 acc_checksum=2721410555 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2143}, snapshot_version={val:1729061896361695000, v:0}, *this={this=0x2b0f22ed10b0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ed14f8 latest_compact_node=(nil) max_trans_version={val:1729061839211310001, v:0} max_trans_id=392202431 max_elr_trans_version={val:1729061839211310001, v:0} max_elr_trans_id=392202431 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839211310001, v:0} min_modify_scn={val:1729061839211310001, v:0}}) [2024-10-16 14:58:16.362641] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=44][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22efbae0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286021} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851221, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851221, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896361695000, v:0}, tx_id:{txid:392286021}, scn:{branch:0, seq:1057}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286021}, tx_desc:{this:0x2b0ec37b5d90, tx_id:{txid:392286021}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896361695, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896360639, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1060}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.362707] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=65][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286021} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286021}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:1060}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.362724] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830041678073856},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286021} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.362745] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.362761] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041678073856},{"BIGINT":1729058400},{"VARCHAR":"?ÔzáG®{?ØQë…¸R?ØQë…¸R?Öffffff?ØõÂ\(ö?Õ¸Që…?Ó× =p£×?Ó× =p£×?ÕÂ\(õÃ?Ó× =p£×?Ù™™™™™š?Öffffff?× =p£× ?ØõÂ\(ö?Öffffff?ÕÂ\(õÃ?ÕÂ\(õÃ?ÔzáG®{?×®záG®?× =p£× ?Ó× =p£×?Öffffff?Öffffff?ØQë…¸R?× =p£× ?× =p£× ?Ú=p£× =?Öffffff?Õ¸Që…?× =p£× ?× =p£× ?Õ¸Që…?Öffffff?× =p£× ?Õ¸Që…?ÔzáG®{?ÚáG®zá?Õ¸Që…?ÞzáG®?Öffffff?Öffffff?Öffffff?×®záG®?ØQë…¸R?Ú=p£× =?Õ¸Që…?ØQë…¸R?ÕÂ\(õÃ?× =p£× ?Ú=p£× =?Õ¸Që…?× =p£× ?ØõÂ\(ö?Öffffff?Öffffff?àõÂ\(ö?×®záG®?× =p£× ¿ð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.362810] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=25][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.363159] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2344}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286021} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.364635] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16E6-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364233028, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894503, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.373633] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107539][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54084-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1003, self="10.0.4.7:2890", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=0, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=499, avg_log_submit_to_slide_time=69465) [2024-10-16 14:58:16.373861] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061896373752000, start_id=1729061896373752000, end_id=1729061896373752001) [2024-10-16 14:58:16.375691] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16E8-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364244083, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894504, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.381581] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16EA-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364249974, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894505, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.382693] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21927][T1001_L5_G0][T1001][YB4A0A000408-00061E9B452E1E0C-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061896381189000, start_id=1729061896381189000, end_id=1729061896381189001) [2024-10-16 14:58:16.383340] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=31][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f488d60 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286025} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1681}, prev=this=0x2b0f3f4523d0 trans_version={val:1729061839228090001, v:0} scn={val:1729061839228090001, v:0} tx_id={txid:392202435} prev=0x2b0f3f452290 next=(nil) modify_count=1 acc_checksum=1139126003 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3183}, snapshot_version={val:1729061896382867000, v:0}, *this={this=0x2b0f3f451f88 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f4523d0 latest_compact_node=(nil) max_trans_version={val:1729061839228090001, v:0} max_trans_id=392202435 max_elr_trans_version={val:1729061839228090001, v:0} max_elr_trans_id=392202435 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839228090001, v:0} min_modify_scn={val:1729061839228090001, v:0}}) [2024-10-16 14:58:16.383370] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f488d60 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286025} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1681}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851222, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851222, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896382867000, v:0}, tx_id:{txid:392286025}, scn:{branch:0, seq:1678}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286025}, tx_desc:{this:0x2b0ec37a6090, tx_id:{txid:392286025}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896382867, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896381190, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1681}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.383423] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=52][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286025} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286025}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:1681}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.383438] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830041904566272},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286025} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.383458] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.383465] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041904566272},{"BIGINT":1729058400},{"VARCHAR":"Al(ýáG®Al)~fffAl)´(õÂAl)TÌÌÍAl)+ÌÌÍAl)6Ä(õÃAl)BtzáHAl)NáG®Al)YÏ =qAl)e|záHAl)q™™šAl)|¼záHAl)ˆeÂ\Al)” (õÂAl)Ÿ»333Al)«w®zAl)·\)Al)ÂÓ…¸Al)ÎnfffAl)Ú%p£×Al)åÒ=p¤Al)ñޏQìAl)ýNfffAl* ë…Al*ÈQë†Al* z\)Al*, ÌÌÌAl*7Ê=p¤Al*Cˆ£× Al*O%¸RAl*ZÒáG®Al*ftzáHAl*r*\)Al*}͸RAl*‰U¸RAl*•®{Al* •p£×Al*¬záHAl*·³…¸Al*ÃL(õÃAl*Îñ™™šAl*ÚK× >Al*åîfffAl*ñŽfffAl*ý/\(öAl+Ë× >Al+g\(öAl+û…¸Al++ˆ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.383510] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.383763] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1684}, to_seq_no={branch:0, seq:1679}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286025} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.388250] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16EC-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364256643, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894506, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.388444] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4559-0-0] [lt=24] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=48055) [2024-10-16 14:58:16.390246] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16EE-0-0] [lt=64][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364258640, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894507, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.390369] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1729061896390344000, start_id=1729061896390344000, end_id=1729061896390344001) [2024-10-16 14:58:16.390715] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22efc1c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286027} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2146}, prev=this=0x2b0f22ed22d8 trans_version={val:1729061839233388001, v:0} scn={val:1729061839233388001, v:0} tx_id={txid:392202437} prev=0x2b0f22ed2198 next=(nil) modify_count=1 acc_checksum=4047721414 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3185}, snapshot_version={val:1729061896390344000, v:0}, *this={this=0x2b0f22ed1e90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ed22d8 latest_compact_node=(nil) max_trans_version={val:1729061839233388001, v:0} max_trans_id=392202437 max_elr_trans_version={val:1729061839233388001, v:0} max_elr_trans_id=392202437 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839233388001, v:0} min_modify_scn={val:1729061839233388001, v:0}}) [2024-10-16 14:58:16.390750] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=33][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22efc1c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286027} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2146}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851213, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851213, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896390344000, v:0}, tx_id:{txid:392286027}, scn:{branch:0, seq:1057}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286027}, tx_desc:{this:0x2b0ec37a40f0, tx_id:{txid:392286027}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896390344, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896388199, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2146}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.390807] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286027} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286027}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:2146}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.390824] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830041992646656},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286027} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.390839] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.390844] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830041992646656},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.390897] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.391856] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061896391401000, start_id=1729061896391401000, end_id=1729061896391401001) [2024-10-16 14:58:16.392104] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3203}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286027} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.392281] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.392512] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] [WRS] update tenant weak read server version(tenant_id=1003, server_version={version:{val:1729061896190843000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1729061896392487}, version_delta=201669) [2024-10-16 14:58:16.396224] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21596][T1_LSArchiveMgr][T1][YB4A0A000407-00061E996C2960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:58:16.396764] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:16.396778] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=27262976, ctx.total_memstore_used_=60817408, ctx.total_memstore_hold_=60817408, memstore_freeze_trigger_limit_=54432140, tenant_id=1003) [2024-10-16 14:58:16.396789] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22338][T1003_Occam][T1003][YB4A0A000407-00061E9982D960C4-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:16.398066] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16F0-0-0] [lt=23][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364266459, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894508, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.403927] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [21657][T1_L0_G0][T1][YB4A0A000408-00061E9B6EFF3E12-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364272321, cache_obj->added_lc()=false, cache_obj->get_object_id()=17410668, cache_obj->get_tenant_id()=1, lbt()="0x1a5796b1 0xd233e9e 0x65b9e8e 0xd1a517d 0xca2e30e 0x657c576 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.415670] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16F2-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364284057, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894509, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.416031] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=44] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=26, this=0x2b0de0acd470, extra_info="") [2024-10-16 14:58:16.416048] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3657, avg_group_log_size=332) [2024-10-16 14:58:16.416079] INFO [LIB] stat (utility.h:1165) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0dee9a1210, extra_info="id:1") [2024-10-16 14:58:16.416455] INFO [LIB] stat (utility.h:1165) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=328, this=0x2b0de0ad7a70, extra_info="") [2024-10-16 14:58:16.416473] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=3657) [2024-10-16 14:58:16.416486] INFO [LIB] stat (utility.h:1182) [21444][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=60, this=0x2b0dde7ddb40, extra_info="") [2024-10-16 14:58:16.416517] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=3657) [2024-10-16 14:58:16.416535] INFO [LIB] stat (utility.h:1165) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x2b0de0ad7ca0, extra_info="") [2024-10-16 14:58:16.416699] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45D5F-0-0] [lt=8] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=4, this=0x2b0de0acd6a0, extra_info="palf_id:1") [2024-10-16 14:58:16.416708] INFO [LIB] stat (utility.h:1165) [106960][T1_L0_G1][T1][YB4A0A000409-00061E9A3BA45D5F-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1179, this=0x2b0de0acd8d0, extra_info="palf_id:1") [2024-10-16 14:58:16.417744] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16F4-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364286138, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894510, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.417794] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.421516] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16F6-0-0] [lt=14][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364289911, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894511, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.422877] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16F8-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364291273, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894512, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.426258] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=50, replace_time=0, replace_start_pos=283113, replace_num=31457) [2024-10-16 14:58:16.426277] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] replace map num details(ret=0, replace_node_count=50, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:16.428049] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=26] [PALF STAT FLUSH CB COST TIME](cur_stat_count=31, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0dacbca0, extra_info="") [2024-10-16 14:58:16.438326] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16FA-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364306717, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894513, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.439893] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16FC-0-0] [lt=27][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364308289, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894514, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.440135] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [21728][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=23] LogLoopThread round_cost_time(us)(round_cost_time=10) [2024-10-16 14:58:16.441445] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16FE-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364309840, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894515, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.442226] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f56e330 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286032} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1068}, prev=this=0x2b0f7f515100 trans_version={val:1729061839252043000, v:0} scn={val:1729061839252043000, v:0} tx_id={txid:392202442} prev=0x2b0f7f514fc0 next=(nil) modify_count=1 acc_checksum=3999786852 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2130}, snapshot_version={val:1729061896441389000, v:0}, *this={this=0x2b0f7f514cb8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f515100 latest_compact_node=(nil) max_trans_version={val:1729061839252043000, v:0} max_trans_id=392202442 max_elr_trans_version={val:1729061839252043000, v:0} max_elr_trans_id=392202442 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839252043000, v:0} min_modify_scn={val:1729061839252043000, v:0}}) [2024-10-16 14:58:16.442255] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f56e330 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286032} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1068}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851237, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851237, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896441389000, v:0}, tx_id:{txid:392286032}, scn:{branch:0, seq:1065}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286032}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392286032}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896441389, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896440325, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1068}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.442321] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=65][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286032} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286032}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1068}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.442336] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830042231721984},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286032} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.442347] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.442353] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042231721984},{"BIGINT":1729058400},{"VARCHAR":"AA8Ã× =AA8ŸRAA8!Që…AA8(ÈõÂAA80õÂAA87+…¸AA8>b\)AA8EË…¸AA8Ló333AA8T1ë…AA8[Ü(õÂAA8c ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.442404] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.442657] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2145}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286032} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.444892] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA8A2-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:16.444921] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000409-00061E9A27AFA8A2-0-0] [lt=13] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061896431806000, v:0}, arg={tenant_id:1002, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1003}, cur_readable_scn:{val:1729061896431806000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:16.445167] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061896443529000, start_id=1729061896443529000, end_id=1729061896443529001) [2024-10-16 14:58:16.446048] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000409-00061E9A27AFA8A2-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061896445191000, start_id=1729061896445191000, end_id=1729061896445191001) [2024-10-16 14:58:16.448163] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061896447313000, start_id=1729061896447313001, end_id=1729061896447313002) [2024-10-16 14:58:16.448238] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=19, avg_log_batch_cnt=1, total_group_log_size=10625, avg_group_log_size=559) [2024-10-16 14:58:16.448253] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10786) [2024-10-16 14:58:16.448514] INFO [LIB] stat (utility.h:1165) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=312, this=0x2b0df99eda70, extra_info="") [2024-10-16 14:58:16.448527] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10625) [2024-10-16 14:58:16.448538] INFO [LIB] stat (utility.h:1182) [21724][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=137, this=0x2b0df7bba530, extra_info="") [2024-10-16 14:58:16.449374] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10786) [2024-10-16 14:58:16.449389] INFO [LIB] stat (utility.h:1165) [21723][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=7, this=0x2b0df99edca0, extra_info="") [2024-10-16 14:58:16.449562] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F971C-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=5, this=0x2b0df99e36a0, extra_info="palf_id:1") [2024-10-16 14:58:16.449571] INFO [LIB] stat (utility.h:1165) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F971C-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=19, stat_interval=1000000, avg (count/cost)=854, this=0x2b0df99e38d0, extra_info="palf_id:1") [2024-10-16 14:58:16.449575] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107369][T1001_L0_G1][T1001][YB4A0A000408-00061E9B7F7F971C-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=19, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=34, avg_log_submit_to_flush_time=537, avg_log_submit_to_slide_time=819) [2024-10-16 14:58:16.449615] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000409-00061E9A27AFA8A2-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061896449420000, start_id=1729061896449420000, end_id=1729061896449420001) [2024-10-16 14:58:16.449689] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get_max_applied_scn(scn={val:1729061896186590000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:246, palf_committed_end_lsn_:{lsn:17959751073}, palf_committed_end_scn_:{val:1729061896384989000, v:0}, last_check_scn_:{val:1729061896384989000, v:0}, max_applied_cb_scn_:{val:1729061896186590000, v:0}}) [2024-10-16 14:58:16.449709] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get_min_unreplayed_log_info(lsn={lsn:7385763025}, scn={val:1722494261705406001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323102, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:1576454, ret_code_:0}}, next_to_submit_lsn_:{lsn:7385763025}, next_to_submit_scn_:{val:1722494261705406001, v:0}, base_lsn_:{lsn:7314419712}, base_scn_:{val:1722490032781494005, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5ba05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:7385762903}, end_lsn:{lsn:7385763025}, read_buf:{buf_len_:3747840, buf_:0x2b0e5ba05000}, block_size:67104768, log_storage_:0x2b0e0f1dba30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261705406000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261705406000, v:0}, data_checksum:2923900223, flag:0}}, init_mode_version:0, accumulate_checksum:3172656480, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261705406000, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:392, miss_cnt:21, cache_read_size:63089, hit_ratio:"9.491525423728813804e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:21, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:38, read_io_size:71905280, read_disk_cost_ts:1085706, start_lsn:{lsn:7314419712}}}}}}) [2024-10-16 14:58:16.449759] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=40] get_max_applied_scn(scn={val:1729061896343529000, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:130, palf_committed_end_lsn_:{lsn:204023298005}, palf_committed_end_scn_:{val:1729061896443529000, v:0}, last_check_scn_:{val:1729061896443529000, v:0}, max_applied_cb_scn_:{val:1729061896343529000, v:0}}) [2024-10-16 14:58:16.449767] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] get_min_unreplayed_log_info(lsn={lsn:57923794664}, scn={val:1722494261263129002, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061778323130, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:881253, ret_code_:0}}, next_to_submit_lsn_:{lsn:57923794664}, next_to_submit_scn_:{val:1722494261263129002, v:0}, base_lsn_:{lsn:57911414784}, base_scn_:{val:1722494112556363002, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e5be05000, next_round_pread_size:3678208, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:57923794542}, end_lsn:{lsn:57923794664}, read_buf:{buf_len_:3747840, buf_:0x2b0e5be05000}, block_size:67104768, log_storage_:0x2b0e0f1f3a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1722494261263129001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1722494261263129001, v:0}, data_checksum:154833137, flag:0}}, init_mode_version:0, accumulate_checksum:589435233, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1722494261263129001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1001, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:754, miss_cnt:3, cache_read_size:1349346, hit_ratio:"9.960369881109643053e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:3, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:5, read_io_size:11059200, read_disk_cost_ts:44381, start_lsn:{lsn:57911414784}}}}}}) [2024-10-16 14:58:16.449792] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:748) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] get_max_applied_scn(scn={val:1722494433932036000, v:0}, this={ls_id_:{id:1002}, role_:2, proposal_id_:87, palf_committed_end_lsn_:{lsn:60468525403}, palf_committed_end_scn_:{val:1722494433932036000, v:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1722494433932036000, v:0}}) [2024-10-16 14:58:16.449798] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1043) [22235][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=5] get_min_unreplayed_log_info(lsn={lsn:213058431755}, scn={val:1729061896426516002, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1729061896428321, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:213058431755}, next_to_submit_scn_:{val:1729061896426516002, v:0}, base_lsn_:{lsn:60461395968}, base_scn_:{val:1722494104078806000, v:0}, iterator_:{iterator_impl:{buf_:0x2b0e7f805000, next_round_pread_size:3678208, curr_read_pos:1624, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1624, log_storage_:{IteratorStorage:{start_lsn:{lsn:213058430131}, end_lsn:{lsn:213058431755}, read_buf:{buf_len_:3747840, buf_:0x2b0e7f805000}, block_size:67104768, log_storage_:0x2b0e0dac7a30}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1729061896426516001, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:1536, scn_:{val:1729061896426516001, v:0}, data_checksum:3410239987, flag:1}}, init_mode_version:0, accumulate_checksum:3885394073, curr_entry_is_padding:0, padding_entry_size:1568, padding_entry_scn:{val:1729061896426516001, v:0}}, io_ctx:{user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561654, miss_cnt:1, cache_read_size:152590871737, hit_ratio:"9.999999928347080846e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}}}}) [2024-10-16 14:58:16.451004] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [21818][T1001_ApplySrv0][T1001][Y0-0000000000000000-0-0] [lt=5] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:16.451674] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] no need to update min/max version(new_version={val:1729061896051711000, v:0}, min_version={val:1729061896051711000, v:0}, max_version={val:1729061896051711000, v:0}) [2024-10-16 14:58:16.456596] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be464c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286034} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1065}, prev=this=0x2b0ec6ddb5f0 trans_version={val:1729061839257418000, v:0} scn={val:1729061839257418000, v:0} tx_id={txid:392202444} prev=0x2b0ec6ddb4b0 next=(nil) modify_count=1 acc_checksum=144213248 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1073}, snapshot_version={val:1729061896455956000, v:0}, *this={this=0x2b0ec6ddb1a8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6ddb5f0 latest_compact_node=(nil) max_trans_version={val:1729061839257418000, v:0} max_trans_id=392202444 max_elr_trans_version={val:1729061839257418000, v:0} max_elr_trans_id=392202444 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839257418000, v:0} min_modify_scn={val:1729061839257418000, v:0}}) [2024-10-16 14:58:16.456640] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=45][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be464c0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286034} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1065}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851254, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851254, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896455956000, v:0}, tx_id:{txid:392286034}, scn:{branch:0, seq:1062}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286034}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392286034}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896455956, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896454895, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1065}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.456698] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286034} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286034}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:1065}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.456713] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830042395299840},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286034} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.456725] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.456731] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042395299840},{"BIGINT":1729058400},{"VARCHAR":"A±cS÷ÏA½n—PAÜó¶E¢Aä'ï²Aê–‡+Aï…¸RAõr° ÅAú†$Ý/Aÿ‘hr°A¹XbA", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.456785] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.457035] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1068}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286034} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.459263] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22267][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4] get number(ret=0, service_type_=0, range=1, base_id=1729061896459147000, start_id=1729061896459147000, end_id=1729061896459147001) [2024-10-16 14:58:16.459847] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C16FF-0-0] [lt=24][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364328241, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894516, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.461816] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1701-0-0] [lt=16][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364330210, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894517, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.464972] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=22][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f56ed18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286036} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2188}, prev=this=0x2b0f7f515800 trans_version={val:1729061839263764001, v:0} scn={val:1729061839263764001, v:0} tx_id={txid:392202446} prev=0x2b0f7f5156c0 next=(nil) modify_count=1 acc_checksum=706682054 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3189}, snapshot_version={val:1729061896463095000, v:0}, *this={this=0x2b0f7f5153b8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f515800 latest_compact_node=(nil) max_trans_version={val:1729061839263764001, v:0} max_trans_id=392202446 max_elr_trans_version={val:1729061839263764001, v:0} max_elr_trans_id=392202446 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839263764001, v:0} min_modify_scn={val:1729061839263764001, v:0}}) [2024-10-16 14:58:16.465007] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f56ed18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286036} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2188}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495852447, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495852447, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896463095000, v:0}, tx_id:{txid:392286036}, scn:{branch:0, seq:1123}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286036}, tx_desc:{this:0x2b0ec37cf8b0, tx_id:{txid:392286036}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896464160, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896461973, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:2188}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.465065] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286036} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286036}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:2188}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:16.465104] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=39][errcode=0] primary key duplication conflict(key={"BIGINT":654830042483380224},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286036} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.465117] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:16.465123] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042483380224},{"BIGINT":1729058400},{"VARCHAR":"BžŸ®ˆ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:16.465174] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:16.465510] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3251}, to_seq_no={branch:0, seq:1124}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286036} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.468178] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [21583][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=21] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0ddc5c62b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:16.470505] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0eeda78e60 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286038} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778320400 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0eeda32f10 trans_version={val:1729061839280619000, v:0} scn={val:1729061839280619000, v:0} tx_id={txid:392202448} prev=0x2b0eeda32dd0 next=(nil) modify_count=1 acc_checksum=362887284 version=1729061778320400 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:12186}, snapshot_version={val:1729061896469401000, v:0}, *this={this=0x2b0eeda32ac8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0eeda32f10 latest_compact_node=(nil) max_trans_version={val:1729061839280619000, v:0} max_trans_id=392202448 max_elr_trans_version={val:1729061839280619000, v:0} max_elr_trans_id=392202448 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061839280619000, v:0} min_modify_scn={val:1729061839280619000, v:0}}) [2024-10-16 14:58:16.470537] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=31][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0eeda78e60 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286038} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778320400 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851354, tablet_id:{id:209264}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851354, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061896469401000, v:0}, tx_id:{txid:392286038}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286038}, tx_desc:{this:0x2b0ec37d5790, tx_id:{txid:392286038}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061896469401, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061896469401, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:16.470820] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1276}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286038} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.491230] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1917) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2024-10-16 14:58:16.491251] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1458) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=17] start lib cache evict(tenant_id=1004, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:58:16.491258] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1475) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=6] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=43933696, mem_limit=53687090, cache_obj_num=103, cache_node_num=103) [2024-10-16 14:58:16.491264] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2698) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:16.492335] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.493448] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2417) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=103, alloc_obj_list=[{obj_id:11971, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178841}, {obj_id:21353, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175244}, {obj_id:24153, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185024}, {obj_id:24666, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175034}, {obj_id:24809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194650}, {obj_id:25391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181029}, {obj_id:25397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179368}, {obj_id:25786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173168}, {obj_id:26460, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175607}, {obj_id:26461, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174793}, {obj_id:26462, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177468}, {obj_id:26724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243192}, {obj_id:26730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173312}, {obj_id:26731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192506}, {obj_id:26732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180425}, {obj_id:26733, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227323}, {obj_id:26739, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173664}, {obj_id:26740, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172783}, {obj_id:26741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173264}, {obj_id:26742, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181521}, {obj_id:26743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181539}, {obj_id:26744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182656}, {obj_id:26745, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185159}, {obj_id:26746, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186088}, {obj_id:26747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176692}, {obj_id:26748, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174544}, {obj_id:26749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176065}, {obj_id:26750, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221250}, {obj_id:26751, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174864}, {obj_id:26752, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212200}, {obj_id:26753, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174128}, {obj_id:26754, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:26755, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243440}, {obj_id:26756, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221698}, {obj_id:26757, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191572}, {obj_id:26761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192910}, {obj_id:26762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174160}, {obj_id:26763, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175696}, {obj_id:26764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175469}, {obj_id:26765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175676}, {obj_id:26766, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228604}, {obj_id:26767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175936}, {obj_id:26768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183165}, {obj_id:26769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201048}, {obj_id:26770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175874}, {obj_id:26771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197834}, {obj_id:26773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185968}, {obj_id:26775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200712}, {obj_id:26776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26777, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172848}, {obj_id:26778, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173472}, {obj_id:26779, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177048}, {obj_id:26780, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199666}, {obj_id:26781, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177500}, {obj_id:26782, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177234}, {obj_id:26783, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177560}, {obj_id:26784, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178901}, {obj_id:26785, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177392}, {obj_id:26786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178817}, {obj_id:26787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26788, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180247}, {obj_id:26789, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178877}, {obj_id:26790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177709}, {obj_id:26791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180687}, {obj_id:26792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26793, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178993}, {obj_id:26794, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181376}, {obj_id:26795, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26796, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177504}, {obj_id:26797, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26798, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179074}, {obj_id:26799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180685}, {obj_id:26800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:26802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178886}, {obj_id:26804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177456}, {obj_id:26805, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26807, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184526}, {obj_id:26808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179073}, {obj_id:26810, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26811, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177552}, {obj_id:26812, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177440}, {obj_id:26814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225488}, {obj_id:26815, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297565}, {obj_id:26820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187924}, {obj_id:26821, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186440}, {obj_id:26822, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179212}, {obj_id:26823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234692}, {obj_id:26852, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236651}, {obj_id:26881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:864025}, {obj_id:26882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182029}, {obj_id:26883, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:26884, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173360}, {obj_id:26885, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174107}, {obj_id:26886, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:26887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173696}, {obj_id:26888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173568}, {obj_id:26889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230696}]) [2024-10-16 14:58:16.493585] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2706) [22640][T1004_PlanCache][T1004][Y0-0000000000000000-0-0] [lt=135] schedule next cache evict task(evict_interval=5000000) [2024-10-16 14:58:16.493791] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061896493726000, start_id=1729061896493726000, end_id=1729061896493726001) [2024-10-16 14:58:16.494517] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=2] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3983}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286043} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.495162] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=42] tx gc loop thread is running(MTL_ID()=1003) [2024-10-16 14:58:16.495172] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=10] try gc retain ctx [2024-10-16 14:58:16.495176] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=3] try to retry start_working [2024-10-16 14:58:16.495819] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [22472][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=2] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729057302398073002, v:0}, clog_base_lsn:{lsn:114212315136}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061062864990000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061909029444000, latest_log_ts:{val:1729061892483852001, v:0}}, {limited_id:201000001, latest_log_ts:{val:1729040770539777001, v:0}}, {limited_id:1570000001, latest_log_ts:{val:1729042351472860001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:33576, palf_env_:0x2b0e2ebb4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2024-10-16 14:58:16.496036] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2691) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=7] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", [SlidingWindow]={palf_id:1, self:"10.0.4.7:2890", lsn_allocator:{max_log_id:133533647, max_lsn:66390261972, max_scn:{val:1729061896250577001, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:33216076378}, reuse_lsn:{lsn:66390261972}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:66348318932}}, last_submit_lsn:{lsn:66390261850}, last_submit_end_lsn:{lsn:66390261972}, last_submit_log_id:133533647, last_submit_log_pid:48, max_flushed_lsn:{lsn:66390261850}, max_flushed_end_lsn:{lsn:66390261972}, max_flushed_log_pid:48, committed_end_lsn:{lsn:66390261972}, last_slide_log_id:133533647, last_slide_scn:{val:1729061896250577001, v:0}, last_slide_lsn:{lsn:66390261850}, last_slide_end_lsn:{lsn:66390261972}, last_slide_log_pid:48, last_slide_log_accum_checksum:1678962221, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x2b0de0acc530}, [StateMgr]={this:0x2b0de0ad0830, self:"10.0.4.7:2890", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, leader:"10.0.4.7:2890", leader_epoch:4, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:133533648, is_changing_config_with_arb:false, reconfirm_start_time_us:1722494261740374, palf_role_change_cb:0x2b0de0ad7230, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"10.0.4.7:2890", alive_paxos_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, reconfig_barrier:{prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_end_lsn:{lsn:33216819320}, prev_mode_pid:48}, persistent_config_version:{proposal_id:48, config_seq:11}, ms_ack_list:0, resend_config_version:{proposal_id:48, config_seq:11}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1722494260838312, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1722494260838312, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x2b0de0acdc70}, [ModeMgr]={palf_id:1, self:"10.0.4.7:2890", applied_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1722494261941355, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1728741813401825001, v:0}, min_block_id:965, min_block_min_scn:{val:1728727703672931001, v:0}, base_lsn_for_block_gc:{lsn:66098196480}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:48}, log_config_meta:{version:3, proposal_id:48, prev:{version:1, config:{config_version:{proposal_id:47, config_seq:10}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:48, config_seq:11}, log_sync_memberlist:3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:47, prev_lsn:{lsn:33216818297}, prev_mode_pid:48}, log_snapshot_meta:{version:1, base_lsn:{lsn:66098196480}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615, v:3}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:48, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:1634304}, readable_log_tail:{lsn:1634304}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66433720320}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/meta", dir_fd:128, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65470464, block_header_serialize_buf_:0x2b0de0ad29f8, flashback_version:0}, log_storage:{log_tail:{lsn:66390261972}, readable_log_tail:{lsn:66390261972}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:66366615552}, min_scn:{val:1729057321384354001, v:0}, curr_block_id:989, palf_id:1}, block_mgr:{log_dir:"/mnt/myoceanbase/oceanbase/data//clog/tenant_1/1/log", dir_fd:134, min_block_id:965, max_block_id:990, curr_writable_block_id:989}, logical_block_size_:67104768, curr_block_writable_size_:43458348, block_header_serialize_buf_:0x2b0de0ad46b8, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1722494261739251, this:0x2b0de0ad0f70}, [Reconfirm]={palf_id:1, self:"10.0.4.7:2890", state:"FINISHED", new_proposal_id:48, prepare_log_ack_list:2"10.0.4.8:2890";"10.0.4.9:2890";, curr_paxos_follower_list:2{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.0.4.7:2890", majority_max_accept_pid:47, majority_max_lsn:{lsn:33216819320}, saved_end_lsn:{lsn:33216819320}, last_submit_prepare_req_time_us:1722494261740552, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x2b0de0ad09b0}) [2024-10-16 14:58:16.496168] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2695) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=129] [PALF_DUMP](palf_id=1, self="10.0.4.7:2890", ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061896416512, last_flushed_end_lsn:{lsn:66390261972}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061896416723, last_flushed_end_lsn:{lsn:66390261972}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061896416663, last_flushed_end_lsn:{lsn:66390261972}}]) [2024-10-16 14:58:16.496179] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4296) [21448][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=9] current log_task status(palf_id=1, self="10.0.4.7:2890", log_id=133533648, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"10.0.4.7:2890", timestamp:1, flag:0}, last_ack_time_us:1729061896416512, last_flushed_end_lsn:{lsn:66390261972}}, {member:{server:"10.0.4.8:2890", timestamp:1, flag:0}, last_ack_time_us:1729061896416723, last_flushed_end_lsn:{lsn:66390261972}}, {member:{server:"10.0.4.9:2890", timestamp:1, flag:0}, last_ack_time_us:1729061896416663, last_flushed_end_lsn:{lsn:66390261972}}]) [2024-10-16 14:58:16.496741] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107618][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2F03-0-0] [lt=4][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178804, log_task={header:{begin_lsn:{lsn:14538530481}, end_lsn:{lsn:14538530603}, log_id:115178804, min_scn:{val:1729061896172130002, v:0}, max_scn:{val:1729061896172130002, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538530359}, prev_proposal_id:23, committed_end_lsn:{lsn:14538530481}, data_checksum:1715074492, accum_checksum:435777405, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061896396665, freeze_ts:1729061896396665, submit_ts:1729061896396670, flushed_ts:1729061896398142, gen_to_freeze cost time:0, gen_to_submit cost time:5, submit_to_flush cost time:1472}, fs_cb_begin_ts=1729061896496733, log_life_time=100068) [2024-10-16 14:58:16.496740] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:263) [21895][T1001_TenantWea][T1001][YB4A0A000407-00061E9B3A4EF36E-0-0] [lt=15] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=6.085253810882568359e+01, this=0x2b0df709f208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=511, Resource Limit(MB)=304, Throttle Trigger(MB)=182, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=2.032918832118009504e+03) [2024-10-16 14:58:16.496995] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061896493726000, start_id=1729061896493726000, end_id=1729061896493726001) [2024-10-16 14:58:16.497407] INFO [LIB] stat (utility.h:1165) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=249, this=0x2b0e43ccba70, extra_info="") [2024-10-16 14:58:16.497422] INFO [LIB] stat (utility.h:1182) [22612][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=128, this=0x2b0e0f9bb3b0, extra_info="") [2024-10-16 14:58:16.497447] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x2b0e43ccbca0, extra_info="") [2024-10-16 14:58:16.497695] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:831) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1001, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=33705, start_service_tstamp_=1722494263684720, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2024-10-16 14:58:16.498079] INFO [LIB] stat (utility.h:1165) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=41, stat_interval=1000000, avg (count/cost)=335, this=0x2b0e0f1f7a70, extra_info="") [2024-10-16 14:58:16.498091] INFO [LIB] stat (utility.h:1182) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=41, stat_interval=1000000, avg (count/cost)=226, this=0x2b0e0d9bac70, extra_info="") [2024-10-16 14:58:16.498111] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT FLUSH CB COST TIME](cur_stat_count=41, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0f1f7ca0, extra_info="") [2024-10-16 14:58:16.499282] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1083}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286044} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.500782] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1] get number(ret=0, service_type_=0, range=1, base_id=1729061896500035000, start_id=1729061896500035000, end_id=1729061896500035001) [2024-10-16 14:58:16.501901] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2AA2-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:16.501919] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22070][T1002_BalanceEx][T1002][YB4A0A000407-00061E9B387A2AA2-0-0] [lt=17] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:16.512852] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=59066) [2024-10-16 14:58:16.513412] INFO [PALF] read (log_cache.cpp:103) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=60689, hit_cnt=42, read_cnt=42, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:16.516892] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [21443][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=3, avg_log_gen_to_submit_time=36, avg_log_submit_to_flush_time=450, avg_log_submit_to_slide_time=1119) [2024-10-16 14:58:16.517865] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.526017] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=8] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1181712384, global_cache_size=1581383680, tenant_max_wash_size=7525990, tenant_min_wash_size=7525990, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:16.528857] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Cache wash clean map node details(ret=0, clean_node_count=15, clean_time=0, clean_start_pos=2201990, clean_num=62914) [2024-10-16 14:58:16.535191] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] ====== check clog disk timer task ====== [2024-10-16 14:58:16.535206] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] get_disk_usage(ret=0, capacity(MB):=5529, used(MB):=4386) [2024-10-16 14:58:16.535220] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [22238][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=1445195404, threshold=1739461755, need_update_checkpoint_scn=false) [2024-10-16 14:58:16.552636] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061896552177000, start_id=1729061896552177000, end_id=1729061896552177001) [2024-10-16 14:58:16.555812] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107618][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD624-0-0] [lt=20] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.0.4.7:2890", committed size=1220) [2024-10-16 14:58:16.558255] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=28] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:16.563116] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1082}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286057} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.565246] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1] get number(ret=0, service_type_=0, range=1, base_id=1729061896564595000, start_id=1729061896564595000, end_id=1729061896564595001) [2024-10-16 14:58:16.569182] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21293][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:16.569202] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21295][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:16.569214] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21294][RpcIO][T0][Y0-0000000000000000-0-0] [lt=3] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:16.569586] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:603) [21296][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-10-16 14:58:16.572706] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:142) [22697][T1004_TntTransf][T1004][YB4A0A000407-00061E9ABD5C893A-0-0] [lt=15] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2024-10-16 14:58:16.573888] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061896573791000, start_id=1729061896573791000, end_id=1729061896573791001) [2024-10-16 14:58:16.577777] INFO [SHARE] pre_run (ob_tenant_base.cpp:314) [107619][][T1003][Y0-0000000000000000-0-0] [lt=0] tenant thread pre_run(MTL_ID()=1003, ret=0, thread_count_=173) [2024-10-16 14:58:16.577797] INFO [RPC.OBRPC] th_init (ob_rpc_translator.cpp:33) [107619][][T1003][Y0-0000000000000000-0-0] [lt=19] Init thread local success [2024-10-16 14:58:16.578006] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:507) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=16] worker thread created(tenant_->id()=1003, group_id_=1, token=1) [2024-10-16 14:58:16.578622] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061896578144000, start_id=1729061896578144000, end_id=1729061896578144001) [2024-10-16 14:58:16.578851] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1729061896578845], gts:1729061896578144000, latest_srr:[mts=1729061896578845]}) [2024-10-16 14:58:16.584417] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [22448][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=19] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=21, total_destructed_detector=21, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x2b0e2c3fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2024-10-16 14:58:16.588079] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1210, avg_group_log_size=121) [2024-10-16 14:58:16.588101] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1210) [2024-10-16 14:58:16.589505] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4572-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1d56a0, extra_info="palf_id:1") [2024-10-16 14:58:16.589516] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F4572-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1041, this=0x2b0e0f1d58d0, extra_info="palf_id:1") [2024-10-16 14:58:16.591209] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=14] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1003, min_version={val:1729061896391401000, v:0}, base_version={val:1729061896190843000, v:0}, server_count=3, svr_array=[{addr:"10.0.4.7:2890", version:{val:1729061896391401000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061896591196, is_skipped:false}, {addr:"10.0.4.9:2890", version:{val:1729061896391401000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061896496074, is_skipped:false}, {addr:"10.0.4.8:2890", version:{val:1729061896391401000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1729061896541755, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2024-10-16 14:58:16.591340] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3275}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286066} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.592099] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1729061896592065000, start_id=1729061896592065000, end_id=1729061896592065001) [2024-10-16 14:58:16.592442] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.592734] INFO [OCCAM] get_idx (ob_occam_time_guard.h:223) [107619][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD382-0-0] [lt=4] init point thread id with(&point=0x55e3435cc740, idx_=1123, point=[thread id=107619, timeout ts=08:00:00.0, last click point="(null):(null):0", last click ts=08:00:00.0], thread_id=107619) [2024-10-16 14:58:16.595811] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C475A-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1001, self="10.0.4.7:2890", total_slide_log_cnt=43, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=28, avg_log_submit_to_flush_time=592, avg_log_submit_to_slide_time=879) [2024-10-16 14:58:16.596656] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061896596330000, start_id=1729061896596330000, end_id=1729061896596330001) [2024-10-16 14:58:16.598038] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107618][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2F05-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=11, avg_log_batch_cnt=0, total_group_log_size=1342, avg_group_log_size=122) [2024-10-16 14:58:16.598057] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107618][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2F05-0-0] [lt=12][errcode=-4389] log_task life cost too much time(palf_id=1002, self="10.0.4.7:2890", log_id=115178805, log_task={header:{begin_lsn:{lsn:14538530603}, end_lsn:{lsn:14538530725}, log_id:115178805, min_scn:{val:1729061896172130003, v:0}, max_scn:{val:1729061896172130003, v:0}, data_len:66, proposal_id:23, prev_lsn:{lsn:14538530481}, prev_proposal_id:23, committed_end_lsn:{lsn:14538530603}, data_checksum:1715074492, accum_checksum:1290086178, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061896496723, freeze_ts:1729061896496723, submit_ts:1729061896496727, flushed_ts:1729061896497432, gen_to_freeze cost time:0, gen_to_submit cost time:4, submit_to_flush cost time:705}, fs_cb_begin_ts=1729061896598051, log_life_time=101328) [2024-10-16 14:58:16.598549] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:16.598567] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22625][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:16.599224] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22254][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=12] start do ls ha handler(ls_id_array_=[{id:1002}, {id:1}, {id:1001}, {id:1003}]) [2024-10-16 14:58:16.599987] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3855}, to_seq_no={branch:0, seq:1087}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286067} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.612203] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B8382A0-0-0] [lt=25] [PALF STAT GROUP LOG INFO](palf_id=1003, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=51, avg_log_batch_cnt=0, total_group_log_size=78306, avg_group_log_size=1535) [2024-10-16 14:58:16.612458] INFO [LIB] stat (utility.h:1165) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT WRITE LOG COST TIME](cur_stat_count=51, stat_interval=1000000, avg (count/cost)=384, this=0x2b0e0db3da70, extra_info="") [2024-10-16 14:58:16.612476] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=78306) [2024-10-16 14:58:16.612487] INFO [LIB] stat (utility.h:1182) [22054][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=51, stat_interval=1000000, avg (count/cost)=161, this=0x2b0e0d9bbaf0, extra_info="") [2024-10-16 14:58:16.612521] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=51, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e0db3dca0, extra_info="") [2024-10-16 14:58:16.612952] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22794][T1004_ReqMemEvi][T1004][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=6089, mem_used=12478464) [2024-10-16 14:58:16.617930] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.618211] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5284}, to_seq_no={branch:0, seq:1047}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286070} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.624940] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=44, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e0dac1470, extra_info="") [2024-10-16 14:58:16.625193] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=66943) [2024-10-16 14:58:16.627617] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3180}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286072} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.627736] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=5] Cache replace map node details(ret=0, replace_node_count=48, replace_time=0, replace_start_pos=314570, replace_num=31457) [2024-10-16 14:58:16.627750] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=48, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:16.632763] INFO [PALF] next (palf_iterator.h:170) [22150][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:139561670, miss_cnt:1, cache_read_size:152590897719, hit_ratio:"9.999999928347088618e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:1, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:1, read_io_size:3678208, read_disk_cost_ts:10947, start_lsn:{lsn:60461395968}}}) [2024-10-16 14:58:16.635519] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=15] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601542, offset=9664392}, end_cursor:ObLogCursor{file_id=192, log_id=47601543, offset=9664676}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2014050, len:2168, log_data_len:284, seq:47601542, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:16.637572] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22067][T1002_TeRec][T1001][YB4A0A000407-00061E9B225FA4C6-0-0] [lt=42] get number(ret=0, service_type_=0, range=1, base_id=1729061896635645000, start_id=1729061896635645000, end_id=1729061896635645001) [2024-10-16 14:58:16.638500] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601543, offset=9664676}, end_cursor:ObLogCursor{file_id=192, log_id=47601544, offset=9664950}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21d8050, len:1884, log_data_len:274, seq:47601543, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:16.643372] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601544, offset=9664950}, end_cursor:ObLogCursor{file_id=192, log_id=47601545, offset=9665224}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2308050, len:1610, log_data_len:274, seq:47601544, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:16.643580] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22144][T1002_OB_SLOG][T1002][Y0-0000000000000000-0-0] [lt=25] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=192, log_id=47601545, offset=9665224}, end_cursor:ObLogCursor{file_id=192, log_id=47601546, offset=9665498}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd23d0050, len:1336, log_data_len:274, seq:47601545, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:16.643646] INFO [PALF] next (palf_iterator.h:170) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:137599262, miss_cnt:9, cache_read_size:149778134003, hit_ratio:"9.999999345926767758e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:9, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:16, read_io_size:31817728, read_disk_cost_ts:299057, start_lsn:{lsn:57777205248}}}) [2024-10-16 14:58:16.645614] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=12] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257890, offset=39069159}, end_cursor:ObLogCursor{file_id=197, log_id=49257891, offset=39069442}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd2188050, len:2585, log_data_len:283, seq:49257890, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:16.647804] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22628][T1004_TeRec][T1003][YB4A0A000407-00061E9B221FB391-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1729061896647354000, start_id=1729061896647354000, end_id=1729061896647354001) [2024-10-16 14:58:16.648353] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=20] get number(ret=0, service_type_=0, range=1, base_id=1729061896647354000, start_id=1729061896647354000, end_id=1729061896647354001) [2024-10-16 14:58:16.649940] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257891, offset=39069442}, end_cursor:ObLogCursor{file_id=197, log_id=49257892, offset=39069724}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21c8050, len:2302, log_data_len:282, seq:49257891, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:16.650198] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2356}, to_seq_no={branch:0, seq:1203}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286078} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.653802] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257892, offset=39069724}, end_cursor:ObLogCursor{file_id=197, log_id=49257893, offset=39070006}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21d4050, len:2020, log_data_len:282, seq:49257892, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:16.654108] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [22706][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=35] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=197, log_id=49257893, offset=39070006}, end_cursor:ObLogCursor{file_id=197, log_id=49257894, offset=39070288}, is_inited:true, is_local:false, buf_size:8192, buf:0x2b0dd21dc050, len:1738, log_data_len:282, seq:49257893, flush_finish:false, flush_ret:0}) [2024-10-16 14:58:16.654201] INFO [PALF] next (palf_iterator.h:170) [22823][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1002, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62650562, miss_cnt:32, cache_read_size:8050459932, hit_ratio:"9.999994892307006999e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:32, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:60, read_io_size:113926144, read_disk_cost_ts:1916262, start_lsn:{lsn:6374952960}}}) [2024-10-16 14:58:16.654213] INFO [PALF] next (palf_iterator.h:170) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT ITERATOR INFO](io_ctx_={user:"REPLAY", palf_id_:1003, iterator_info_:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:62657302, miss_cnt:17, cache_read_size:8050207435, hit_ratio:"9.999997286829332976e-01"}, cold_cache_stat:{hit_cnt:0, miss_cnt:17, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:32, read_io_size:61861888, read_disk_cost_ts:1562327, start_lsn:{lsn:6240743424}}}) [2024-10-16 14:58:16.666992] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2153}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286082} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.669417] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1729061896668986000, start_id=1729061896668986000, end_id=1729061896668986001) [2024-10-16 14:58:16.670851] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1091}, to_seq_no={branch:0, seq:1086}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286083} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.674082] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0DB6-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061896673240000, start_id=1729061896673240000, end_id=1729061896673240001) [2024-10-16 14:58:16.674262] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0DB3-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061896673240000, start_id=1729061896673240000, end_id=1729061896673240001) [2024-10-16 14:58:16.674289] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0DB4-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061896673240000, start_id=1729061896673240000, end_id=1729061896673240001) [2024-10-16 14:58:16.674318] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0DB7-0-0] [lt=5] get number(ret=0, service_type_=0, range=1, base_id=1729061896674311000, start_id=1729061896674311000, end_id=1729061896674311001) [2024-10-16 14:58:16.674953] INFO [PALF] read (log_cache.cpp:103) [22712][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:16.677643] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22471][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=18] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:16.679100] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2517}, to_seq_no={branch:0, seq:1460}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286086} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.684570] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3165}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286087} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.688147] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=21, this=0x2b0e0f1d5470, extra_info="") [2024-10-16 14:58:16.688178] INFO [LIB] stat (utility.h:1165) [22255][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e260a3210, extra_info="id:1") [2024-10-16 14:58:16.688330] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=49, avg_log_batch_cnt=1, total_group_log_size=75061, avg_group_log_size=1531) [2024-10-16 14:58:16.688342] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=73559) [2024-10-16 14:58:16.688408] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=976) [2024-10-16 14:58:16.688435] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=976) [2024-10-16 14:58:16.688762] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4769-0-0] [lt=16] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=49, stat_interval=1000000, avg (count/cost)=4, this=0x2b0e0f1ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:16.688777] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B806C4769-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=49, stat_interval=1000000, avg (count/cost)=908, this=0x2b0e0f1ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:16.692504] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.695451] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=107, stat_interval=1000000, avg (count/cost)=34, this=0x2b0e0c2b08f8, extra_info="") [2024-10-16 14:58:16.695464] INFO [LIB] stat (utility.h:1165) [22263][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=107, stat_interval=1000000, avg (count/cost)=91, this=0x2b0e0c2b0b28, extra_info="") [2024-10-16 14:58:16.696657] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2129}, to_seq_no={branch:0, seq:1057}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286089} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.696896] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061896695708000, start_id=1729061896695708000, end_id=1729061896695708001) [2024-10-16 14:58:16.697757] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:336) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1001, new_min_version={val:1729061896449420000, v:0}, new_max_version={val:1729061896449420000, v:0}, last_min_version={val:1729061896258099000, v:0}, last_max_version={val:1729061896258099000, v:0}, total_time_us=1749, delta_us=248336) [2024-10-16 14:58:16.697777] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:695) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1001, current_version={val:1729061896449420000, v:0}, min_version={val:1729061896449420000, v:0}, max_version={val:1729061896449420000, v:0}, in_service=true, leader_epoch=33705) [2024-10-16 14:58:16.697826] INFO [STORAGE] runTimerTask (ob_locality_manager.cpp:708) [21260][LocalityReload][T0][Y0-0000000000000000-0-0] [lt=9] runTimer to refresh locality_info(ret=0) [2024-10-16 14:58:16.698698] INFO [SHARE] set_server_locality_array (ob_server_locality_cache.cpp:346) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=11] set_server_locality_array success(server_locality_array=[{inited:true, addr:"10.0.4.7:2890", zone:"myoceanbase1", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494256558000, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.8:2890", zone:"myoceanbase2", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494430723273, server_stop_time:0, server_status:1}, {inited:true, addr:"10.0.4.9:2890", zone:"myoceanbase3", zone_type:0, idc:"", region:"sys_region", is_active:true, start_service_time:1722494593087951, server_stop_time:0, server_status:1}], has_readonly_zone=false) [2024-10-16 14:58:16.698722] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=20] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1, primary_zone=RANDOM) [2024-10-16 14:58:16.698728] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=6] add_locality_zone success(locality_zone={tenant_id:1, region_priority:18446744073709551615}) [2024-10-16 14:58:16.698736] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=5] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1001, primary_zone=RANDOM) [2024-10-16 14:58:16.698738] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1001, region_priority:18446744073709551615}) [2024-10-16 14:58:16.698745] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1002, primary_zone=RANDOM) [2024-10-16 14:58:16.698748] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1002, region_priority:18446744073709551615}) [2024-10-16 14:58:16.698752] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1003, primary_zone=RANDOM) [2024-10-16 14:58:16.698754] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] add_locality_zone success(locality_zone={tenant_id:1003, region_priority:18446744073709551615}) [2024-10-16 14:58:16.698758] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1004, primary_zone=RANDOM) [2024-10-16 14:58:16.698760] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=2] add_locality_zone success(locality_zone={tenant_id:1004, region_priority:18446744073709551615}) [2024-10-16 14:58:16.698764] INFO [SHARE] load_region (ob_locality_table_operator.cpp:373) [21259][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] load region(ret=0, locality_info={version:0, local_region:"sys_region", local_zone:"myoceanbase1", local_idc:"", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["myoceanbase1", "myoceanbase2", "myoceanbase3"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:18446744073709551615}, {tenant_id:1002, region_priority:18446744073709551615}, {tenant_id:1003, region_priority:18446744073709551615}, {tenant_id:1004, region_priority:18446744073709551615}]}) [2024-10-16 14:58:16.700804] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061896700297000, start_id=1729061896700297000, end_id=1729061896700297001) [2024-10-16 14:58:16.701349] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1024}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286090} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.704202] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061896703410000, start_id=1729061896703410000, end_id=1729061896703410001) [2024-10-16 14:58:16.705251] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [21901][T1001_LSArchive][T1001][YB4A0A000407-00061E996D8960C1-0-0] [lt=14] gc stale ls task succ [2024-10-16 14:58:16.706956] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286091} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.716071] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D56CF-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=57, stat_interval=1000000, avg (count/cost)=6, this=0x2b0e0db336a0, extra_info="palf_id:1003") [2024-10-16 14:58:16.716087] INFO [LIB] stat (utility.h:1165) [107580][T1002_L0_G1][T1002][YB4A0A000407-00061E9B3B4D56CF-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=57, stat_interval=1000000, avg (count/cost)=7449, this=0x2b0e0db338d0, extra_info="palf_id:1003") [2024-10-16 14:58:16.717218] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [21607][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=3779) [2024-10-16 14:58:16.717996] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.719577] INFO [COMMON] generate_mod_stat_task (memory_dump.cpp:218) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] task info(*task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:58:16.719591] INFO [COMMON] handle (memory_dump.cpp:550) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=14] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-10-16 14:58:16.720438] INFO [COMMON] update_check_range (ob_sql_mem_leak_checker.cpp:62) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] update_check_range(min_check_version=652902, max_check_version=652903, global_version=652904) [2024-10-16 14:58:16.722642] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107619][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DE-0-0] [lt=16] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061896712139, keepalive_ts:-1}) [2024-10-16 14:58:16.722657] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107619][T1003_L0_G1][T1003][YB4A0A000408-00061E9B4A7E79DE-0-0] [lt=13] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.8:2890", region:"sys_region", register_time_us:1729061896712139, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:16.723264] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1002, self="10.0.4.7:2890", role="FOLLOWER", total_group_log_cnt=50, avg_log_batch_cnt=0, total_group_log_size=78188, avg_group_log_size=1563) [2024-10-16 14:58:16.726548] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286095} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.728065] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF STAT WRITE LOG COST TIME](cur_stat_count=50, stat_interval=1000000, avg (count/cost)=246, this=0x2b0e0dacba70, extra_info="") [2024-10-16 14:58:16.728082] INFO [LIB] stat (utility.h:1182) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=50, stat_interval=1000000, avg (count/cost)=132, this=0x2b0e0d9bb3b0, extra_info="") [2024-10-16 14:58:16.729168] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-1177518080, global_cache_size=1581383680, tenant_max_wash_size=9623142, tenant_min_wash_size=9623142, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:16.729444] INFO [COMMON] wash (ob_kvcache_store.cpp:349) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash memory, (tenant_id=1002, cache_size=87392256, lower_mem_limit=1073741824, upper_mem_limit=1073741824, min_wash_size=9623142, max_wash_size=9623142, mem_usage=975990784, reserve_mem=107374182, wash_size=9623142) [2024-10-16 14:58:16.729565] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Wash time detail, (compute_wash_size_time=115, refresh_score_time=68, wash_time=202) [2024-10-16 14:58:16.732403] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:636) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=5] Cache wash clean map node details(ret=0, clean_node_count=15, clean_time=0, clean_start_pos=2264904, clean_num=62914) [2024-10-16 14:58:16.734298] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21606][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:16.740101] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2141}, to_seq_no={branch:0, seq:1069}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286096} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.750933] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21683][T1001_Occam][T1001][YB4A0A000407-00061E9B362A15FA-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:16.756740] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061896755850000, start_id=1729061896755850000, end_id=1729061896755850001) [2024-10-16 14:58:16.757035] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1001, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:16.757051] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:16.758424] INFO handle (memory_dump.cpp:677) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 206 259 0 1 2 117 1 0 1 4 1 1 0 1 5 38 4 0 1 7 1 2 0 1 8 65 1 0 1 12 18 3 0 1 13 6 1 0 1 14 1 2 0 1 15 1 1 0 1 18 26 3 0 1 20 2 2 0 1 22 1 1 0 500 0 139 249 0 500 7 7 4 0 500 8 50 1 0 500 9 2 1 0 500 10 10 2 0 500 16 3 1 0 500 17 10 7 0 500 22 111 63 0 500 23 61 11 0 508 0 4 13 0 508 8 5 1 0 509 0 3 8 0 509 8 5 1 0 1001 0 132 232 0 1001 2 22 1 0 1001 4 1 1 0 1001 5 23 4 0 1001 8 44 1 0 1001 12 10 3 0 1001 13 3 1 0 1001 14 1 2 0 1001 18 14 3 0 1001 20 1 2 0 1002 0 176 258 0 1002 2 22 1 0 1002 4 2 1 0 1002 5 24 4 0 1002 8 49 1 0 1002 12 19 3 0 1002 13 7 1 0 1002 14 1 2 0 1002 18 14 3 0 1002 20 7 2 0 1003 0 137 235 0 1003 2 29 1 0 1003 4 1 1 0 1003 5 23 4 0 1003 8 44 1 0 1003 12 10 3 0 1003 13 3 1 0 1003 14 1 2 0 1003 18 13 3 0 1003 20 1 2 0 1004 0 292 256 0 1004 2 19 1 0 1004 4 2 1 0 1004 5 34 4 0 1004 8 52 1 0 1004 12 19 3 0 1004 13 10 1 0 1004 14 2 2 0 1004 15 2 1 0 1004 16 2 2 0 1004 18 25 3 0 1004 20 11 2 0 cost_time: 38017 [2024-10-16 14:58:16.758454] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=11] [MEMORY] tenant: 1, limit: 2,147,483,648 hold: 1,988,595,712 rpc_hold: 0 cache_hold: 861,929,472 cache_used: 861,929,472 cache_item_count: 411 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 543,637,504 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MEMSTORE_CTX_ID hold_bytes= 245,366,784 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TRANS_CTX_MGR_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PLAN_CACHE_CTX_ID hold_bytes= 79,691,776 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 136,314,880 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= META_OBJ_CTX_ID hold_bytes= 37,748,736 limit= 429,496,720 [MEMORY] ctx_id= TX_CALLBACK_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOB_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PS_CACHE_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= RPC_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= TX_DATA_TABLE hold_bytes= 54,546,432 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= MDS_DATA_ID hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [2024-10-16 14:58:16.758674] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7] [MEMORY] tenant_id= 1 ctx_id= DEFAULT_CTX_ID hold= 543,637,504 used= 431,011,728 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 32 [MEMORY] hold= 56,180,736 used= 56,153,088 count= 27 avg_used= 2,079,744 block_cnt= 27 chunk_cnt= 27 mod=CACHE_MAP_NODE [MEMORY] hold= 41,963,520 used= 41,943,040 count= 1 avg_used= 41,943,040 block_cnt= 1 chunk_cnt= 1 mod=LogGroupBuffer [MEMORY] hold= 30,965,760 used= 27,525,120 count= 420 avg_used= 65,536 block_cnt= 420 chunk_cnt= 50 mod=MinorMergeMgr [MEMORY] hold= 23,511,040 used= 23,433,792 count= 8 avg_used= 2,929,224 block_cnt= 8 chunk_cnt= 7 mod=IoControl [MEMORY] hold= 22,880,256 used= 21,568,040 count= 161 avg_used= 133,962 block_cnt= 161 chunk_cnt= 38 mod=SqlPlanMon [MEMORY] hold= 18,874,368 used= 16,967,424 count= 256 avg_used= 66,279 block_cnt= 256 chunk_cnt= 63 mod=SqlDtlBuf [MEMORY] hold= 17,694,720 used= 16,942,080 count= 240 avg_used= 70,592 block_cnt= 240 chunk_cnt= 31 mod=[T]ObSessionDIB [MEMORY] hold= 16,797,696 used= 16,777,336 count= 1 avg_used= 16,777,336 block_cnt= 1 chunk_cnt= 1 mod=MrgMemPoolChk [MEMORY] hold= 15,034,848 used= 14,886,881 count= 98 avg_used= 151,906 block_cnt= 31 chunk_cnt= 7 mod=OmtTenant [MEMORY] hold= 11,771,904 used= 9,838,592 count= 236 avg_used= 41,688 block_cnt= 236 chunk_cnt= 61 mod=[T]char [MEMORY] hold= 11,223,040 used= 11,209,664 count= 7 avg_used= 1,601,380 block_cnt= 7 chunk_cnt= 7 mod=MysqlRequesReco [MEMORY] hold= 10,911,744 used= 9,699,328 count= 148 avg_used= 65,536 block_cnt= 148 chunk_cnt= 38 mod=MajorMergeMgr [MEMORY] hold= 8,777,728 used= 8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue [MEMORY] hold= 8,540,160 used= 8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv [MEMORY] hold= 8,409,088 used= 8,388,640 count= 1 avg_used= 8,388,640 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 6,094,848 used= 6,033,408 count= 8 avg_used= 754,176 block_cnt= 8 chunk_cnt= 7 mod=ResourceGroup [MEMORY] hold= 4,964,352 used= 4,959,488 count= 3 avg_used= 1,653,162 block_cnt= 3 chunk_cnt= 3 mod=SqlFltSpanRec [MEMORY] hold= 4,943,872 used= 4,915,456 count= 2 avg_used= 2,457,728 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe [MEMORY] hold= 4,235,264 used= 4,194,304 count= 2 avg_used= 2,097,152 block_cnt= 2 chunk_cnt= 2 mod=SHARE_BLOCK [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMem [MEMORY] hold= 3,865,600 used= 3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 12 mod=CkptDgnMemCU [MEMORY] hold= 3,772,416 used= 3,751,936 count= 1 avg_used= 3,751,936 block_cnt= 1 chunk_cnt= 1 mod=DiskIteratorSto [MEMORY] hold= 3,723,264 used= 3,698,688 count= 2 avg_used= 1,849,344 block_cnt= 2 chunk_cnt= 2 mod=LogDIOAligned [MEMORY] hold= 3,162,112 used= 3,145,808 count= 2 avg_used= 1,572,904 block_cnt= 2 chunk_cnt= 2 mod=TenantDLMgr [MEMORY] hold= 3,145,728 used= 2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 5 mod=SqlDtlQueue [MEMORY] hold= 2,828,928 used= 2,708,352 count= 604 avg_used= 4,484 block_cnt= 576 chunk_cnt= 72 mod=WorkerMap [MEMORY] hold= 2,777,088 used= 2,760,192 count= 1 avg_used= 2,760,192 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo [MEMORY] hold= 2,415,648 used= 2,387,720 count= 104 avg_used= 22,958 block_cnt= 104 chunk_cnt= 4 mod=TabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=FastFrezCkr [MEMORY] hold= 2,375,680 used= 2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta [MEMORY] hold= 2,248,704 used= 2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb [MEMORY] hold= 2,198,864 used= 416,700 count= 25,277 avg_used= 16 block_cnt= 270 chunk_cnt= 2 mod=Number [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=LogReplayTask [MEMORY] hold= 1,843,200 used= 1,770,696 count= 9 avg_used= 196,744 block_cnt= 9 chunk_cnt= 2 mod=DagMap [MEMORY] hold= 1,777,664 used= 1,770,696 count= 1 avg_used= 1,770,696 block_cnt= 1 chunk_cnt= 1 mod=TableProc [MEMORY] hold= 1,548,288 used= 1,507,118 count= 5 avg_used= 301,423 block_cnt= 5 chunk_cnt= 1 mod=LDIOSetup [MEMORY] hold= 1,376,256 used= 1,322,752 count= 8 avg_used= 165,344 block_cnt= 8 chunk_cnt= 7 mod=SQLSessionInfo [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=TransTimeWheel [MEMORY] hold= 1,294,336 used= 1,280,384 count= 2 avg_used= 640,192 block_cnt= 2 chunk_cnt= 1 mod=XATimeWheel [MEMORY] hold= 1,261,568 used= 681,912 count= 77 avg_used= 8,856 block_cnt= 77 chunk_cnt= 31 mod=[T]ObDSActionAr [MEMORY] hold= 1,196,032 used= 1,182,720 count= 2 avg_used= 591,360 block_cnt= 2 chunk_cnt= 2 mod=ServerObjecPool [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsInfo [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPsCache [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=HashBuckPlanCac [MEMORY] hold= 1,187,840 used= 1,179,768 count= 1 avg_used= 1,179,768 block_cnt= 1 chunk_cnt= 1 mod=RewriteRuleMap [MEMORY] hold= 1,032,192 used= 924,784 count= 14 avg_used= 66,056 block_cnt= 14 chunk_cnt= 14 mod=CommSysVarFac [MEMORY] hold= 1,024,000 used= 1,015,808 count= 1 avg_used= 1,015,808 block_cnt= 1 chunk_cnt= 1 mod=CommitVersions [MEMORY] hold= 983,488 used= 960,256 count= 121 avg_used= 7,936 block_cnt= 121 chunk_cnt= 9 mod=TltMemtablMgr [MEMORY] hold= 958,464 used= 950,272 count= 1 avg_used= 950,272 block_cnt= 1 chunk_cnt= 1 mod=IOWorkerLQ [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=ArcSenderQueue [MEMORY] hold= 933,888 used= 931,072 count= 1 avg_used= 931,072 block_cnt= 1 chunk_cnt= 1 mod=TransService [MEMORY] hold= 802,816 used= 394,352 count= 49 avg_used= 8,048 block_cnt= 49 chunk_cnt= 21 mod=[T]ObPerfEventR [MEMORY] hold= 798,400 used= 779,200 count= 100 avg_used= 7,792 block_cnt= 100 chunk_cnt= 20 mod=CkptDgnMemNode [MEMORY] hold= 798,112 used= 789,624 count= 3 avg_used= 263,208 block_cnt= 2 chunk_cnt= 2 mod=PartitTableTask [MEMORY] hold= 786,688 used= 524,352 count= 33 avg_used= 15,889 block_cnt= 33 chunk_cnt= 4 mod=LogAlloc [MEMORY] hold= 765,440 used= 730,112 count= 184 avg_used= 3,968 block_cnt= 184 chunk_cnt= 44 mod=CkpDgn [MEMORY] hold= 663,552 used= 659,200 count= 1 avg_used= 659,200 block_cnt= 1 chunk_cnt= 1 mod=MulLevelQueue [MEMORY] hold= 663,552 used= 655,360 count= 1 avg_used= 655,360 block_cnt= 1 chunk_cnt= 1 mod=FetchLog [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=FrzTrigger [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MultiVersionGC [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTF [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DetectorTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=ElectTimer [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=CoordTR [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=MdsT [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=DupTbLease [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=OBJLockGC [MEMORY] hold= 598,016 used= 590,232 count= 1 avg_used= 590,232 block_cnt= 1 chunk_cnt= 1 mod=DagNetIdMap [MEMORY] hold= 458,752 used= 440,928 count= 3 avg_used= 146,976 block_cnt= 3 chunk_cnt= 3 mod=bf_queue [MEMORY] hold= 421,504 used= 404,416 count= 89 avg_used= 4,544 block_cnt= 89 chunk_cnt= 32 mod=[T]ObTraceEvent [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ReplaySrv [MEMORY] hold= 409,600 used= 401,408 count= 1 avg_used= 401,408 block_cnt= 1 chunk_cnt= 1 mod=ApplySrv [MEMORY] hold= 406,400 used= 396,800 count= 50 avg_used= 7,936 block_cnt= 50 chunk_cnt= 29 mod=LinearHashMapNo [MEMORY] hold= 399,200 used= 389,600 count= 50 avg_used= 7,792 block_cnt= 50 chunk_cnt= 32 mod=CkptDgnCUNode [MEMORY] hold= 385,024 used= 377,120 count= 1 avg_used= 377,120 block_cnt= 1 chunk_cnt= 1 mod=ClogGe [MEMORY] hold= 338,112 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 44 chunk_cnt= 2 mod=LSMap [MEMORY] hold= 311,296 used= 262,144 count= 256 avg_used= 1,024 block_cnt= 44 chunk_cnt= 1 mod=TabletToLS [MEMORY] hold= 311,104 used= 303,904 count= 6 avg_used= 50,650 block_cnt= 6 chunk_cnt= 2 mod=PoolFreeList [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompactMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompStMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=DmlStatHashMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 294,912 used= 261,632 count= 4 avg_used= 65,408 block_cnt= 4 chunk_cnt= 3 mod=MemTblObj [MEMORY] hold= 268,224 used= 261,888 count= 33 avg_used= 7,936 block_cnt= 33 chunk_cnt= 18 mod=CtxBlockPl [MEMORY] hold= 260,224 used= 253,952 count= 31 avg_used= 8,192 block_cnt= 31 chunk_cnt= 22 mod=ParamCharsType [MEMORY] hold= 227,584 used= 222,208 count= 28 avg_used= 7,936 block_cnt= 28 chunk_cnt= 22 mod=SqlSession [MEMORY] hold= 212,992 used= 212,160 count= 1 avg_used= 212,160 block_cnt= 1 chunk_cnt= 1 mod=MetaMemMgr [MEMORY] hold= 211,328 used= 206,336 count= 26 avg_used= 7,936 block_cnt= 26 chunk_cnt= 19 mod=QueryAllocator [MEMORY] hold= 204,800 used= 196,744 count= 1 avg_used= 196,744 block_cnt= 1 chunk_cnt= 1 mod=DagNetMap [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=ResourMapLock [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=T3MBucket [MEMORY] hold= 204,800 used= 196,616 count= 1 avg_used= 196,616 block_cnt= 1 chunk_cnt= 1 mod=TenDLBucket [MEMORY] hold= 163,776 used= 156,741 count= 2 avg_used= 78,370 block_cnt= 2 chunk_cnt= 2 mod=CsSstableReader [MEMORY] hold= 147,456 used= 139,264 count= 1 avg_used= 139,264 block_cnt= 1 chunk_cnt= 1 mod=RFLTaskQueue [MEMORY] hold= 147,456 used= 138,240 count= 2 avg_used= 69,120 block_cnt= 2 chunk_cnt= 1 mod=LDBlockBitMap [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 11 mod=RawPl [MEMORY] hold= 128,576 used= 33,168 count= 471 avg_used= 70 block_cnt= 175 chunk_cnt= 32 mod=Coro [MEMORY] hold= 128,000 used= 124,928 count= 16 avg_used= 7,808 block_cnt= 16 chunk_cnt= 2 mod=TabletIDSetNode [MEMORY] hold= 122,880 used= 110,608 count= 2 avg_used= 55,304 block_cnt= 2 chunk_cnt= 1 mod=TabletIDSetBkt [MEMORY] hold= 121,920 used= 119,040 count= 15 avg_used= 7,936 block_cnt= 15 chunk_cnt= 11 mod=SqlSessiVarMap [MEMORY] hold= 114,688 used= 111,096 count= 1 avg_used= 111,096 block_cnt= 1 chunk_cnt= 1 mod=AsyncFreezeTabl [MEMORY] hold= 114,688 used= 106,496 count= 1 avg_used= 106,496 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlMgr [MEMORY] hold= 113,792 used= 111,104 count= 14 avg_used= 7,936 block_cnt= 14 chunk_cnt= 11 mod=PlanVaIdx [MEMORY] hold= 106,496 used= 98,432 count= 1 avg_used= 98,432 block_cnt= 1 chunk_cnt= 1 mod=tenant_mf_mgr [MEMORY] hold= 106,496 used= 98,528 count= 1 avg_used= 98,528 block_cnt= 1 chunk_cnt= 1 mod=LSSvr [MEMORY] hold= 105,664 used= 103,168 count= 13 avg_used= 7,936 block_cnt= 13 chunk_cnt= 11 mod=DictPl [MEMORY] hold= 89,408 used= 87,296 count= 11 avg_used= 7,936 block_cnt= 11 chunk_cnt= 10 mod=RlePl [MEMORY] hold= 81,920 used= 73,896 count= 1 avg_used= 73,896 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairCache [MEMORY] hold= 81,280 used= 79,360 count= 10 avg_used= 7,936 block_cnt= 10 chunk_cnt= 10 mod=ConstPl [MEMORY] hold= 77,824 used= 65,536 count= 64 avg_used= 1,024 block_cnt= 12 chunk_cnt= 3 mod=TxCtxMgr [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushLog [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=DEVICE_MANAGER [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=StorMetaCacheIO [MEMORY] hold= 73,728 used= 72,736 count= 1 avg_used= 72,736 block_cnt= 1 chunk_cnt= 1 mod=LogSharedQueueT [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=FlushMeta [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxDataMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=TxCtxMemObj [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LockMemObj [MEMORY] hold= 73,728 used= 66,176 count= 1 avg_used= 66,176 block_cnt= 1 chunk_cnt= 1 mod=Rpc [MEMORY] hold= 73,728 used= 65,536 count= 1 avg_used= 65,536 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoMgr [MEMORY] hold= 73,152 used= 71,424 count= 9 avg_used= 7,936 block_cnt= 9 chunk_cnt= 9 mod=IntDiffPl [MEMORY] hold= 72,240 used= 1,024 count= 896 avg_used= 1 block_cnt= 10 chunk_cnt= 1 mod=CharsetUtil [MEMORY] hold= 65,216 used= 60,712 count= 4 avg_used= 15,178 block_cnt= 4 chunk_cnt= 4 mod=HB_SERVICE [MEMORY] hold= 65,024 used= 63,488 count= 8 avg_used= 7,936 block_cnt= 8 chunk_cnt= 8 mod=MdsTableMgr [MEMORY] hold= 64,000 used= 62,464 count= 8 avg_used= 7,808 block_cnt= 8 chunk_cnt= 8 mod=HashNodeLCSta [MEMORY] hold= 58,400 used= 55,296 count= 16 avg_used= 3,456 block_cnt= 11 chunk_cnt= 8 mod=CACHE_INST [MEMORY] hold= 57,344 used= 50,496 count= 1 avg_used= 50,496 block_cnt= 1 chunk_cnt= 1 mod=palf_env [MEMORY] hold= 54,000 used= 5,760 count= 240 avg_used= 24 block_cnt= 146 chunk_cnt= 27 mod=[T]MemoryContex [MEMORY] hold= 48,960 used= 44,840 count= 2 avg_used= 22,420 block_cnt= 2 chunk_cnt= 2 mod=DynamicFactor [MEMORY] hold= 48,880 used= 44,760 count= 2 avg_used= 22,380 block_cnt= 2 chunk_cnt= 2 mod=Autoincrement [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 4 mod=HexStrPl [MEMORY] hold= 48,768 used= 47,616 count= 6 avg_used= 7,936 block_cnt= 6 chunk_cnt= 5 mod=ColEqualPl [MEMORY] hold= 45,792 used= 35,616 count= 53 avg_used= 672 block_cnt= 51 chunk_cnt= 23 mod=[T]ObWarningBuf [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SuspectInfoBkt [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DagWarnHisBkt [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 1 mod=ObDMMDL [MEMORY] hold= 40,640 used= 39,680 count= 5 avg_used= 7,936 block_cnt= 5 chunk_cnt= 5 mod=StrDiffPl [MEMORY] hold= 32,896 used= 22,464 count= 54 avg_used= 416 block_cnt= 24 chunk_cnt= 7 mod=OMT_Worker [MEMORY] hold= 32,768 used= 17,408 count= 2 avg_used= 8,704 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTTP [MEMORY] hold= 32,768 used= 25,664 count= 1 avg_used= 25,664 block_cnt= 1 chunk_cnt= 1 mod=TSQLSessionMgr [MEMORY] hold= 32,768 used= 24,688 count= 2 avg_used= 12,344 block_cnt= 2 chunk_cnt= 2 mod=TLD_AssignedMgr [MEMORY] hold= 32,704 used= 26,079 count= 2 avg_used= 13,039 block_cnt= 2 chunk_cnt= 2 mod=CGReadInfoMgr [MEMORY] hold= 32,512 used= 31,744 count= 4 avg_used= 7,936 block_cnt= 4 chunk_cnt= 3 mod=StrPrefixPl [MEMORY] hold= 32,000 used= 31,232 count= 4 avg_used= 7,808 block_cnt= 4 chunk_cnt= 4 mod=HashNodePlanCac [MEMORY] hold= 24,576 used= 22,592 count= 1 avg_used= 22,592 block_cnt= 1 chunk_cnt= 1 mod=IO_MGR [MEMORY] hold= 24,576 used= 16,384 count= 1 avg_used= 16,384 block_cnt= 1 chunk_cnt= 1 mod=SlogWriteBuffer [MEMORY] hold= 24,368 used= 20,136 count= 2 avg_used= 10,068 block_cnt= 2 chunk_cnt= 2 mod=TabletStats [MEMORY] hold= 24,336 used= 20,104 count= 2 avg_used= 10,052 block_cnt= 2 chunk_cnt= 2 mod=TLD_ResourceMgr [MEMORY] hold= 21,152 used= 16,896 count= 22 avg_used= 768 block_cnt= 15 chunk_cnt= 12 mod=TGTimer [MEMORY] hold= 17,056 used= 8,448 count= 3 avg_used= 2,816 block_cnt= 2 chunk_cnt= 2 mod=BaseLogWriter [MEMORY] hold= 16,384 used= 11,584 count= 1 avg_used= 11,584 block_cnt= 1 chunk_cnt= 1 mod=LogApplyStatus [MEMORY] hold= 16,384 used= 10,248 count= 1 avg_used= 10,248 block_cnt= 1 chunk_cnt= 1 mod=TabletSvrBucket [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_ClientTask [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MedCheckSet [MEMORY] hold= 16,384 used= 13,824 count= 1 avg_used= 13,824 block_cnt= 1 chunk_cnt= 1 mod=LogReplayStatus [MEMORY] hold= 16,384 used= 8,448 count= 1 avg_used= 8,448 block_cnt= 1 chunk_cnt= 1 mod=PalfEnv [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=DiaTabletMap [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TLD_TableCtxMgr [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=SlogNopLog [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 2 mod=ColSubStrPl [MEMORY] hold= 15,648 used= 15,264 count= 2 avg_used= 7,632 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoNode [MEMORY] hold= 11,984 used= 11,592 count= 2 avg_used= 5,796 block_cnt= 2 chunk_cnt= 1 mod=StopWordSet [MEMORY] hold= 10,496 used= 10,112 count= 2 avg_used= 5,056 block_cnt= 1 chunk_cnt= 1 mod=Scheduler [MEMORY] hold= 10,000 used= 7,244 count= 14 avg_used= 517 block_cnt= 13 chunk_cnt= 12 mod=SqlSessiQuerSql [MEMORY] hold= 9,376 used= 8,984 count= 2 avg_used= 4,492 block_cnt= 2 chunk_cnt= 1 mod=FTPluginMap [MEMORY] hold= 8,784 used= 5,936 count= 14 avg_used= 424 block_cnt= 14 chunk_cnt= 9 mod=SequenceIdMap [MEMORY] hold= 8,784 used= 5,936 count= 14 avg_used= 424 block_cnt= 12 chunk_cnt= 7 mod=PackStateMap [MEMORY] hold= 8,752 used= 5,936 count= 14 avg_used= 424 block_cnt= 13 chunk_cnt= 8 mod=ContextsMap [MEMORY] hold= 8,752 used= 5,936 count= 14 avg_used= 424 block_cnt= 13 chunk_cnt= 8 mod=SequenceMap [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapDi [MEMORY] hold= 8,576 used= 8,192 count= 2 avg_used= 4,096 block_cnt= 2 chunk_cnt= 2 mod=LinearHashMapCn [MEMORY] hold= 8,192 used= 7,760 count= 2 avg_used= 3,880 block_cnt= 1 chunk_cnt= 1 mod=HashBucPsSessIn [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FetchLogTask [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=PurgeThrottle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=FTPluginHandle [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LockWaitMgr [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=APPLY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LCLSender [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=REPLAY_STATUS [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MatchLsnMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=MdsDebugMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=DASIDCache [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ShareBlksMap [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompPairs [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=HandleSubmit [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LsElectRefInfo [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LSIter [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=IORunners [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=RSCompCkmItems [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ls_id [MEMORY] hold= 8,096 used= 7,904 count= 1 avg_used= 7,904 block_cnt= 1 chunk_cnt= 1 mod=SqlSessionSbloc [MEMORY] hold= 8,096 used= 7,904 count= 1 avg_used= 7,904 block_cnt= 1 chunk_cnt= 1 mod=PsSessionInfo [MEMORY] hold= 8,000 used= 7,808 count= 1 avg_used= 7,808 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsInfo [MEMORY] hold= 7,984 used= 7,792 count= 1 avg_used= 7,792 block_cnt= 1 chunk_cnt= 1 mod=ColUsagNode [MEMORY] hold= 7,920 used= 7,728 count= 1 avg_used= 7,728 block_cnt= 1 chunk_cnt= 1 mod=HashNodePsCache [MEMORY] hold= 7,904 used= 7,712 count= 1 avg_used= 7,712 block_cnt= 1 chunk_cnt= 1 mod=DmlStatNode [MEMORY] hold= 7,824 used= 7,632 count= 1 avg_used= 7,632 block_cnt= 1 chunk_cnt= 1 mod=CheckMapNode [MEMORY] hold= 7,136 used= 66 count= 33 avg_used= 2 block_cnt= 31 chunk_cnt= 11 mod=[T]TSIUseWeak [MEMORY] hold= 6,800 used= 6,216 count= 3 avg_used= 2,072 block_cnt= 2 chunk_cnt= 2 mod=DetectManager [MEMORY] hold= 5,056 used= 4,656 count= 2 avg_used= 2,328 block_cnt= 2 chunk_cnt= 2 mod=CaLsInfoMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=PxPoolBkt [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResRuleIdMap [MEMORY] hold= 4,832 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ResGrpIdMap [MEMORY] hold= 4,288 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 4,160 used= 3,968 count= 1 avg_used= 3,968 block_cnt= 1 chunk_cnt= 1 mod=TinyAllocator [MEMORY] hold= 3,328 used= 1,792 count= 8 avg_used= 224 block_cnt= 5 chunk_cnt= 2 mod=LogIOTask [MEMORY] hold= 3,328 used= 256 count= 16 avg_used= 16 block_cnt= 16 chunk_cnt= 14 mod=[T]TSILastOper [MEMORY] hold= 3,200 used= 3,008 count= 1 avg_used= 3,008 block_cnt= 1 chunk_cnt= 1 mod=SqlDtlDfc [MEMORY] hold= 2,944 used= 1,920 count= 5 avg_used= 384 block_cnt= 5 chunk_cnt= 3 mod=TableLockNode [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=IRMMemHashBuck [MEMORY] hold= 2,528 used= 2,328 count= 1 avg_used= 2,328 block_cnt= 1 chunk_cnt= 1 mod=MedCheckMap [MEMORY] hold= 2,048 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=LogIOWS [MEMORY] hold= 2,000 used= 1,800 count= 1 avg_used= 1,800 block_cnt= 1 chunk_cnt= 1 mod=PxResMgr [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=MSTXCTX [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TXBFRetryId [MEMORY] hold= 1,744 used= 1,544 count= 1 avg_used= 1,544 block_cnt= 1 chunk_cnt= 1 mod=TabStatMgrLock [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=DUP_LS_SET [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GROUP_INDEX_MAP [MEMORY] hold= 1,472 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=GCMemtableMap [MEMORY] hold= 1,344 used= 960 count= 2 avg_used= 480 block_cnt= 1 chunk_cnt= 1 mod=PoolArenaArray [MEMORY] hold= 1,280 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=ModuleInitCtx [MEMORY] hold= 1,248 used= 1,056 count= 1 avg_used= 1,056 block_cnt= 1 chunk_cnt= 1 mod=LOG_HASH_MAP [MEMORY] hold= 1,216 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=LockMemtableMgr [MEMORY] hold= 1,152 used= 960 count= 1 avg_used= 960 block_cnt= 1 chunk_cnt= 1 mod=FTParserMgr [MEMORY] hold= 1,120 used= 120 count= 5 avg_used= 24 block_cnt= 2 chunk_cnt= 1 mod=FreezeTask [MEMORY] hold= 976 used= 776 count= 1 avg_used= 776 block_cnt= 1 chunk_cnt= 1 mod=DepTabletSet [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=CheckPointTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TabShellTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=DiskUsageTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=tenant_rm_mgr [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=FlushTimer [MEMORY] hold= 960 used= 768 count= 1 avg_used= 768 block_cnt= 1 chunk_cnt= 1 mod=TLD_TIMER [MEMORY] hold= 848 used= 648 count= 1 avg_used= 648 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfo [MEMORY] hold= 752 used= 552 count= 1 avg_used= 552 block_cnt= 1 chunk_cnt= 1 mod=LSFreeze [MEMORY] hold= 672 used= 288 count= 2 avg_used= 144 block_cnt= 2 chunk_cnt= 2 mod=ObTxLSLogCb [MEMORY] hold= 672 used= 472 count= 1 avg_used= 472 block_cnt= 1 chunk_cnt= 1 mod=Election [MEMORY] hold= 576 used= 384 count= 1 avg_used= 384 block_cnt= 1 chunk_cnt= 1 mod=HAScheduler [MEMORY] hold= 544 used= 144 count= 2 avg_used= 72 block_cnt= 1 chunk_cnt= 1 mod=TntSrvObjPool [MEMORY] hold= 512 used= 120 count= 2 avg_used= 60 block_cnt= 2 chunk_cnt= 2 mod=UserResourceMgr [MEMORY] hold= 448 used= 64 count= 2 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfFSCbNode [MEMORY] hold= 416 used= 16 count= 2 avg_used= 8 block_cnt= 2 chunk_cnt= 1 mod=ObLogEXTHandler [MEMORY] hold= 400 used= 208 count= 1 avg_used= 208 block_cnt= 1 chunk_cnt= 1 mod=[T]ObMacroBlock [MEMORY] hold= 320 used= 102 count= 1 avg_used= 102 block_cnt= 1 chunk_cnt= 1 mod=TxLSLogBuf [MEMORY] hold= 304 used= 104 count= 1 avg_used= 104 block_cnt= 1 chunk_cnt= 1 mod=RedoLogBuckLock [MEMORY] hold= 304 used= 112 count= 1 avg_used= 112 block_cnt= 1 chunk_cnt= 1 mod=Coordinator [MEMORY] hold= 256 used= 64 count= 1 avg_used= 64 block_cnt= 1 chunk_cnt= 1 mod=NegParamIdex [MEMORY] hold= 256 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=ResRuleInfoMap [MEMORY] hold= 256 used= 48 count= 1 avg_used= 48 block_cnt= 1 chunk_cnt= 1 mod=[T]TSITabletTab [MEMORY] hold= 240 used= 41 count= 1 avg_used= 41 block_cnt= 1 chunk_cnt= 1 mod=KeepAliveBuf [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PalfRCCbNode [MEMORY] hold= 224 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=RebuildCbNode [MEMORY] hold= 208 used= 16 count= 1 avg_used= 16 block_cnt= 1 chunk_cnt= 1 mod=logservice [MEMORY] hold= 431,011,728 used= 413,814,060 count= 33,092 avg_used= 12,504 mod=SUMMARY [2024-10-16 14:58:16.758739] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=57] [MEMORY] tenant_id= 1 ctx_id= MEMSTORE_CTX_ID hold= 245,366,784 used= 243,449,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 block_cnt= 117 chunk_cnt= 117 mod=Memstore [MEMORY] hold= 243,449,856 used= 243,338,472 count= 117 avg_used= 2,079,816 mod=SUMMARY [2024-10-16 14:58:16.758749] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TRANS_CTX_MGR_ID hold= 2,097,152 used= 729,088 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 block_cnt= 1 chunk_cnt= 1 mod=PartTranCtxMgr [MEMORY] hold= 729,088 used= 722,560 count= 1 avg_used= 722,560 mod=SUMMARY [2024-10-16 14:58:16.758759] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= PLAN_CACHE_CTX_ID hold= 79,691,776 used= 73,331,136 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 68,663,616 used= 62,143,518 count= 2,832 avg_used= 21,943 block_cnt= 2,832 chunk_cnt= 37 mod=SqlPhyPlan [MEMORY] hold= 3,414,336 used= 3,146,720 count= 326 avg_used= 9,652 block_cnt= 326 chunk_cnt= 37 mod=SqlPlanCache [MEMORY] hold= 1,251,200 used= 1,140,800 count= 575 avg_used= 1,984 block_cnt= 192 chunk_cnt= 37 mod=CreateContext [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 73,331,136 used= 66,432,638 count= 3,735 avg_used= 17,786 mod=SUMMARY [2024-10-16 14:58:16.758771] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 1 ctx_id= GLIBC hold= 2,097,152 used= 81,040 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 80,944 used= 59,018 count= 263 avg_used= 224 block_cnt= 20 chunk_cnt= 1 mod=PlJit [MEMORY] hold= 96 used= 32 count= 1 avg_used= 32 block_cnt= 1 chunk_cnt= 1 mod=PlCodeGen [MEMORY] hold= 81,040 used= 59,050 count= 264 avg_used= 223 mod=SUMMARY [2024-10-16 14:58:16.758778] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= CO_STACK hold= 136,314,880 used= 133,668,864 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 block_cnt= 259 chunk_cnt= 65 mod=CoStack [MEMORY] hold= 133,668,864 used= 133,420,224 count= 259 avg_used= 515,136 mod=SUMMARY [2024-10-16 14:58:16.758791] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= KVSTORE_CACHE_ID hold= 861,929,472 used= 861,929,472 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 861,929,472 used= 0 count= 411 avg_used= 0 mod=SUMMARY [2024-10-16 14:58:16.758800] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= META_OBJ_CTX_ID hold= 37,748,736 used= 36,290,496 limit= 429,496,720 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 29,491,200 used= 26,236,400 count= 400 avg_used= 65,591 block_cnt= 400 chunk_cnt= 18 mod=L_TabletPool [MEMORY] hold= 6,709,248 used= 5,952,128 count= 91 avg_used= 65,408 block_cnt= 91 chunk_cnt= 15 mod=N_TabletPool [MEMORY] hold= 90,048 used= 84,768 count= 2 avg_used= 42,384 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 36,290,496 used= 32,273,296 count= 493 avg_used= 65,463 mod=SUMMARY [2024-10-16 14:58:16.758808] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= TX_CALLBACK_CTX_ID hold= 12,582,912 used= 6,234,176 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 block_cnt= 767 chunk_cnt= 6 mod=MemtableCallbac [MEMORY] hold= 6,234,176 used= 6,086,912 count= 767 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:58:16.758816] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= LOB_CTX_ID hold= 2,097,152 used= 81,856 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 73,728 used= 65,408 count= 1 avg_used= 65,408 block_cnt= 1 chunk_cnt= 1 mod=LobAllocator [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 81,856 used= 73,344 count= 2 avg_used= 36,672 mod=SUMMARY [2024-10-16 14:58:16.758823] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= PS_CACHE_CTX_ID hold= 2,097,152 used= 16,288 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 block_cnt= 4 chunk_cnt= 1 mod=SqlPsCache [MEMORY] hold= 16,288 used= 14,294 count= 10 avg_used= 1,429 mod=SUMMARY [2024-10-16 14:58:16.758829] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= RPC_CTX_ID hold= 2,097,152 used= 0 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [2024-10-16 14:58:16.758840] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= TX_DATA_TABLE hold= 54,546,432 used= 52,803,968 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 49,613,312 used= 48,441,344 count= 6,104 avg_used= 7,936 block_cnt= 6,104 chunk_cnt= 25 mod=TX_DATA_SLICE [MEMORY] hold= 3,182,528 used= 3,153,728 count= 3 avg_used= 1,051,242 block_cnt= 3 chunk_cnt= 3 mod=MEMTABLE_ARENA [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 52,803,968 used= 51,603,008 count= 6,108 avg_used= 8,448 mod=SUMMARY [2024-10-16 14:58:16.758851] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1 ctx_id= MDS_DATA_ID hold= 4,194,304 used= 398,272 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 260,096 used= 253,952 count= 32 avg_used= 7,936 block_cnt= 32 chunk_cnt= 2 mod=MdsTable [MEMORY] hold= 138,176 used= 134,912 count= 17 avg_used= 7,936 block_cnt= 17 chunk_cnt= 2 mod=TX_OP [MEMORY] hold= 398,272 used= 388,864 count= 49 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:58:16.758860] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant_id= 1 ctx_id= SCHEMA_SERVICE hold= 2,097,152 used= 8,128 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=NonPartTenCac [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 mod=SUMMARY [2024-10-16 14:58:16.758877] INFO [LIB] operator() (ob_malloc_allocator.cpp:524) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=3] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,082,318,848 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 371,380,224 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 14,680,064 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 104,857,600 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 20,971,520 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= RPC_CTX_ID hold_bytes= 6,291,456 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 26,357,760 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= SCHEMA_SERVICE hold_bytes= 237,785,088 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= UNEXPECTED_IN_500 hold_bytes= 295,800,832 limit= 9,223,372,036,854,775,807 [2024-10-16 14:58:16.759075] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:180) [21170][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 371,380,224 used= 311,619,136 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 66,584,784 used= 66,551,824 count= 33 avg_used= 2,016,721 block_cnt= 33 chunk_cnt= 33 mod=Compressor [MEMORY] hold= 31,477,760 used= 31,458,304 count= 1 avg_used= 31,458,304 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 23,584,768 used= 23,499,200 count= 9 avg_used= 2,611,022 block_cnt= 9 chunk_cnt= 8 mod=IoControl [MEMORY] hold= 15,925,248 used= 15,247,872 count= 216 avg_used= 70,592 block_cnt= 216 chunk_cnt= 42 mod=[T]ObSessionDIB [MEMORY] hold= 12,873,728 used= 12,856,352 count= 1 avg_used= 12,856,352 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 9,457,664 used= 9,437,784 count= 1 avg_used= 9,437,784 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 5,505,024 used= 5,238,784 count= 32 avg_used= 163,712 block_cnt= 32 chunk_cnt= 22 mod=Iteratorget_logical_del_time()=9510364671165, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894557, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.804247] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1757-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364672642, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894558, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.806688] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5532}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286110} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.814504] INFO [PALF] read (log_cache.cpp:103) [22264][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=17] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=84562, hit_cnt=53, read_cnt=53, hit rate=1.000000000000000000e+00) [2024-10-16 14:58:16.818044] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.818843] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1759-0-0] [lt=20][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364687224, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894559, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.822359] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654967884924702720},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286114} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.822606] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2119}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286114} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.825562] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] get number(ret=0, service_type_=0, range=1, base_id=1729061896824633000, start_id=1729061896824633000, end_id=1729061896824633001) [2024-10-16 14:58:16.826689] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C175B-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364695082, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894560, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.828139] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C175D-0-0] [lt=11][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364696535, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894561, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.829280] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=50, replace_time=0, replace_start_pos=346027, replace_num=31457) [2024-10-16 14:58:16.829293] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=50, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:16.829713] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684547669581484032},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286115} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.829956] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2521}, to_seq_no={branch:0, seq:1454}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286115} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.831685] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2307-0-0] [lt=12][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364700079, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894562, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.833092] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC589-0-0] [lt=22] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2024-10-16 14:58:16.833848] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC589-0-0] [lt=11] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1002) [2024-10-16 14:58:16.833857] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1001) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC589-0-0] [lt=7] no transfer partition job [2024-10-16 14:58:16.833864] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1031) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC589-0-0] [lt=4] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer={value_str:"True"}) [2024-10-16 14:58:16.835205] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC589-0-0] [lt=6] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1002, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2024-10-16 14:58:16.835224] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:482) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC589-0-0] [lt=14] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2024-10-16 14:58:16.835685] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [22812][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=20] start do ls ha handler(ls_id_array_=[{id:1003}, {id:1}, {id:1001}, {id:1002}]) [2024-10-16 14:58:16.836379] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:500) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC589-0-0] [lt=12][errcode=-4152] get obj error(ret=-4152) [2024-10-16 14:58:16.836412] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [22074][T1002_TBalance][T1002][YB4A0A000407-00061E9AB38CC589-0-0] [lt=9] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, job_cnt=0, primary_zone_num_=3, unit_group_array_=[{unit_group_id:1001, status:0}], ls_array_=[{tenant_id:1002, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1002}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase2", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, {tenant_id:1002, ls_id:{id:1003}, ls_group_id:1001, status:"NORMAL", unit_group_id:1001, primary_zone:"myoceanbase3", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1729057174083520, last_statistic_bg_stat_time=1729008957751717, last_statistic_schema_version=1729008953019688, last_statistic_max_transfer_task_id={id:-1}, enable_rebalance=true, enable_transfer=true) [2024-10-16 14:58:16.837271] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1760-0-0] [lt=34][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364705667, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894563, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.837559] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684620839126683648},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286116} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.838842] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1761-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364707237, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894564, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.839139] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7338}, to_seq_no={branch:0, seq:5217}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286116} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.846759] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1763-0-0] [lt=22][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364715153, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894565, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.848572] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061896848307000, start_id=1729061896848307000, end_id=1729061896848307001) [2024-10-16 14:58:16.849203] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E6-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}) [2024-10-16 14:58:16.849225] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [22836][T1004_L0_G0][T1004][YB4A0A000409-00061E9A279FA9E6-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061896180210004, v:0}, arg={tenant_id:1004, ls_id:{id:1003}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1003}, cur_readable_scn:{val:1729061896180210004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:16.850463] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000409-00061E9A279FA9E6-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061896849754000, start_id=1729061896849754000, end_id=1729061896849754001) [2024-10-16 14:58:16.852916] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E6-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061896852560000, start_id=1729061896852560000, end_id=1729061896852560001) [2024-10-16 14:58:16.852975] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E6-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=17, avg_log_batch_cnt=1, total_group_log_size=10208, avg_group_log_size=600) [2024-10-16 14:58:16.852984] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22492][T1003_L0_G0][T1003][YB4A0A000409-00061E9A279FA9E6-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=10208) [2024-10-16 14:58:16.853436] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107619][T1003_L0_G1][T1003][YB4A0A000408-00061E9B803CD388-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="10.0.4.7:2890", committed size=10208) [2024-10-16 14:58:16.853917] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C1765-0-0] [lt=17][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364722311, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894566, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.854269] INFO [LIB] stat (utility.h:1165) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT WRITE LOG COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=314, this=0x2b0e2efeda70, extra_info="") [2024-10-16 14:58:16.854286] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=10208) [2024-10-16 14:58:16.854300] INFO [LIB] stat (utility.h:1182) [22313][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=156, this=0x2b0e2ebba530, extra_info="") [2024-10-16 14:58:16.854330] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e2efe36a0, extra_info="palf_id:1") [2024-10-16 14:58:16.854341] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1249, this=0x2b0e2efe38d0, extra_info="palf_id:1") [2024-10-16 14:58:16.854345] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=607, avg_log_submit_to_slide_time=1219) [2024-10-16 14:58:16.854352] INFO [LIB] stat (utility.h:1165) [22312][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e2efedca0, extra_info="") [2024-10-16 14:58:16.854375] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] rpc statistics(total_trans_msg_count=2, total_batch_msg_count=0) [2024-10-16 14:58:16.859312] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830041564827648},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286120} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.860197] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2427}, to_seq_no={branch:0, seq:1344}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286120} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.863798] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21894][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=7101, mem_used=14558208) [2024-10-16 14:58:16.865124] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:644) [22703][T1004_TntShared][T1004][Y0-0000000000000000-0-0] [lt=14] clear dtl interm result cost(us)(clear_cost=935, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2024-10-16 14:58:16.870323] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684547670034468864},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286122} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.870623] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3224}, to_seq_no={branch:0, seq:2161}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286122} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.872101] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061896871590000, start_id=1729061896871590000, end_id=1729061896871590001) [2024-10-16 14:58:16.873239] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684620839202181120},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286123} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.873501] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1108}, to_seq_no={branch:0, seq:1103}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286123} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.875372] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [123287][T1002_L0_G0][T1002][YB4A0A000407-00061E9B33BF2308-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364743766, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894567, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.884097] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [12153][T1002_L0_G0][T1002][YB4A0A000407-00061E9B3B6C176B-0-0] [lt=18][errcode=0] set logical del time(cache_obj->get_logical_del_time()=9510364752490, cache_obj->added_lc()=false, cache_obj->get_object_id()=78894568, cache_obj->get_tenant_id()=1002, lbt()="0x1a5796b1 0xd233e9e 0x659dcbb 0x6590bfa 0x6589157 0x657ea10 0x657bded 0x656ae28 0xc6a3014 0x1a6176ae 0x2b0dadc8cea5 0x2b0dadf9ef3d") [2024-10-16 14:58:16.892628] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.894830] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":655028974719193089},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286130} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.895120] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C73-0-0] [lt=0] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:16.895142] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B733E8C73-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061896884962001, v:0}, arg={tenant_id:1002, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1002, ls_id:{id:1002}, cur_readable_scn:{val:1729061896884962001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:16.895228] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286130} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.896107] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:688) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] no need to update min/max version(new_version={val:1729061896449420000, v:0}, min_version={val:1729061896449420000, v:0}, max_version={val:1729061896449420000, v:0}) [2024-10-16 14:58:16.896284] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B733E8C73-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061896895604000, start_id=1729061896895604000, end_id=1729061896895604001) [2024-10-16 14:58:16.900226] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21932][T1001_L0_G0][T1001][YB4A0A000408-00061E9B733E8C73-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061896899391000, start_id=1729061896899391000, end_id=1729061896899391001) [2024-10-16 14:58:16.901420] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22052][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=105781) [2024-10-16 14:58:16.904303] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061896903892000, start_id=1729061896903892000, end_id=1729061896903892001) [2024-10-16 14:58:16.904875] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":655028911833993216},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286132} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.905906] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2120}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286132} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.918107] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.921531] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2024-10-16 14:58:16.921549] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [22460][T1003_TabletShe][T1003][Y0-0000000000000000-0-0] [lt=11] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=220) [2024-10-16 14:58:16.923368] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:398) [21077][ServerTracerTim][T0][YB4A0A000407-00061E9B36F98DE0-0-0] [lt=14] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2024-10-16 14:58:16.925798] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":684618825860763648},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286134} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.927835] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1655) [21257][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=12] skip inspect bad block(last_check_time=1728893950088695, last_macro_idx=-1) [2024-10-16 14:58:16.927886] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5235}, to_seq_no={branch:0, seq:3163}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286134} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.929560] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:195) [21911][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}]) [2024-10-16 14:58:16.931452] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=4] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:16.932047] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=19] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:16.932096] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=13] inner load succ(latest_freeze_info={frozen_version:{val:1729015201670298000, v:0}, schema_version:1722495466719360, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061887261758000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}) [2024-10-16 14:58:16.932139] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:651) [22392][T1003_FreInfoRe][T1003][Y0-0000000000000000-0-0] [lt=42] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1729061887261758000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1003, freeze_info:{latest_snapshot_gc_scn:{val:1729061887261758000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336801688164000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726423200685081000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726509604454498000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726596003373674000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726682402478626000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726768801666423000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726855200744303000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1726941604611663000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727028003583922000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727114402519118000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727200801382296000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727287200296135000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727373604335787000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727460003461607000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727546402419927000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727632801384381000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727719200286728000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727805604264622000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727892003305470000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1727978402400856000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728064801285512000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728151200151248000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728237604341951000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728324003264713000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728410402425424000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728496801649060000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728583200612472000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728669604998985000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728756004282144000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728842403471188000, v:0}, schema_version:1722495466719360, data_version:17180066304}, {frozen_version:{val:1728928802456398000, v:0}, schema_version:1722495466719360, data_version:17180066304}]}}) [2024-10-16 14:58:16.932720] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1188003840, global_cache_size=1575141376, tenant_max_wash_size=1234534, tenant_min_wash_size=1234534, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:16.934424] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:16.934437] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=9] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=6291456, ctx.total_memstore_used_=39845888, ctx.total_memstore_hold_=39845888, memstore_freeze_trigger_limit_=27208900, tenant_id=1004) [2024-10-16 14:58:16.934446] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [22642][T1004_Occam][T1004][YB4A0A000407-00061E99753C0BE3-0-0] [lt=7] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:16.935964] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":673087342012649472},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286138} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.936184] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830051354333184},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286139} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.936268] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1181}, to_seq_no={branch:0, seq:8}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286138} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.936349] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1180}, to_seq_no={branch:0, seq:9}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286139} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.940090] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":673087342037815296},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286142} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.940230] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830051392081920},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286143} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.940297] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:10}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286142} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.940532] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=14] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1057}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286143} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.943986] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830051417247744},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286145} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.944266] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1026}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286145} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.946801] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654942781902016512},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286146} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.946985] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1068}, to_seq_no={branch:0, seq:1063}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286146} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.950052] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830051631157248},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286149} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.950457] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2105}, to_seq_no={branch:0, seq:1050}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286149} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.950510] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [22234][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5865, mem_used=12478464) [2024-10-16 14:58:16.952287] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830041879400448},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286151} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.952722] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286151} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.953440] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1498) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF8-0-0] [lt=4] receive register_req from acceptor, ignore(self="10.0.4.7:2890", palf_id=1, child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061896921102, keepalive_ts:-1}) [2024-10-16 14:58:16.953456] INFO [PALF] handle_register_parent_req (palf_handle_impl.cpp:1517) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3544DBF8-0-0] [lt=15] handle_register_parent_req finished(ret=0, ret="OB_SUCCESS", palf_id=1, self="10.0.4.7:2890", child={server:"10.0.4.9:2890", region:"sys_region", register_time_us:1729061896921102, keepalive_ts:-1}, is_to_leader=true) [2024-10-16 14:58:16.956873] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061896956716000, start_id=1729061896956716000, end_id=1729061896956716001) [2024-10-16 14:58:16.957172] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x2b0e431ed470, extra_info="") [2024-10-16 14:58:16.957186] INFO [LIB] stat (utility.h:1165) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e5af19210, extra_info="id:1001") [2024-10-16 14:58:16.959749] INFO [STORAGE] exec_wait (ob_tmp_file_cache.cpp:1593) [108403][T1003_ComTimerT][T1003][Y0-0000000000000000-0-0] [lt=23] succeed to do one round of tmp block io(ret=0, loop_nums=0, wait_io_cnt=0, washing_count=0, block_cache_num=0, page_cache_num=0) [2024-10-16 14:58:16.960751] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":684618575372734464},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286157} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.960875] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:581) [22704][T1004_TenantMet][T1004][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:1344, free_obj_hold_cnt:368, allocator used:6587776, allocator total:6998656}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:381, free_obj_hold_cnt:19, allocator used:26236400, allocator total:26236400}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():3968, full allocator used:1600, full allocator total:69504}, tablets_mem=33304560, tablets_mem_limit=214748360, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:6, free_obj_hold_cnt:55, allocator used:124928, allocator total:130816}, pending_cnt=0, wait_gc_count=0, tablet count=1725) [2024-10-16 14:58:16.960945] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2529) [107610][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBF-0-0] [lt=4] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:"10.0.4.8:2890", ls_array_:[{id:1}]}) [2024-10-16 14:58:16.960966] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1479) [107610][T1_L0_G19][T1][YB4A0A000408-00061E9B43BCDFBF-0-0] [lt=17] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:"10.0.4.7:2890", timestamp:1, flag:0}{server:"10.0.4.8:2890", timestamp:1, flag:0}{server:"10.0.4.9:2890", timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr="10.0.4.8:2890", this={role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}) [2024-10-16 14:58:16.961061] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2170}, to_seq_no={branch:0, seq:1047}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286157} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.962724] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22267][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061896962079000, start_id=1729061896962079000, end_id=1729061896962079001) [2024-10-16 14:58:16.967951] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] [COMPACTION DAG STATUS] (start_time=1729061886966588, end_time=1729061896967950, dag_status={}) [2024-10-16 14:58:16.967968] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=15] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:16.967976] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:16.967980] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:16.967983] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:16.967986] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:16.967990] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:16.967997] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:16.968000] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:16.968006] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:16.968009] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968018] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968022] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968029] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968033] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968040] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968044] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968051] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968056] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968062] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968066] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968073] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968078] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968082] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968085] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968089] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968093] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968096] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968102] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968106] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968113] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968116] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968124] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968128] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968138] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968142] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968146] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968149] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968153] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968157] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968161] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968165] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968168] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968172] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968176] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968180] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968183] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968187] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968191] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968197] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968201] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968204] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968208] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968215] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968219] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968226] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968229] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968233] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968240] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968243] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968250] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968254] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968260] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968264] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968271] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968275] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968279] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968282] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:16.968286] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:58:16.968290] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:58:16.968296] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:58:16.968298] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:58:16.968304] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:58:16.968307] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:58:16.968310] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:58:16.968315] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:58:16.968318] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:58:16.968320] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:58:16.968327] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22695][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:58:16.968739] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830051958312960},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286163} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.969070] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2110}, to_seq_no={branch:0, seq:1047}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286163} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.969473] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":675512509582135296},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286164} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.969878] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1174}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286164} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.972289] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830041929732096},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286166} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.972623] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:1057}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286166} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.973106] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654830052021227520},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286167} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.973331] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1044}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286167} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.974169] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] get number(ret=0, service_type_=0, range=1, base_id=1729061896973871000, start_id=1729061896973871000, end_id=1729061896973871001) [2024-10-16 14:58:16.974170] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061896973871000, start_id=1729061896973871000, end_id=1729061896973871001) [2024-10-16 14:58:16.978436] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830052058976256},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286170} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.978819] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1073}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286170} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.981875] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830042093309952},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286173} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.982356] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286173} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.984154] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830052121890816},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286175} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.984426] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1703}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286175} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.987796] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910255238074368},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286176} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.988467] INFO [LIB] stat (utility.h:1165) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=307, this=0x2b0e0f1dfa70, extra_info="") [2024-10-16 14:58:16.988483] INFO [LIB] stat (utility.h:1182) [22051][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=252, this=0x2b0e0d9ba530, extra_info="") [2024-10-16 14:58:16.988501] INFO [LIB] stat (utility.h:1165) [22050][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=8, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e0f1dfca0, extra_info="") [2024-10-16 14:58:16.989111] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4213}, to_seq_no={branch:0, seq:1053}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286176} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.992179] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1729061896991446000, start_id=1729061896991446000, end_id=1729061896991446001) [2024-10-16 14:58:16.992721] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:16.992800] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=48, this=0x2b0e3af18280, extra_info="") [2024-10-16 14:58:16.992810] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=1114, this=0x2b0e3af184b0, extra_info="") [2024-10-16 14:58:16.992814] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=0, this=0x2b0e3af186e0, extra_info="") [2024-10-16 14:58:16.992817] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=-623, this=0x2b0e3af18910, extra_info="") [2024-10-16 14:58:16.992820] INFO [LIB] stat (utility.h:1165) [22406][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=34, stat_interval=5000000, avg (count/cost)=539, this=0x2b0e3af18b40, extra_info="") [2024-10-16 14:58:16.993583] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:671) [22149][T1002_ApplySrv0][T1002][YB4A0A000407-00061E9B3B6C11C4-0-0] [lt=2] rpc statistics(total_trans_msg_count=167, total_batch_msg_count=0) [2024-10-16 14:58:16.997063] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] get number(ret=0, service_type_=0, range=1, base_id=1729061896996730000, start_id=1729061896996730000, end_id=1729061896996730001) [2024-10-16 14:58:16.997282] INFO [LIB] stat (utility.h:1165) [107618][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2F0D-0-0] [lt=12] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43cc16a0, extra_info="palf_id:1002") [2024-10-16 14:58:16.997292] INFO [LIB] stat (utility.h:1165) [107618][T1004_L0_G1][T1004][YB4A0A000408-00061E9B7FBF2F0D-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=9, stat_interval=1000000, avg (count/cost)=33805, this=0x2b0e43cc18d0, extra_info="palf_id:1002") [2024-10-16 14:58:16.997537] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21749][T1001_Occam][T1001][YB4A0A000407-00061E9983E960C2-0-0] [lt=13] ====== tenant freeze timer task ====== [2024-10-16 14:58:16.997649] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=25] [REPLAY STAT REPLAY TASK IN QUEUE TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=94, this=0x2b0e400b08f8, extra_info="") [2024-10-16 14:58:16.997656] INFO [LIB] stat (utility.h:1165) [22824][T1004_ReplaySrv][T1004][Y0-0000000000000000-0-0] [lt=7] [REPLAY STAT REPLAY TASK EXECUTE COST TIME](cur_stat_count=20, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e400b0b28, extra_info="") [2024-10-16 14:58:16.998874] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830052172222464},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286181} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:16.999126] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286181} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:16.999369] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [21154][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=13][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-5024, dropped:316, tid:12153}]) [2024-10-16 14:58:17.002573] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f48e938 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286184} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1057}, prev=this=0x2b0f3f4515d0 trans_version={val:1729061839083031000, v:0} scn={val:1729061839083031000, v:0} tx_id={txid:392285694} prev=0x2b0f3f451490 next=(nil) modify_count=3 acc_checksum=3389920567 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2121}, snapshot_version={val:1729061897002056000, v:0}, *this={this=0x2b0e5abd9290 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0f3f4515d0 latest_compact_node=(nil) max_trans_version={val:1729061839083031000, v:0} max_trans_id=392285694 max_elr_trans_version={val:1729061839083031000, v:0} max_elr_trans_id=392285694 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839083031000, v:0} min_modify_scn={val:1729061778412320001, v:0}}) [2024-10-16 14:58:17.002612] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f48e938 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286184} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1057}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851221, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851221, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897002056000, v:0}, tx_id:{txid:392286184}, scn:{branch:0, seq:1054}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286184}, tx_desc:{this:0x2b0ec37cd910, tx_id:{txid:392286184}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897002056, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897001003, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1057}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.002667] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286184} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286184}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:1057}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.002686] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":654830042219139072},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286184} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.002698] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.002703] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042219139072},{"BIGINT":1729058400},{"VARCHAR":"A¯Q¢¦", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.002752] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.003056] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:1055}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286184} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.005868] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061897005223000, start_id=1729061897005223000, end_id=1729061897005223001) [2024-10-16 14:58:17.015170] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=7] Garbage Collector is running(seq_=656702, gc_interval=10000000) [2024-10-16 14:58:17.015205] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=15] self is leader, skip it(ls->get_ls_id()={id:1}) [2024-10-16 14:58:17.015220] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=9] gc_check_member_list_ cost time(ret=0, time_us=34) [2024-10-16 14:58:17.015229] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] execute_gc cost time(ret=0, time_us=1) [2024-10-16 14:58:17.016018] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=6] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2024-10-16 14:58:17.016040] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=18] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2024-10-16 14:58:17.016048] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=9) [2024-10-16 14:58:17.016059] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [21825][T1001_GCCollect][T1001][Y0-0000000000000000-0-0] [lt=9] ObSafeDestroyHandler start process [2024-10-16 14:58:17.017199] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f5757e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286196} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1026}, prev=this=0x2b0f7f511ff0 trans_version={val:1729061839098849000, v:0} scn={val:1729061839098849000, v:0} tx_id={txid:392285701} prev=0x2b0f7f511eb0 next=(nil) modify_count=3 acc_checksum=2715185522 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2129}, snapshot_version={val:1729061897016756000, v:0}, *this={this=0x2b0f7f48d288 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0f7f511ff0 latest_compact_node=(nil) max_trans_version={val:1729061839098849000, v:0} max_trans_id=392285701 max_elr_trans_version={val:1729061839098849000, v:0} max_elr_trans_id=392285701 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839098849000, v:0} min_modify_scn={val:1729061778489617000, v:0}}) [2024-10-16 14:58:17.017231] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f5757e0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286196} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1026}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851208, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851208, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897016756000, v:0}, tx_id:{txid:392286196}, scn:{branch:0, seq:1023}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286196}, tx_desc:{this:0x2b0ec37b1e50, tx_id:{txid:392286196}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897016756, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897015734, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1026}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.017297] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=65][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286196} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286196}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1026}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.017314] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":684545341818261504},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286196} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.017329] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.017334] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684545341818261504},{"BIGINT":1729058400},{"VARCHAR":"AoY‹à", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.017390] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.017473] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=5][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb4f208 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286197} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1024}, prev=this=0x2b0fadaf4a28 trans_version={val:1729061837287789000, v:0} scn={val:1729061837287789000, v:0} tx_id={txid:392285282} prev=0x2b0fadaf48e8 next=(nil) modify_count=1 acc_checksum=1320620877 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3198}, snapshot_version={val:1729061897016756001, v:0}, *this={this=0x2b0fadaf45e0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaf4a28 latest_compact_node=(nil) max_trans_version={val:1729061837287789000, v:0} max_trans_id=392285282 max_elr_trans_version={val:1729061837287789000, v:0} max_elr_trans_id=392285282 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837287789000, v:0} min_modify_scn={val:1729061837287789000, v:0}}) [2024-10-16 14:58:17.017492] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=19][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb4f208 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286197} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1024}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810315, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810315, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897016756001, v:0}, tx_id:{txid:392286197}, scn:{branch:0, seq:1021}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286197}, tx_desc:{this:0x2b0ec37d37f0, tx_id:{txid:392286197}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897016756, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897015737, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1024}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.017527] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=35][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286197} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286197}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1024}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.017540] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654910192197685248},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286197} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.017550] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.017554] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192197685248},{"BIGINT":1729058400},{"VARCHAR":"BSkÏÕ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.017582] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=17][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.017688] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1029}, to_seq_no={branch:0, seq:1024}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286196} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.017711] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=13] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1027}, to_seq_no={branch:0, seq:1022}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286197} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.018239] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:17.019252] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061897018874000, start_id=1729061897018874000, end_id=1729061897018874001) [2024-10-16 14:58:17.020467] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f5b93dcd8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286198} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778513125 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, prev=this=0x2b0f5b901748 trans_version={val:1729061839100937002, v:0} scn={val:1729061839100937002, v:0} tx_id={txid:392285702} prev=0x2b0f5b901608 next=(nil) modify_count=3 acc_checksum=2385770028 version=1729061778513125 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2093}, snapshot_version={val:1729061897019937000, v:0}, *this={this=0x2b0f5b8a6158 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0f5b901748 latest_compact_node=(nil) max_trans_version={val:1729061839100937002, v:0} max_trans_id=392285702 max_elr_trans_version={val:1729061839100937002, v:0} max_elr_trans_id=392285702 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839100937002, v:0} min_modify_scn={val:1729061778525821001, v:0}}) [2024-10-16 14:58:17.020503] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=35][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f5b93dcd8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286198} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778513125 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851217, tablet_id:{id:209288}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851217, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897019937000, v:0}, tx_id:{txid:392286198}, scn:{branch:0, seq:1058}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286198}, tx_desc:{this:0x2b0ec37b7d30, tx_id:{txid:392286198}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897019937, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897018883, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1062}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.020553] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=49][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286198} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286198}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778513125, seq_no:{branch:0, seq:1062}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.020569] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":684618575846690817},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286198} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.020583] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.020588] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":684618575846690817},{"BIGINT":1729058400},{"VARCHAR":"AlºÃà", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.020634] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=28][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.020830] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [22268][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286198} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.022452] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=11][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22f01b68 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286200} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1055}, prev=this=0x2b0f22ebcc98 trans_version={val:1729061837292038000, v:0} scn={val:1729061837292038000, v:0} tx_id={txid:392285284} prev=0x2b0f22ebcb58 next=(nil) modify_count=1 acc_checksum=210815944 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3178}, snapshot_version={val:1729061897022038000, v:0}, *this={this=0x2b0f22ebc850 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ebcc98 latest_compact_node=(nil) max_trans_version={val:1729061837292038000, v:0} max_trans_id=392285284 max_elr_trans_version={val:1729061837292038000, v:0} max_elr_trans_id=392285284 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837292038000, v:0} min_modify_scn={val:1729061837292038000, v:0}}) [2024-10-16 14:58:17.022479] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=27][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22f01b68 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286200} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1055}, ctx={this:0x2b0e2b854080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810320, tablet_id:{id:209267}, table_iter:0x2b0e2b853210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810320, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897022038000, v:0}, tx_id:{txid:392286200}, scn:{branch:0, seq:1052}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286200}, tx_desc:{this:0x2b0e0fcba2d0, tx_id:{txid:392286200}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897022038, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897020988, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1055}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.022527] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=47][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286200} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286200}, data:0x2b0e2b850438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:1055}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.022543] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654910192248016896},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286200} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.022554] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.022559] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192248016896},{"BIGINT":1729058400},{"VARCHAR":"BWŒ%æç", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.022593] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=18][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.022807] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1058}, to_seq_no={branch:0, seq:1053}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286200} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.030767] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:693) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=6] Cache replace map node details(ret=0, replace_node_count=48, replace_time=0, replace_start_pos=377484, replace_num=31457) [2024-10-16 14:58:17.030785] INFO [COMMON] replace_map (ob_kv_storecache.cpp:766) [21172][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=16] replace map num details(ret=0, replace_node_count=48, map_once_replace_num_=31457, map_replace_skip_count_=0) [2024-10-16 14:58:17.034725] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb4f8e8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286205} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1053}, prev=this=0x2b0fadaf5128 trans_version={val:1729061837301501000, v:0} scn={val:1729061837301501000, v:0} tx_id={txid:392285287} prev=0x2b0fadaf4fe8 next=(nil) modify_count=1 acc_checksum=3987562311 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4100}, snapshot_version={val:1729061897033765000, v:0}, *this={this=0x2b0fadaf4ce0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaf5128 latest_compact_node=(nil) max_trans_version={val:1729061837301501000, v:0} max_trans_id=392285287 max_elr_trans_version={val:1729061837301501000, v:0} max_elr_trans_id=392285287 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837301501000, v:0} min_modify_scn={val:1729061837301501000, v:0}}) [2024-10-16 14:58:17.034763] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb4f8e8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286205} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1053}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810329, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810329, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897033765000, v:0}, tx_id:{txid:392286205}, scn:{branch:0, seq:1050}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286205}, tx_desc:{this:0x2b0ec37c7a30, tx_id:{txid:392286205}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897033765, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897032716, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1053}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.034838] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=74][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286205} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286205}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1053}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.034856] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654910192323514368},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286205} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.034872] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.034878] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192323514368},{"BIGINT":1729058400},{"VARCHAR":"A°¼p6", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.034935] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.035181] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2110}, to_seq_no={branch:0, seq:1051}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286205} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.039683] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=27][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f576138 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286208} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1034}, prev=this=0x2b0f7f4eead0 trans_version={val:1729061837311396000, v:0} scn={val:1729061837311396000, v:0} tx_id={txid:392285289} prev=0x2b0f7f4ee990 next=(nil) modify_count=1 acc_checksum=1030148699 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3157}, snapshot_version={val:1729061897039067000, v:0}, *this={this=0x2b0f7f4ee688 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4eead0 latest_compact_node=(nil) max_trans_version={val:1729061837311396000, v:0} max_trans_id=392285289 max_elr_trans_version={val:1729061837311396000, v:0} max_elr_trans_id=392285289 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837311396000, v:0} min_modify_scn={val:1729061837311396000, v:0}}) [2024-10-16 14:58:17.039722] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f576138 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286208} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1034}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810328, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810328, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897039067000, v:0}, tx_id:{txid:392286208}, scn:{branch:0, seq:1031}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286208}, tx_desc:{this:0x2b0ec379e210, tx_id:{txid:392286208}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897039067, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897038037, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1034}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.039808] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=85][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286208} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286208}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1034}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.039822] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654910192361263104},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286208} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.039834] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.039840] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192361263104},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.039887] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=27][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.040155] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1038}, to_seq_no={branch:0, seq:1032}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286208} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.040510] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be4c668 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286209} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, prev=this=0x2b0ec6dd9420 trans_version={val:1729061839149522000, v:0} scn={val:1729061839149522000, v:0} tx_id={txid:392285706} prev=0x2b0ec6dd92e0 next=(nil) modify_count=3 acc_checksum=1992114948 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2086}, snapshot_version={val:1729061897040128000, v:0}, *this={this=0x2b0ec6d42030 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0ec6dd9420 latest_compact_node=(nil) max_trans_version={val:1729061839149522000, v:0} max_trans_id=392285706 max_elr_trans_version={val:1729061839149522000, v:0} max_elr_trans_id=392285706 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839149522000, v:0} min_modify_scn={val:1729061778615657000, v:0}}) [2024-10-16 14:58:17.040533] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=22][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be4c668 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286209} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1060}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851201, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851201, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897040128000, v:0}, tx_id:{txid:392286209}, scn:{branch:0, seq:1057}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286209}, tx_desc:{this:0x2b0ec37afeb0, tx_id:{txid:392286209}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897040128, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897039072, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1060}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.040574] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=40][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286209} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286209}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:1060}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.040586] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11][errcode=0] primary key duplication conflict(key={"BIGINT":654830042521128960},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286209} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.040607] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.040611] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042521128960},{"BIGINT":1729058400},{"VARCHAR":"?Ô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.040645] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.040883] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1063}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286209} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.048059] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:360) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] ====== check clog disk timer task ====== [2024-10-16 14:58:17.048072] INFO [PALF] get_disk_usage (palf_env_impl.cpp:894) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=10] get_disk_usage(ret=0, capacity(MB):=2048, used(MB):=1622) [2024-10-16 14:58:17.048084] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:267) [21593][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] cannot_recycle_log_size statistics(cannot_recycle_log_size=292067619, threshold=644245094, need_update_checkpoint_scn=false) [2024-10-16 14:58:17.048587] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22235][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1729061897047526000, start_id=1729061897047526000, end_id=1729061897047526001) [2024-10-16 14:58:17.052855] INFO eloop_run (eloop.c:140) [21189][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO [ratelimit] time: 1729061897052854, bytes: 4446831098556, bw: 2.151112 MB/s, add_ts: 1000452, add_bytes: 2256624 [2024-10-16 14:58:17.054295] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be4cfc0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286213} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5537}, prev=this=0x2b0ec6dd9b00 trans_version={val:1729061839154904001, v:0} scn={val:1729061839154904001, v:0} tx_id={txid:392285708} prev=0x2b0ec6dd99c0 next=(nil) modify_count=3 acc_checksum=3974994711 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3274}, snapshot_version={val:1729061897052918000, v:0}, *this={this=0x2b0ec6d42730 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0ec6dd9b00 latest_compact_node=(nil) max_trans_version={val:1729061839154904001, v:0} max_trans_id=392285708 max_elr_trans_version={val:1729061839154904001, v:0} max_elr_trans_id=392285708 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839154904001, v:0} min_modify_scn={val:1729061778689469001, v:0}}) [2024-10-16 14:58:17.054328] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be4cfc0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286213} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5537}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495852213, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495852213, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897052918000, v:0}, tx_id:{txid:392286213}, scn:{branch:0, seq:3161}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286213}, tx_desc:{this:0x2b0ec37bdc10, tx_id:{txid:392286213}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897054147, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897048611, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:5537}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.054389] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286213} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286213}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:5537}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.054406] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830042584043520},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286213} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.054420] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.054429] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042584043520},{"BIGINT":1729058400},{"VARCHAR":"Aô", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.054476] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.054775] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5540}, to_seq_no={branch:0, seq:3162}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286213} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.055059] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:128) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=4] ====== tenant freeze timer task ====== [2024-10-16 14:58:17.055069] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1656) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=7] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=146800640, ctx.total_memstore_used_=245366784, ctx.total_memstore_hold_=245366784, memstore_freeze_trigger_limit_=151364800, tenant_id=1) [2024-10-16 14:58:17.055081] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1660) [21469][T1_Occam][T1][YB4A0A000407-00061E9984F960C2-0-0] [lt=6] [TenantFreezer] oldest frozen memtable(list="") [2024-10-16 14:58:17.056029] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb4ffc8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286215} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0fadaf5828 trans_version={val:1729061837328811000, v:0} scn={val:1729061837328811000, v:0} tx_id={txid:392285293} prev=0x2b0fadaf56e8 next=(nil) modify_count=1 acc_checksum=845985774 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5307}, snapshot_version={val:1729061897055209000, v:0}, *this={this=0x2b0fadaf53e0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaf5828 latest_compact_node=(nil) max_trans_version={val:1729061837328811000, v:0} max_trans_id=392285293 max_elr_trans_version={val:1729061837328811000, v:0} max_elr_trans_id=392285293 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837328811000, v:0} min_modify_scn={val:1729061837328811000, v:0}}) [2024-10-16 14:58:17.056071] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=41][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb4ffc8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286215} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810330, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810330, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897055209000, v:0}, tx_id:{txid:392286215}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286215}, tx_desc:{this:0x2b0ec37d1850, tx_id:{txid:392286215}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897055209, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897055209, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.056117] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=45][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286215} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286215}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.056134] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654910192449343488},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286215} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.056153] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.056159] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192449343488},{"BIGINT":1729058400},{"VARCHAR":"Aƒ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.056209] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.056428] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286215} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.057475] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb50380 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286216} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1048}, prev=this=0x2b0fadb0fc78 trans_version={val:1729061839160334001, v:0} scn={val:1729061839160334001, v:0} tx_id={txid:392285709} prev=0x2b0fadb0f810 next=(nil) modify_count=3 acc_checksum=2385139197 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3335}, snapshot_version={val:1729061897056267000, v:0}, *this={this=0x2b0fada9e0f8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0fadb0fc78 latest_compact_node=(nil) max_trans_version={val:1729061839160334001, v:0} max_trans_id=392285709 max_elr_trans_version={val:1729061839160334001, v:0} max_elr_trans_id=392285709 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839160334001, v:0} min_modify_scn={val:1729061778760459000, v:0}}) [2024-10-16 14:58:17.057520] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=45][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb50380 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286216} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1048}, ctx={this:0x2b0e2b854080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851279, tablet_id:{id:209279}, table_iter:0x2b0e2b853210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851279, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897056267000, v:0}, tx_id:{txid:392286216}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286216}, tx_desc:{this:0x2b0e0fceb970, tx_id:{txid:392286216}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897057316, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897056270, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1048}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.057567] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=47][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286216} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286216}, data:0x2b0e2b850438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1048}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.057583] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830042634375168},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286216} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.057596] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.057607] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042634375168},{"BIGINT":1729058400},{"VARCHAR":"?à", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.057659] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=36][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.057802] INFO [LIB] stat (utility.h:1165) [107618][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD633-0-0] [lt=22] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=3, this=0x2b0e431ed6a0, extra_info="palf_id:1001") [2024-10-16 14:58:17.057813] INFO [LIB] stat (utility.h:1165) [107618][T1004_L0_G1][T1004][YB4A0A000408-00061E9B805CD633-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1029, this=0x2b0e431ed8d0, extra_info="palf_id:1001") [2024-10-16 14:58:17.058765] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2117}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286216} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.061516] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=31][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22f02248 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286219} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0f22ecef10 trans_version={val:1729061839163510000, v:0} scn={val:1729061839163510000, v:0} tx_id={txid:392285710} prev=0x2b0f22ecedd0 next=(nil) modify_count=3 acc_checksum=3081204072 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2111}, snapshot_version={val:1729061897060503001, v:0}, *this={this=0x2b0f22e79ad0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0f22ecef10 latest_compact_node=(nil) max_trans_version={val:1729061839163510000, v:0} max_trans_id=392285710 max_elr_trans_version={val:1729061839163510000, v:0} max_elr_trans_id=392285710 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839163510000, v:0} min_modify_scn={val:1729061778825173000, v:0}}) [2024-10-16 14:58:17.061556] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=39][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22f02248 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286219} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851213, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851213, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897060503001, v:0}, tx_id:{txid:392286219}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286219}, tx_desc:{this:0x2b0ec37c99d0, tx_id:{txid:392286219}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897060503, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897060505, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.061610] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286219} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286219}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.061625] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830042747621376},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286219} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.061638] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.061643] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042747621376},{"BIGINT":1729058400},{"VARCHAR":"B", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.061682] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.061941] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1073}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286219} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.064612] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb51000 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286221} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0fadb10358 trans_version={val:1729061839166707000, v:0} scn={val:1729061839166707000, v:0} tx_id={txid:392285711} prev=0x2b0fadb10218 next=(nil) modify_count=3 acc_checksum=3810100756 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2123}, snapshot_version={val:1729061897063669001, v:0}, *this={this=0x2b0fada9e7f8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0fadb10358 latest_compact_node=(nil) max_trans_version={val:1729061839166707000, v:0} max_trans_id=392285711 max_elr_trans_version={val:1729061839166707000, v:0} max_elr_trans_id=392285711 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839166707000, v:0} min_modify_scn={val:1729061778831515000, v:0}}) [2024-10-16 14:58:17.064642] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb51000 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286221} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851222, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851222, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897063669001, v:0}, tx_id:{txid:392286221}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286221}, tx_desc:{this:0x2b0ec37c7a30, tx_id:{txid:392286221}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897063669, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897063669, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.064689] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=47][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286221} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286221}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.064705] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830042823118848},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286221} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.064718] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.064723] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830042823118848},{"BIGINT":1729058400},{"VARCHAR":"Aínb€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.064767] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.064937] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286221} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.065089] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=21][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be4dc20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286222} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1063}, prev=this=0x2b0ec6dbe8a8 trans_version={val:1729061837348540000, v:0} scn={val:1729061837348540000, v:0} tx_id={txid:392285297} prev=0x2b0ec6dbe768 next=(nil) modify_count=1 acc_checksum=3013142779 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4248}, snapshot_version={val:1729061897064732000, v:0}, *this={this=0x2b0ec6dbe460 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6dbe8a8 latest_compact_node=(nil) max_trans_version={val:1729061837348540000, v:0} max_trans_id=392285297 max_elr_trans_version={val:1729061837348540000, v:0} max_elr_trans_id=392285297 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837348540000, v:0} min_modify_scn={val:1729061837348540000, v:0}}) [2024-10-16 14:58:17.065111] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=21][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be4dc20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286222} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1063}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810315, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810315, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897064732000, v:0}, tx_id:{txid:392286222}, scn:{branch:0, seq:1060}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286222}, tx_desc:{this:0x2b0ec37abf70, tx_id:{txid:392286222}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897064732, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897063673, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1063}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.065152] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=40][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286222} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286222}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:1063}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.065166] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654910192524840960},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286222} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.065178] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.065182] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192524840960},{"BIGINT":1729058400},{"VARCHAR":"Ả ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.065210] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=16][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.065684] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=4] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286222} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.073527] INFO [LIB] stat (utility.h:1165) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54092-0-0] [lt=4] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e43d33470, extra_info="") [2024-10-16 14:58:17.073548] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54092-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.0.4.7:2890", committed size=1342) [2024-10-16 14:58:17.073560] INFO [LIB] stat (utility.h:1165) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54092-0-0] [lt=6] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x2b0e43d336a0, extra_info="palf_id:1003") [2024-10-16 14:58:17.073564] INFO [LIB] stat (utility.h:1165) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54092-0-0] [lt=4] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=50480, this=0x2b0e43d338d0, extra_info="palf_id:1003") [2024-10-16 14:58:17.073568] WDIAG [PALF] sliding_cb (log_sliding_window.cpp:2280) [107618][T1004_L0_G1][T1004][YB4A0A000409-00061E9A3BD54092-0-0] [lt=3][errcode=-4389] log_task life cost too much time(palf_id=1003, self="10.0.4.7:2890", log_id=115173504, log_task={header:{begin_lsn:{lsn:14352417580}, end_lsn:{lsn:14352417702}, log_id:115173504, min_scn:{val:1729061896680014002, v:0}, max_scn:{val:1729061896680014002, v:0}, data_len:66, proposal_id:17, prev_lsn:{lsn:14352417458}, prev_proposal_id:17, committed_end_lsn:{lsn:14352417580}, data_checksum:940009244, accum_checksum:1104335085, is_padding_log:false, is_raw_write:false}, state_map:{val:58}, ref_cnt:0, gen_ts:1729061896973487, freeze_ts:1729061896973487, submit_ts:1729061896973490, flushed_ts:1729061896974509, gen_to_freeze cost time:0, gen_to_submit cost time:3, submit_to_flush cost time:1019}, fs_cb_begin_ts=1729061897073556, log_life_time=100069) [2024-10-16 14:58:17.073762] INFO [LIB] stat (utility.h:1165) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=453, this=0x2b0e43d3da70, extra_info="") [2024-10-16 14:58:17.073772] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:17.073792] INFO [LIB] stat (utility.h:1182) [22613][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=137, this=0x2b0e0f9bbaf0, extra_info="") [2024-10-16 14:58:17.073815] INFO [LIB] stat (utility.h:1165) [22609][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16, this=0x2b0e43d3dca0, extra_info="") [2024-10-16 14:58:17.074610] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be4e608 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286228} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1041}, prev=this=0x2b0ec6dbefa8 trans_version={val:1729061837358861000, v:0} scn={val:1729061837358861000, v:0} tx_id={txid:392285300} prev=0x2b0ec6dbee68 next=(nil) modify_count=1 acc_checksum=3188262205 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3965}, snapshot_version={val:1729061897074310000, v:0}, *this={this=0x2b0ec6dbeb60 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6dbefa8 latest_compact_node=(nil) max_trans_version={val:1729061837358861000, v:0} max_trans_id=392285300 max_elr_trans_version={val:1729061837358861000, v:0} max_elr_trans_id=392285300 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837358861000, v:0} min_modify_scn={val:1729061837358861000, v:0}}) [2024-10-16 14:58:17.074640] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be4e608 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286228} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1041}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810325, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810325, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897074310000, v:0}, tx_id:{txid:392286228}, scn:{branch:0, seq:1038}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286228}, tx_desc:{this:0x2b0ec379a2d0, tx_id:{txid:392286228}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897074310, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897073273, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1041}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.074700] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286228} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286228}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:1041}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.074717] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654910192587755520},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286228} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.074731] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=13][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.074736] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192587755520},{"BIGINT":1729058400},{"VARCHAR":"AÉùöM€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.074782] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.075039] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1045}, to_seq_no={branch:0, seq:1039}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286228} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.076763] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1729061897076404000, start_id=1729061897076404000, end_id=1729061897076404001) [2024-10-16 14:58:17.078042] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f576b20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286230} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1058}, prev=this=0x2b0f7f512e80 trans_version={val:1729061839182822000, v:0} scn={val:1729061839182822000, v:0} tx_id={txid:392285716} prev=0x2b0f7f512d40 next=(nil) modify_count=3 acc_checksum=3361709105 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2121}, snapshot_version={val:1729061897076404001, v:0}, *this={this=0x2b0f7f48e460 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0f7f512e80 latest_compact_node=(nil) max_trans_version={val:1729061839182822000, v:0} max_trans_id=392285716 max_elr_trans_version={val:1729061839182822000, v:0} max_elr_trans_id=392285716 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839182822000, v:0} min_modify_scn={val:1729061778858356000, v:0}}) [2024-10-16 14:58:17.078069] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f576b20 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286230} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1058}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851219, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851219, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897076404001, v:0}, tx_id:{txid:392286230}, scn:{branch:0, seq:5}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286230}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392286230}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897077464, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897076407, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1058}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.078118] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=49][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286230} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286230}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1058}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.078131] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654830043112525824},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286230} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.078152] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.078156] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043112525824},{"BIGINT":1729058400},{"VARCHAR":"BG½$", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.078195] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.078659] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2123}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286230} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.079092] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21292][TsMgr][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061897078529000, start_id=1729061897078529000, end_id=1729061897078529001) [2024-10-16 14:58:17.079112] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1729061897079087], gts:1729061897078529000, latest_srr:[mts=1729061897079087]}) [2024-10-16 14:58:17.079121] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [21292][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2024-10-16 14:58:17.084479] INFO [LIB] stat (utility.h:1165) [22053][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG IO WORKER QUEUE SIZE](cur_stat_count=6, stat_interval=1000000, avg (count/cost)=1, this=0x2b0e0d9bb068, extra_info="log_io_worker_num:4, cb_thread_pool_tg_id:485") [2024-10-16 14:58:17.085484] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb51958 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286237} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1059}, prev=this=0x2b0fadaf5f28 trans_version={val:1729061837379002000, v:0} scn={val:1729061837379002000, v:0} tx_id={txid:392285304} prev=0x2b0fadaf5de8 next=(nil) modify_count=1 acc_checksum=1846039482 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3572}, snapshot_version={val:1729061897083820002, v:0}, *this={this=0x2b0fadaf5ae0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaf5f28 latest_compact_node=(nil) max_trans_version={val:1729061837379002000, v:0} max_trans_id=392285304 max_elr_trans_version={val:1729061837379002000, v:0} max_elr_trans_id=392285304 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837379002000, v:0} min_modify_scn={val:1729061837379002000, v:0}}) [2024-10-16 14:58:17.085517] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb51958 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286237} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1059}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810332, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810332, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897083820002, v:0}, tx_id:{txid:392286237}, scn:{branch:0, seq:8}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286237}, tx_desc:{this:0x2b0ec37cf8b0, tx_id:{txid:392286237}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897084881, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897083825, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1059}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.085573] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=55][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286237} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286237}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1059}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.085589] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654910192763916288},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286237} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.085612] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.085617] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192763916288},{"BIGINT":1729058400},{"VARCHAR":"APh€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.085665] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.085977] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2124}, to_seq_no={branch:0, seq:9}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286237} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.086177] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22f02ba0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286238} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1067}, prev=this=0x2b0f22ed03f0 trans_version={val:1729061839189947001, v:0} scn={val:1729061839189947001, v:0} tx_id={txid:392285720} prev=0x2b0f22ed02b0 next=(nil) modify_count=3 acc_checksum=2918575886 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2772}, snapshot_version={val:1729061897084881001, v:0}, *this={this=0x2b0f22e83878 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0f22ed03f0 latest_compact_node=(nil) max_trans_version={val:1729061839189947001, v:0} max_trans_id=392285720 max_elr_trans_version={val:1729061839189947001, v:0} max_elr_trans_id=392285720 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839189947001, v:0} min_modify_scn={val:1729061778878082000, v:0}}) [2024-10-16 14:58:17.086206] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22f02ba0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286238} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1067}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851215, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851215, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897084881001, v:0}, tx_id:{txid:392286238}, scn:{branch:0, seq:5}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286238}, tx_desc:{this:0x2b0ec37cb970, tx_id:{txid:392286238}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897085948, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897084883, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1067}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.086261] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286238} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286238}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:1067}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.086277] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830043250937856},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286238} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.086329] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=51][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.086335] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043250937856},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.086375] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.086723] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1071}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286238} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.088597] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=6][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f48f290 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286239} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1067}, prev=this=0x2b0f3f427270 trans_version={val:1729061837384815000, v:0} scn={val:1729061837384815000, v:0} tx_id={txid:392285305} prev=0x2b0f3f427130 next=(nil) modify_count=1 acc_checksum=3740814763 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3163}, snapshot_version={val:1729061897088057000, v:0}, *this={this=0x2b0f3f426e28 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f427270 latest_compact_node=(nil) max_trans_version={val:1729061837384815000, v:0} max_trans_id=392285305 max_elr_trans_version={val:1729061837384815000, v:0} max_elr_trans_id=392285305 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837384815000, v:0} min_modify_scn={val:1729061837384815000, v:0}}) [2024-10-16 14:58:17.088629] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f48f290 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286239} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1067}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810362, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810362, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897088057000, v:0}, tx_id:{txid:392286239}, scn:{branch:0, seq:1064}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286239}, tx_desc:{this:0x2b0ec37c99d0, tx_id:{txid:392286239}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897088057, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897086994, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1067}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.088673] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=42][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286239} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286239}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:1067}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.088686] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=12][errcode=0] primary key duplication conflict(key={"BIGINT":654910192789082112},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286239} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.088696] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.088700] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192789082112},{"BIGINT":1729058400},{"VARCHAR":"APi€", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.088732] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=18][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.088747] INFO [PALF] sliding_cb (log_sliding_window.cpp:2269) [107580][T1002_L0_G1][T1002][YB4A0A000409-00061E9A3B9F45F6-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="10.0.4.7:2890", total_slide_log_cnt=7, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=27, avg_log_submit_to_flush_time=494, avg_log_submit_to_slide_time=855) [2024-10-16 14:58:17.088989] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1072}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286239} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.091248] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3f48f970 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286241} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:10}, prev=this=0x2b0f3f427970 trans_version={val:1729061837392253000, v:0} scn={val:1729061837392253000, v:0} tx_id={txid:392285306} prev=0x2b0f3f427830 next=(nil) modify_count=1 acc_checksum=4136456675 version=1729061778354485 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4254}, snapshot_version={val:1729061897090173001, v:0}, *this={this=0x2b0f3f427528 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f3f427970 latest_compact_node=(nil) max_trans_version={val:1729061837392253000, v:0} max_trans_id=392285306 max_elr_trans_version={val:1729061837392253000, v:0} max_elr_trans_id=392285306 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837392253000, v:0} min_modify_scn={val:1729061837392253000, v:0}}) [2024-10-16 14:58:17.091282] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3f48f970 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286241} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778354485 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:10}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810334, tablet_id:{id:209273}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810334, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897090173001, v:0}, tx_id:{txid:392286241}, scn:{branch:0, seq:7}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286241}, tx_desc:{this:0x2b0ec37c7a30, tx_id:{txid:392286241}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897090173, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897090176, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:10}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.091339] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286241} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286241}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778354485, seq_no:{branch:0, seq:10}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.091356] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654910192851996672},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286241} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.091369] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.091373] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910192851996672},{"BIGINT":1729058400},{"VARCHAR":"Ao<`", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.091408] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.091629] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1072}, to_seq_no={branch:0, seq:8}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286241} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.092703] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1755) [107580][T1002_L0_G1][T1002][YB4A0A000408-00061E9B802DB3AE-0-0] [lt=27] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self="10.0.4.7:2890", committed size=149399) [2024-10-16 14:58:17.092759] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:17.092772] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"10.0.4.7:2890", switch_to_leader_ts:-1}) [2024-10-16 14:58:17.092815] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=42][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:17.097626] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f577780 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286246} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1099}, prev=this=0x2b0f7f513560 trans_version={val:1729061839205995001, v:0} scn={val:1729061839205995001, v:0} tx_id={txid:392285724} prev=0x2b0f7f513420 next=(nil) modify_count=3 acc_checksum=2395037402 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7579}, snapshot_version={val:1729061897096646000, v:0}, *this={this=0x2b0f7f48f260 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0f7f513560 latest_compact_node=(nil) max_trans_version={val:1729061839205995001, v:0} max_trans_id=392285724 max_elr_trans_version={val:1729061839205995001, v:0} max_elr_trans_id=392285724 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839205995001, v:0} min_modify_scn={val:1729061778912395000, v:0}}) [2024-10-16 14:58:17.097657] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f577780 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286246} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1099}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851201, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851201, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897096646000, v:0}, tx_id:{txid:392286246}, scn:{branch:0, seq:1094}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286246}, tx_desc:{this:0x2b0ec379e210, tx_id:{txid:392286246}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897096646, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897095554, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1099}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.097715] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286246} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286246}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1099}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.097732] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830043364184064},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286246} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.097747] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.097752] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043364184064},{"BIGINT":1729058400},{"VARCHAR":"A­±ÊØ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.097796] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=24][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.097971] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2147}, to_seq_no={branch:0, seq:1095}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286246} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.106495] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061897106171000, start_id=1729061897106171000, end_id=1729061897106171001) [2024-10-16 14:58:17.109035] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22f03800 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286257} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1083}, prev=this=0x2b0f22ed1bd8 trans_version={val:1729061839223545001, v:0} scn={val:1729061839223545001, v:0} tx_id={txid:392285730} prev=0x2b0f22ed1a98 next=(nil) modify_count=3 acc_checksum=3272171228 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3153}, snapshot_version={val:1729061897108327000, v:0}, *this={this=0x2b0f22e84350 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0f22ed1bd8 latest_compact_node=(nil) max_trans_version={val:1729061839223545001, v:0} max_trans_id=392285730 max_elr_trans_version={val:1729061839223545001, v:0} max_elr_trans_id=392285730 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839223545001, v:0} min_modify_scn={val:1729061778952034000, v:0}}) [2024-10-16 14:58:17.109073] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=37][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22f03800 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286257} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1083}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851199, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851199, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897108327000, v:0}, tx_id:{txid:392286257}, scn:{branch:0, seq:1080}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286257}, tx_desc:{this:0x2b0ec37cd910, tx_id:{txid:392286257}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897108327, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897107252, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1083}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.109130] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286257} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286257}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:1083}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.109147] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17][errcode=0] primary key duplication conflict(key={"BIGINT":654830043565510656},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286257} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.109159] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.109165] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043565510656},{"BIGINT":1729058400},{"VARCHAR":"AªÜÛ”", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.109214] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.109556] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2134}, to_seq_no={branch:0, seq:1081}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286257} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.110545] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f22f03bb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286259} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:8}, prev=this=0x2b0f22ebd398 trans_version={val:1729061837425085000, v:0} scn={val:1729061837425085000, v:0} tx_id={txid:392285315} prev=0x2b0f22ebd258 next=(nil) modify_count=1 acc_checksum=1690218799 version=1729061778338659 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3145}, snapshot_version={val:1729061897109385000, v:0}, *this={this=0x2b0f22ebcf50 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f22ebd398 latest_compact_node=(nil) max_trans_version={val:1729061837425085000, v:0} max_trans_id=392285315 max_elr_trans_version={val:1729061837425085000, v:0} max_elr_trans_id=392285315 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837425085000, v:0} min_modify_scn={val:1729061837425085000, v:0}}) [2024-10-16 14:58:17.110636] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=91][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f22f03bb8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286259} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778338659 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:8}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810311, tablet_id:{id:209267}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810311, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897109385000, v:0}, tx_id:{txid:392286259}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286259}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392286259}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897110449, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897109386, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:8}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.110701] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=64][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286259} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286259}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778338659, seq_no:{branch:0, seq:8}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.110722] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=21][errcode=0] primary key duplication conflict(key={"BIGINT":654910193191735296},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286259} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.110746] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=22][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.110756] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910193191735296},{"BIGINT":1729058400},{"VARCHAR":"@¹.", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.110809] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=31][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.110988] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286259} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.112143] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061893108199, last_first_register_time_us=1722494150998907, register_time_us=1729061893108199, curr_time_us=1729061897112141) [2024-10-16 14:58:17.112173] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061897112160, last_submit_register_req_time_us=1729061897112160, register_parent_reason=3) [2024-10-16 14:58:17.112187] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22058][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:17.114734] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb52038 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286262} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0fadaf6628 trans_version={val:1729061837431466000, v:0} scn={val:1729061837431466000, v:0} tx_id={txid:392285317} prev=0x2b0fadaf64e8 next=(nil) modify_count=1 acc_checksum=3322813128 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:5315}, snapshot_version={val:1729061897113663000, v:0}, *this={this=0x2b0fadaf61e0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0fadaf6628 latest_compact_node=(nil) max_trans_version={val:1729061837431466000, v:0} max_trans_id=392285317 max_elr_trans_version={val:1729061837431466000, v:0} max_elr_trans_id=392285317 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837431466000, v:0} min_modify_scn={val:1729061837431466000, v:0}}) [2024-10-16 14:58:17.114795] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=60][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb52038 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286262} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810322, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810322, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897113663000, v:0}, tx_id:{txid:392286262}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286262}, tx_desc:{this:0x2b0ec37cb970, tx_id:{txid:392286262}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897113663, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897113663, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.114845] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=50][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286262} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286262}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.114859] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654910193267232768},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286262} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.114870] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.114875] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910193267232768},{"BIGINT":1729058400},{"VARCHAR":"@È", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.114940] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.115148] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1361}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286262} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.115623] INFO [PALF] check_parent_health (log_config_mgr.cpp:2803) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=15] re_register_parent reason(palf_id=1003, self="10.0.4.7:2890", is_registering_timeout=true, first_registration=false, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=1729061893104510, last_first_register_time_us=1722494157071428, register_time_us=1729061893104510, curr_time_us=1729061897115621) [2024-10-16 14:58:17.115650] INFO [PALF] register_parent_ (log_config_mgr.cpp:2650) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=13] register_parent_(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890", reason=3, leader="10.0.4.9:2890", register_time_us=1729061897115637, last_submit_register_req_time_us=1729061897115637, register_parent_reason=3) [2024-10-16 14:58:17.115658] INFO [PALF] check_parent_health (log_config_mgr.cpp:2812) [22617][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=7] re register_parent success(ret=0, ret="OB_SUCCESS", palf_id=1003, self="10.0.4.7:2890") [2024-10-16 14:58:17.115957] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb52360 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286263} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0fadb11138 trans_version={val:1729061839232328000, v:0} scn={val:1729061839232328000, v:0} tx_id={txid:392285732} prev=0x2b0fadb10ff8 next=(nil) modify_count=3 acc_checksum=279349860 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3178}, snapshot_version={val:1729061897115023000, v:0}, *this={this=0x2b0fada9eef8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0fadb11138 latest_compact_node=(nil) max_trans_version={val:1729061839232328000, v:0} max_trans_id=392285732 max_elr_trans_version={val:1729061839232328000, v:0} max_elr_trans_id=392285732 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839232328000, v:0} min_modify_scn={val:1729061778982964001, v:0}}) [2024-10-16 14:58:17.115982] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=25][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb52360 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286263} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851207, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851207, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897115023000, v:0}, tx_id:{txid:392286263}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286263}, tx_desc:{this:0x2b0ec37b7d30, tx_id:{txid:392286263}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897115023, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897115024, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.116037] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=54][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286263} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286263}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.116053] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830043641008128},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286263} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.116066] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.116071] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043641008128},{"BIGINT":1729058400},{"VARCHAR":"Br åø", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.116108] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.116277] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1057}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286263} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.118634] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:17.120787] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=4] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1002, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2024-10-16 14:58:17.121469] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:427) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=14] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2024-10-16 14:58:17.121504] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=18][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f5786e8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286267} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, prev=this=0x2b0f7f4ef980 trans_version={val:1729061837438837000, v:0} scn={val:1729061837438837000, v:0} tx_id={txid:392285319} prev=0x2b0f7f4ef840 next=(nil) modify_count=1 acc_checksum=4063463896 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3130}, snapshot_version={val:1729061897120376001, v:0}, *this={this=0x2b0f7f4ef538 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4ef980 latest_compact_node=(nil) max_trans_version={val:1729061837438837000, v:0} max_trans_id=392285319 max_elr_trans_version={val:1729061837438837000, v:0} max_elr_trans_id=392285319 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837438837000, v:0} min_modify_scn={val:1729061837438837000, v:0}}) [2024-10-16 14:58:17.121525] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [22135][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=19] inner load succ(latest_freeze_info={frozen_version:{val:1729015204839740000, v:0}, schema_version:1729008953019688, data_version:17180066304}, freeze_info_={latest_snapshot_gc_scn:{val:1729061890214512000, v:0}, frozen_statuses_.count():33, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180066048}, {frozen_version:{val:1726336803027591000, v:0}, schema_version:1726330447530864, data_version:17180066304}, {frozen_version:{val:1726423201599680000, v:0}, schema_version:1726416850921416, data_version:17180066304}, {frozen_version:{val:1726509600418208000, v:0}, schema_version:1726503253979528, data_version:17180066304}, {frozen_version:{val:1726596003869606000, v:0}, schema_version:1726589657412560, data_version:17180066304}, {frozen_version:{val:1726682402414737000, v:0}, schema_version:1726676060732912, data_version:17180066304}, {frozen_version:{val:1726768801279913000, v:0}, schema_version:1726762464009480, data_version:17180066304}, {frozen_version:{val:1726855203468001000, v:0}, schema_version:1726848866954512, data_version:17180066304}, {frozen_version:{val:1726941601501247000, v:0}, schema_version:1726935269817704, data_version:17180066304}, {frozen_version:{val:1727028000405859000, v:0}, schema_version:1727021672924832, data_version:17180066304}, {frozen_version:{val:1727114403159554000, v:0}, schema_version:1727108076097088, data_version:17180066304}, {frozen_version:{val:1727200801973619000, v:0}, schema_version:1727194479248832, data_version:17180066304}, {frozen_version:{val:1727287204233303000, v:0}, schema_version:1727280881926464, data_version:17180066304}, {frozen_version:{val:1727373603437629000, v:0}, schema_version:1727367285117040, data_version:17180066304}, {frozen_version:{val:1727460001722137000, v:0}, schema_version:1727453688315936, data_version:17180066304}, {frozen_version:{val:1727546403657975000, v:0}, schema_version:1727540093725336, data_version:17180066304}, {frozen_version:{val:1727632802050730000, v:0}, schema_version:1727626497164864, data_version:17180066304}, {frozen_version:{val:1727719200685235000, v:0}, schema_version:1727712900844584, data_version:17180066304}, {frozen_version:{val:1727805604484914000, v:0}, schema_version:1727799304077032, data_version:17180066304}, {frozen_version:{val:1727892002796507001, v:0}, schema_version:1727885707212016, data_version:17180066304}, {frozen_version:{val:1727978404750455000, v:0}, schema_version:1727972110103184, data_version:17180066304}, {frozen_version:{val:1728064803590053000, v:0}, schema_version:1728058513541616, data_version:17180066304}, {frozen_version:{val:1728151202274374000, v:0}, schema_version:1728144917434528, data_version:17180066304}, {frozen_version:{val:1728237600677416000, v:0}, schema_version:1728231320972160, data_version:17180066304}, {frozen_version:{val:1728324004745798000, v:0}, schema_version:1728317724299072, data_version:17180066304}, {frozen_version:{val:1728410403682257000, v:0}, schema_version:1728404127467920, data_version:17180066304}, {frozen_version:{val:1728496802271875000, v:0}, schema_version:1728490531529992, data_version:17180066304}, {frozen_version:{val:1728583204352655000, v:0}, schema_version:1728576934974112, data_version:17180066304}, {frozen_version:{val:1728669603150919000, v:0}, schema_version:1728663338039328, data_version:17180066304}, {frozen_version:{val:1728756002024115000, v:0}, schema_version:1728749741449960, data_version:17180066304}, {frozen_version:{val:1728842400420020000, v:0}, schema_version:1728836144836096, data_version:17180066304}, {frozen_version:{val:1728928802547408000, v:0}, schema_version:1728922548218920, data_version:17180066304}]}) [2024-10-16 14:58:17.121534] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f5786e8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286267} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1062}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810322, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810322, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897120376001, v:0}, tx_id:{txid:392286267}, scn:{branch:0, seq:1059}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286267}, tx_desc:{this:0x2b0ec379e210, tx_id:{txid:392286267}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897120376, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897119318, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1062}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.121580] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=45][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286267} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286267}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1062}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.121594] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654910193304981504},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286267} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.121611] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.121616] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910193304981504},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.121654] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.122131] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2376}, to_seq_no={branch:0, seq:1060}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286267} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.122244] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f578a10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286268} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1317}, prev=this=0x2b0f7f514628 trans_version={val:1729061839241480000, v:0} scn={val:1729061839241480000, v:0} tx_id={txid:392285735} prev=0x2b0f7f5144e8 next=(nil) modify_count=3 acc_checksum=1173490849 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3159}, snapshot_version={val:1729061897121691000, v:0}, *this={this=0x2b0f7f490810 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0f7f514628 latest_compact_node=(nil) max_trans_version={val:1729061839241480000, v:0} max_trans_id=392285735 max_elr_trans_version={val:1729061839241480000, v:0} max_elr_trans_id=392285735 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839241480000, v:0} min_modify_scn={val:1729061779011214000, v:0}}) [2024-10-16 14:58:17.122268] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=23][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f578a10 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286268} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1317}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851205, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851205, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897121691000, v:0}, tx_id:{txid:392286268}, scn:{branch:0, seq:1314}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286268}, tx_desc:{this:0x2b0ec37afeb0, tx_id:{txid:392286268}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897121691, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897120378, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1317}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.122318] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=50][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286268} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286268}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1317}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.122333] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654830043741671424},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286268} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.122346] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.122351] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043741671424},{"BIGINT":1729058400},{"VARCHAR":"", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.122388] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.122609] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1321}, to_seq_no={branch:0, seq:1315}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286268} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.125877] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=9][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be4ece8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286272} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:8}, prev=this=0x2b0ec6ddabe8 trans_version={val:1729061839251004000, v:0} scn={val:1729061839251004000, v:0} tx_id={txid:392285737} prev=0x2b0ec6ddaaa8 next=(nil) modify_count=3 acc_checksum=3164338106 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7410}, snapshot_version={val:1729061897124864003, v:0}, *this={this=0x2b0ec6d44b90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0ec6ddabe8 latest_compact_node=(nil) max_trans_version={val:1729061839251004000, v:0} max_trans_id=392285737 max_elr_trans_version={val:1729061839251004000, v:0} max_elr_trans_id=392285737 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839251004000, v:0} min_modify_scn={val:1729061779038478000, v:0}}) [2024-10-16 14:58:17.125905] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be4ece8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286272} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:8}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851197, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851197, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897124864003, v:0}, tx_id:{txid:392286272}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286272}, tx_desc:{this:0x2b0ec379c270, tx_id:{txid:392286272}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897124864, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897124869, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:8}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.125958] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286272} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286272}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:8}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.125975] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":654830043842334720},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286272} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.125988] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.125993] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830043842334720},{"BIGINT":1729058400},{"VARCHAR":"Aþß@xà", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.126031] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.126212] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1199}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286272} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.128913] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:497) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:17.128932] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [22064][T1002_TenantInf][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2024-10-16 14:58:17.130273] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] [COMPACTION DAG STATUS] (start_time=1729061887129230, end_time=1729061897130272, dag_status={}) [2024-10-16 14:58:17.130286] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:17.130293] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:17.130297] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:17.130301] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:17.130304] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:17.130308] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:17.130311] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:17.130314] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:17.130317] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2826) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2024-10-16 14:58:17.130321] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130328] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130332] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130336] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130340] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130343] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130347] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130351] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130355] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:7, dag_type_str:"WRITE_CKPT", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130359] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130362] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:0, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130367] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130371] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130374] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130378] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130382] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130385] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130389] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130393] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130397] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130404] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130408] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130412] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130416] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130419] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130423] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130427] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130431] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130434] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130438] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130442] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130445] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130449] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130453] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130457] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130461] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130465] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130469] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130472] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130476] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130479] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130483] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130487] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130491] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130495] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130498] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130502] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130506] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130509] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130513] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130517] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130520] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130524] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130528] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130531] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130535] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130539] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4272) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2024-10-16 14:58:17.130543] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3591) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2024-10-16 14:58:17.130547] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2024-10-16 14:58:17.130550] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2024-10-16 14:58:17.130553] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2024-10-16 14:58:17.130555] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2024-10-16 14:58:17.130558] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2024-10-16 14:58:17.130560] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2024-10-16 14:58:17.130562] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2024-10-16 14:58:17.130565] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=2] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2024-10-16 14:58:17.130567] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3594) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2024-10-16 14:58:17.130570] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4286) [22391][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=3] dump_dag_status(dag_cnt=0, total_worker_cnt_=43, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2024-10-16 14:58:17.133054] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1133) [21171][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-1181712384, global_cache_size=1579302912, tenant_max_wash_size=5428838, tenant_min_wash_size=5428838, tenant_ids_=[cnt:8, 500, 508, 509, 1, 1001, 1002, 1003, 1004]) [2024-10-16 14:58:17.134562] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be4f948 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286278} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, prev=this=0x2b0ec6dbf6a8 trans_version={val:1729061837460645000, v:0} scn={val:1729061837460645000, v:0} tx_id={txid:392285325} prev=0x2b0ec6dbf568 next=(nil) modify_count=1 acc_checksum=4230045926 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2125}, snapshot_version={val:1729061897133418002, v:0}, *this={this=0x2b0ec6dbf260 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0ec6dbf6a8 latest_compact_node=(nil) max_trans_version={val:1729061837460645000, v:0} max_trans_id=392285325 max_elr_trans_version={val:1729061837460645000, v:0} max_elr_trans_id=392285325 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837460645000, v:0} min_modify_scn={val:1729061837460645000, v:0}}) [2024-10-16 14:58:17.134598] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=34][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be4f948 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286278} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:7}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810341, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810341, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897133418002, v:0}, tx_id:{txid:392286278}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286278}, tx_desc:{this:0x2b0ec37cf8b0, tx_id:{txid:392286278}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897134478, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897133419, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:7}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.134657] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286278} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286278}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:7}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.134671] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654910193846046720},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286278} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.134682] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=10][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.134687] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910193846046720},{"BIGINT":1729058400},{"VARCHAR":"@û£", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.134754] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.135133] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=5] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1062}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286278} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.135276] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=19][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0fadb53070 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286279} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1058}, prev=this=0x2b0fadb11818 trans_version={val:1729061839259513000, v:0} scn={val:1729061839259513000, v:0} tx_id={txid:392285740} prev=0x2b0fadb116d8 next=(nil) modify_count=3 acc_checksum=4168689526 version=1729061778358094 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3196}, snapshot_version={val:1729061897134478000, v:0}, *this={this=0x2b0fada9f5f8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0fadb11818 latest_compact_node=(nil) max_trans_version={val:1729061839259513000, v:0} max_trans_id=392285740 max_elr_trans_version={val:1729061839259513000, v:0} max_elr_trans_id=392285740 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839259513000, v:0} min_modify_scn={val:1729061779071702002, v:0}}) [2024-10-16 14:58:17.135306] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=30][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0fadb53070 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286279} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778358094 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1058}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851271, tablet_id:{id:209279}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851271, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897134478000, v:0}, tx_id:{txid:392286279}, scn:{branch:0, seq:1055}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286279}, tx_desc:{this:0x2b0ec37cb970, tx_id:{txid:392286279}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897134478, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897133424, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1058}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.135349] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=43][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286279} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286279}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778358094, seq_no:{branch:0, seq:1058}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.135364] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=14][errcode=0] primary key duplication conflict(key={"BIGINT":654830044018495488},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286279} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.135377] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.135381] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044018495488},{"BIGINT":1729058400},{"VARCHAR":"@ ", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.135457] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.135666] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2109}, to_seq_no={branch:0, seq:1056}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286279} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.139145] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [21589][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=5631, mem_used=10398720) [2024-10-16 14:58:17.143879] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=16][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f579a28 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286284} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1061}, prev=this=0x2b0f7f4f0458 trans_version={val:1729061837480680000, v:0} scn={val:1729061837480680000, v:0} tx_id={txid:392285328} prev=0x2b0f7f4f0318 next=(nil) modify_count=1 acc_checksum=4042357839 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4243}, snapshot_version={val:1729061897142956001, v:0}, *this={this=0x2b0f7f4f0010 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4f0458 latest_compact_node=(nil) max_trans_version={val:1729061837480680000, v:0} max_trans_id=392285328 max_elr_trans_version={val:1729061837480680000, v:0} max_elr_trans_id=392285328 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837480680000, v:0} min_modify_scn={val:1729061837480680000, v:0}}) [2024-10-16 14:58:17.143910] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=29][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f579a28 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286284} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1061}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810315, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810315, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897142956001, v:0}, tx_id:{txid:392286284}, scn:{branch:0, seq:1058}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286284}, tx_desc:{this:0x2b0ec37b3df0, tx_id:{txid:392286284}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897142956, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897141901, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1061}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.143963] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=52][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286284} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286284}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778319065, seq_no:{branch:0, seq:1061}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.143979] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":654910194122870784},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286284} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.144000] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.144005] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654910194122870784},{"BIGINT":1729058400},{"VARCHAR":"@¨È", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.144048] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=23][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.144344] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=3] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2153}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286284} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.144705] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=21][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0ee461eff0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286285} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779088644 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1096}, prev=this=0x2b0ecd3e12a8 trans_version={val:1729061839269048000, v:0} scn={val:1729061839269048000, v:0} tx_id={txid:392285743} prev=0x2b0ecd3e1168 next=(nil) modify_count=3 acc_checksum=4145824491 version=1729061779088644 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:2081}, snapshot_version={val:1729061897144053000, v:0}, *this={this=0x2b0ecd38a060 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0ecd3e12a8 latest_compact_node=(nil) max_trans_version={val:1729061839269048000, v:0} max_trans_id=392285743 max_elr_trans_version={val:1729061839269048000, v:0} max_elr_trans_id=392285743 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839269048000, v:0} min_modify_scn={val:1729061779111884000, v:0}}) [2024-10-16 14:58:17.144731] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=26][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0ee461eff0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286285} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061779088644 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1096}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851234, tablet_id:{id:209270}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851234, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897144053000, v:0}, tx_id:{txid:392286285}, scn:{branch:0, seq:1093}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286285}, tx_desc:{this:0x2b0ec37a6090, tx_id:{txid:392286285}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897144053, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897142962, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1096}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.144773] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=42][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286285} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286285}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061779088644, seq_no:{branch:0, seq:1096}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.144786] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=12][errcode=0] primary key duplication conflict(key={"BIGINT":654830044228210689},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286285} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.144796] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=9][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.144800] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044228210689},{"BIGINT":1729058400},{"VARCHAR":"A#о", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.144839] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=22][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.145429] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=6] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2185}, to_seq_no={branch:0, seq:1094}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286285} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.149975] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=20][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f3be50330 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286289} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1054}, prev=this=0x2b0ec6ddbcd0 trans_version={val:1729061839291271000, v:0} scn={val:1729061839291271000, v:0} tx_id={txid:392285746} prev=0x2b0ec6ddbb90 next=(nil) modify_count=3 acc_checksum=2712421250 version=1729061778409743 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:4712}, snapshot_version={val:1729061897149376001, v:0}, *this={this=0x2b0ec6d468f0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x2b0ec6ddbcd0 latest_compact_node=(nil) max_trans_version={val:1729061839291271000, v:0} max_trans_id=392285746 max_elr_trans_version={val:1729061839291271000, v:0} max_elr_trans_id=392285746 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=4 max_modify_scn={val:1729061839291271000, v:0} min_modify_scn={val:1729061779147112001, v:0}}) [2024-10-16 14:58:17.150004] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=28][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f3be50330 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286289} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778409743 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1054}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062495851209, tablet_id:{id:209261}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062495851209, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897149376001, v:0}, tx_id:{txid:392286289}, scn:{branch:0, seq:1051}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286289}, tx_desc:{this:0x2b0ec379e210, tx_id:{txid:392286289}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769447, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897149376, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897148328, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c20c5d0, mem_ctx:0x2b0e0c20db50, tx_scn:{branch:0, seq:1054}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.150058] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:339) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, mem_ctx={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286289} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}, arg={tx_id:{txid:392286289}, data:0x2b0e2b7ca438, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1729061778409743, seq_no:{branch:0, seq:1054}, scn:{val:4611686018427387903, v:0}, column_cnt:4}) [2024-10-16 14:58:17.150072] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":654830044534394880},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286289} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.150083] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4116) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=11][errcode=-5024] fail to set memtable(ret=-5024) [2024-10-16 14:58:17.150088] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5334) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=3][errcode=-5024] failed to write table row(ret=-5024, table id=500007, col_descs=[cnt:4, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0, column_id=19 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0], tbl_row={flag:"INSERT", flag_type:0} capacity_=0 is_get=0 from_base=0 trans_id=8217090173679840641 scan_index=0 multi_version_row_flag=0 {first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0} row_val={count=4,cells=[{"BIGINT":654830044534394880},{"BIGINT":1729058400},{"VARCHAR":"B2•Íð", collation:"binary", coercibility:"INVALID"},{"TINYINT":60},] is_sparse_row=0 snapshot_version=0 fast_filtered=0 group_idx_=0 last_purge_ts=0 ) [2024-10-16 14:58:17.150136] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:2863) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=29][errcode=-5024] failed to write row(ret=-5024) [2024-10-16 14:58:17.150385] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=7] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1058}, to_seq_no={branch:0, seq:1052}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286289} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.150546] WDIAG [STORAGE.TRANS] check_double_insert_ (ob_mvcc_row.cpp:933) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=17][errcode=-5024] find double insert node(ret=-5024, node=this=0x2b0f7f57a108 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286290} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1057}, prev=this=0x2b0f7f4f0b58 trans_version={val:1729061837490705000, v:0} scn={val:1729061837490705000, v:0} tx_id={txid:392285330} prev=0x2b0f7f4f0a18 next=(nil) modify_count=1 acc_checksum=2004668930 version=1729061778319065 type=0 flag=12 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:3883}, snapshot_version={val:1729061897149376002, v:0}, *this={this=0x2b0f7f4f0710 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x2b0f7f4f0b58 latest_compact_node=(nil) max_trans_version={val:1729061837490705000, v:0} max_trans_id=392285330 max_elr_trans_version={val:1729061837490705000, v:0} max_elr_trans_id=392285330 latest_compact_ts=0 last_compact_cnt=0 total_trans_node_cnt=2 max_modify_scn={val:1729061837490705000, v:0} min_modify_scn={val:1729061837490705000, v:0}}) [2024-10-16 14:58:17.150567] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:975) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=21][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x2b0f7f57a108 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:392286290} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1729061778319065 type=0 flag=0 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:522} seq_no={branch:0, seq:1057}, ctx={this:0x2b0e2b7ce080, ls_id:{id:1001}, ls:0x2b0e2610c150, branch:0, timeout:1729062496810310, tablet_id:{id:209285}, table_iter:0x2b0e2b7cd210, table_version:1729008952878920, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1729062496810310, tx_lock_timeout_us:-1, snapshot:{version:{val:1729061897149376002, v:0}, tx_id:{txid:392286290}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x2b0e2611aa50, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:392286290}, tx_desc:{this:0x2b0ec37afeb0, tx_id:{txid:392286290}, state:1, addr:"10.0.4.8:2890", tenant_id:1002, session_id:3221769135, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1729061897150438, commit_ts:-1, finish_ts:-1, timeout_us:600000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1717559948, cluster_version:17180066304, seq_base:1729061897149382, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, tx_ctx:0x2b0e0c214b50, mem_ctx:0x2b0e0c2160d0, tx_scn:{branch:0, seq:1057}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false}) [2024-10-16 14:58:17.150617] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910194248699904},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286290} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.150737] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:584) [22461][T1003_LSArchive][T1003][YB4A0A000407-00061E99707960C1-0-0] [lt=21] gc stale ls task succ [2024-10-16 14:58:17.150974] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286290} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.153037] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044584726528},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286291} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.153598] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1065}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286291} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.153924] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910194286448640},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286292} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.154443] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1057}, to_seq_no={branch:0, seq:6}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286292} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.156994] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21590][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] get number(ret=0, service_type_=0, range=1, base_id=1729061897156793000, start_id=1729061897156793000, end_id=1729061897156793001) [2024-10-16 14:58:17.157255] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1090) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=4] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.0.4.7:2890", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2024-10-16 14:58:17.157269] INFO [PALF] submit_log (palf_handle_impl.cpp:456) [22815][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, append size=1220) [2024-10-16 14:58:17.157470] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2180) [22610][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=3] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"10.0.4.7:2890", has_set_deleted:false}, accum_size=1220) [2024-10-16 14:58:17.158101] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910194349363200},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286294} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.158588] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2270}, to_seq_no={branch:0, seq:1052}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286294} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.158702] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [123287][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044672806912},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286295} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.158887] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1221}, to_seq_no={branch:0, seq:1215}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286295} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.163116] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [24957][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=15] current monitor number(seq_=-1) [2024-10-16 14:58:17.164271] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:520) [21385][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=11] worker thread began to shrink(tenant_->id()=1004, group_id_=1, token=0) [2024-10-16 14:58:17.165397] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044773470208},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286300} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.165689] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1165}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286300} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.168056] INFO [SERVER.OMT] check_worker_count (ob_tenant.cpp:535) [107539][T1004_L0_G1][T1004][Y0-0000000000000000-0-0] [lt=16] worker thread exit(tenant_->id()=1004, workers_.get_size()=2) [2024-10-16 14:58:17.168071] INFO [SHARE] end_run (ob_tenant_base.cpp:332) [107539][T1004_L0_G1][T0][Y0-0000000000000000-0-0] [lt=12] tenant thread end_run(id_=1004, ret=0, thread_count_=191) [2024-10-16 14:58:17.168909] INFO [STORAGE.TRANS] get_tx_ctx_ (ob_trans_ctx_mgr_v4.cpp:495) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=16] transaction statistics(ls_id={id:1001}, total_tx_ctx_count=1) [2024-10-16 14:58:17.171333] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044798636032},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286302} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.171960] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2105}, to_seq_no={branch:0, seq:1090}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286302} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.173964] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [23204][T1003_L0_G5][T1003][YB4A0A000408-00061E9B800D0DBB-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1729061897173940000, start_id=1729061897173940000, end_id=1729061897173940001) [2024-10-16 14:58:17.174183] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26159][T1_L0_G5][T1][YB4A0A000408-00061E9B800D0DB8-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1729061897173940000, start_id=1729061897173940000, end_id=1729061897173940001) [2024-10-16 14:58:17.174296] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061897173940000, start_id=1729061897173940001, end_id=1729061897173940002) [2024-10-16 14:58:17.174364] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=17] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=22, this=0x2b0e2efe3470, extra_info="") [2024-10-16 14:58:17.174378] INFO [LIB] stat (utility.h:1165) [22795][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=0, this=0x2b0e3aed1210, extra_info="id:1") [2024-10-16 14:58:17.174498] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [31511][T1001_L0_G5][T1001][YB4A0A000408-00061E9B800D0DB9-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061897173940000, start_id=1729061897173940000, end_id=1729061897173940001) [2024-10-16 14:58:17.174617] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [26165][T1004_L0_G5][T1004][YB4A0A000408-00061E9B800D0DBC-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1729061897173940000, start_id=1729061897173940000, end_id=1729061897173940001) [2024-10-16 14:58:17.175924] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044836384768},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286304} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.176160] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1061}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286304} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.177576] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061897177136000, start_id=1729061897177136000, end_id=1729061897177136001) [2024-10-16 14:58:17.178715] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044861550592},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286306} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.179069] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1070}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286306} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.180025] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15F6-0-0] [lt=16] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:17.180771] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3197) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D90-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}) [2024-10-16 14:58:17.180792] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3238) [35843][T1004_L0_G0][T1004][YB4A0A000408-00061E9B732E8D90-0-0] [lt=11] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1729061896674311004, v:0}, arg={tenant_id:1004, ls_id:{id:1002}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1002}, cur_readable_scn:{val:1729061896674311004, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"10.0.4.7:2890"}, offline_scn={val:18446744073709551615, v:3}) [2024-10-16 14:58:17.182031] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [21936][T1002_Occam][T1002][YB4A0A000407-00061E9B363A15F8-0-0] [lt=17] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:17.182240] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=1][errcode=0] primary key duplication conflict(key={"BIGINT":654910194663936000},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286307} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.184217] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:5254}, to_seq_no={branch:0, seq:2110}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286307} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.187094] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:241) [22497][T1004_Occam][T1004][YB4A0A000407-00061E9B365A15F2-0-0] [lt=12] for each ls(succ_num=4, ret=0, ret="OB_SUCCESS") [2024-10-16 14:58:17.190309] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22492][T1003_L0_G0][T1003][YB4A0A000408-00061E9B732E8D90-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1729061897189691000, start_id=1729061897189691000, end_id=1729061897189691001) [2024-10-16 14:58:17.192253] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [22455][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1729061897190721000, start_id=1729061897190721000, end_id=1729061897190721001) [2024-10-16 14:58:17.192834] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22233][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2024-10-16 14:58:17.195404] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910194915594240},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286312} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.195752] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1051}, to_seq_no={branch:0, seq:7}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286312} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.197339] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [21895][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=17] get number(ret=0, service_type_=0, range=1, base_id=1729061897196577000, start_id=1729061897196577000, end_id=1729061897196577001) [2024-10-16 14:58:17.199095] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:314) [21592][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=5] ====== traversal_flush timer task ====== [2024-10-16 14:58:17.199218] INFO [MDS] flush (mds_table_mgr.cpp:172) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D8C-0-0] [lt=10] [FREEZE]no valid mds table there, no need do flush(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:0, ordered_tablets:{}, max_consequent_callbacked_scn={val:18446744073709551615, v:3}, *this={this:0x2b0dee994510, is_inited:true, freezing_scn:{val:1729015200303983000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1729015200503866000, v:0}, clog_base_lsn:{lsn:66098196480}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1729061597348854000, v:0}, all_id_meta:{id_meta:[{limited_id:1729061907008855000, latest_log_ts:{val:1729061890536200001, v:0}}, {limited_id:131900001, latest_log_ts:{val:1728972978713650001, v:0}}, {limited_id:1540000001, latest_log_ts:{val:1728992247895678001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}}, switch_epoch:1, log_handler:{role:1, proposal_id:48, palf_env_:0x2b0dde7f0030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2024-10-16 14:58:17.199280] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:349) [21592][T1_Flush][T1][YB4A0A000407-00061E9B37798D8C-0-0] [lt=59] succeed to traversal_flush(ret=0, ls_cnt=1) [2024-10-16 14:58:17.199903] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910194978508800},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286314} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.201439] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2125}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286314} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.201528] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:704) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=61] success to update min reserved snapshot(reserved_snapshot=1729060094447008000, duration=1800, snapshot_gc_ts=1729061894447008000) [2024-10-16 14:58:17.202131] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:188) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=13] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1729060094447008000) [2024-10-16 14:58:17.202144] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:731) [21523][T1_FreInfoReloa][T1][Y0-0000000000000000-0-0] [lt=11] update reserved snapshot finished(cost_ts=619, reserved_snapshot=1729060094447008000) [2024-10-16 14:58:17.206122] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910195167252480},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286317} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.206338] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1088}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286317} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.208128] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:391) [25949][T1002_L0_G5][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1729061897207262000, start_id=1729061897207262001, end_id=1729061897207262002) [2024-10-16 14:58:17.209577] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830044999962624},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286319} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.209816] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286319} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.212215] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654830045100625920},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286321} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.212549] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B802DCF97-0-0] [lt=1] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1115}, to_seq_no={branch:0, seq:1109}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286321} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.212749] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:326) [12153][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=0][errcode=0] primary key duplication conflict(key={"BIGINT":654910195406327808},{"BIGINT":1729058400}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:392286322} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2024-10-16 14:58:17.213095] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [29808][T1002_L0_G0][T1002][YB4A0A000408-00061E9B7E7ECF24-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1046}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1729008952878920 max_table_version=1729008952878920 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:392286322} ls_id=1001 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2024-10-16 14:58:17.215213] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504F-0-0] [lt=7] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2024-10-16 14:58:17.215231] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [22631][T1004_BalanceEx][T1004][YB4A0A000407-00061E9B2AFD504F-0-0] [lt=16] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2024-10-16 14:58:17.218673] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [22793][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")